How do I implement getattribute without an infinite recursion error

Question

I want to override access to one variable in a class  but return all others normally   How do I accomplish this with   getattribute     I tried the following  which should also illustrate what I m trying to do  but I get a recursion error   class D object       def   init   self           self test 20         self test2 21     def   getattribute   self name           if name   test               return 0          else              return self   dict   name    gt  gt  gt  print D   test 0 0  gt  gt  gt  print D   test2     RuntimeError  maximum recursion depth exceeded in cmp

User · Accepted Answer

You get a recursion error because your attempt to access the self.__dict__ attribute inside __getattribute__ invokes your __getattribute__ again. If you use object's __getattribute__ instead, it works:

class D(object):
    def __init__(self):
        self.test=20
        self.test2=21
    def __getattribute__(self,name):
        if name=='test':
            return 0.
        else:
            return object.__getattribute__(self, name)

This works because object (in this example) is the base class. By calling the base version of __getattribute__ you avoid the recursive hell you were in before.

Ipython output with code in foo.py:

In [1]: from foo import *

In [2]: d = D()

In [3]: d.test
Out[3]: 0.0

In [4]: d.test2
Out[4]: 21

Update:

There's something in the section titled More attribute access for new-style classes in the current documentation, where they recommend doing exactly this to avoid the infinite recursion.

User · Answer

Python language reference      In order to avoid infinite recursion   in this method  its implementation   should always call the base class   method with the same name to access   any attributes it needs  for example    object   getattribute   self  name     Meaning   def   getattribute   self name                   return self   dict   name    You re calling for an attribute called   dict    Because it s an attribute    getattribute   gets called in search for   dict   which calls   getattribute   which calls     yada yada yada  return  object   getattribute   self  name    Using the base classes   getattribute   helps finding the real attribute

User · Answer

Here is a more reliable version   class D object       def   init   self           self test   20         self test2   21     def   getattribute   self  name           if name     test               return 0          else              return super D  self    getattribute   name    It calls   getattribute   method from parent class  eventually falling back to object   getattribute   method if other ancestors don t override it

User · Answer

Python language reference      In order to avoid infinite recursion   in this method  its implementation   should always call the base class   method with the same name to access   any attributes it needs  for example    object   getattribute   self  name     Meaning   def   getattribute   self name                   return self   dict   name    You re calling for an attribute called   dict    Because it s an attribute    getattribute   gets called in search for   dict   which calls   getattribute   which calls     yada yada yada  return  object   getattribute   self  name    Using the base classes   getattribute   helps finding the real attribute

User · Answer

Actually  I believe you want to use the   getattr   special method instead  Quote from the Python docs     getattr    self  name  Called when an attribute lookup has not found the attribute in the usual places  i e  it is not an instance attribute nor is it found in the class tree for self   name is the attribute name  This method should return the  computed  attribute value or raise an AttributeError exception  Note that if the attribute is found through the normal mechanism    getattr     is not called   This is an intentional asymmetry between   getattr     and   setattr       This is done both for efficiency reasons and because otherwise   setattr     would have no way to access other attributes of the instance  Note that at least for instance variables  you can fake total control by not inserting any values in the instance attribute dictionary  but instead inserting them in another object   See the   getattribute     method below for a way to actually get total control in new-style classes   Note  for this to work  the instance should not have a test attribute  so the line self test 20 should be removed

User · Answer

Are you sure you want to use   getattribute     What are you actually trying to achieve   The easiest way to do what you ask is   class D object       def   init   self           self test   20         self test2   21      test   0   or   class D object       def   init   self           self test   20         self test2   21       property     def test self           return 0   Edit  Note that an instance of D would have different values of test in each case   In the first case d test would be 20  in the second it would be 0   I ll leave it to you to work out why   Edit2  Greg pointed out that example 2 will fail because the property is read only and the   init   method tried to set it to 20   A more complete example for that would be   class D object       def   init   self           self test   20         self test2   21       test   0      def get test self           return self  test      def set test self  value           self  test   value      test   property get test  set test    Obviously  as a class this is almost entirely useless  but it gives you an idea to move on from

User · Answer

Python language reference      In order to avoid infinite recursion   in this method  its implementation   should always call the base class   method with the same name to access   any attributes it needs  for example    object   getattribute   self  name     Meaning   def   getattribute   self name                   return self   dict   name    You re calling for an attribute called   dict    Because it s an attribute    getattribute   gets called in search for   dict   which calls   getattribute   which calls     yada yada yada  return  object   getattribute   self  name    Using the base classes   getattribute   helps finding the real attribute

User · Answer

Are you sure you want to use   getattribute     What are you actually trying to achieve   The easiest way to do what you ask is   class D object       def   init   self           self test   20         self test2   21      test   0   or   class D object       def   init   self           self test   20         self test2   21       property     def test self           return 0   Edit  Note that an instance of D would have different values of test in each case   In the first case d test would be 20  in the second it would be 0   I ll leave it to you to work out why   Edit2  Greg pointed out that example 2 will fail because the property is read only and the   init   method tried to set it to 20   A more complete example for that would be   class D object       def   init   self           self test   20         self test2   21       test   0      def get test self           return self  test      def set test self  value           self  test   value      test   property get test  set test    Obviously  as a class this is almost entirely useless  but it gives you an idea to move on from

User · Answer

Are you sure you want to use   getattribute     What are you actually trying to achieve   The easiest way to do what you ask is   class D object       def   init   self           self test   20         self test2   21      test   0   or   class D object       def   init   self           self test   20         self test2   21       property     def test self           return 0   Edit  Note that an instance of D would have different values of test in each case   In the first case d test would be 20  in the second it would be 0   I ll leave it to you to work out why   Edit2  Greg pointed out that example 2 will fail because the property is read only and the   init   method tried to set it to 20   A more complete example for that would be   class D object       def   init   self           self test   20         self test2   21       test   0      def get test self           return self  test      def set test self  value           self  test   value      test   property get test  set test    Obviously  as a class this is almost entirely useless  but it gives you an idea to move on from

User · Answer

Here is a more reliable version   class D object       def   init   self           self test   20         self test2   21     def   getattribute   self  name           if name     test               return 0          else              return super D  self    getattribute   name    It calls   getattribute   method from parent class  eventually falling back to object   getattribute   method if other ancestors don t override it

User · Answer

Python language reference      In order to avoid infinite recursion   in this method  its implementation   should always call the base class   method with the same name to access   any attributes it needs  for example    object   getattribute   self  name     Meaning   def   getattribute   self name                   return self   dict   name    You re calling for an attribute called   dict    Because it s an attribute    getattribute   gets called in search for   dict   which calls   getattribute   which calls     yada yada yada  return  object   getattribute   self  name    Using the base classes   getattribute   helps finding the real attribute

User · Answer

Actually  I believe you want to use the   getattr   special method instead  Quote from the Python docs     getattr    self  name  Called when an attribute lookup has not found the attribute in the usual places  i e  it is not an instance attribute nor is it found in the class tree for self   name is the attribute name  This method should return the  computed  attribute value or raise an AttributeError exception  Note that if the attribute is found through the normal mechanism    getattr     is not called   This is an intentional asymmetry between   getattr     and   setattr       This is done both for efficiency reasons and because otherwise   setattr     would have no way to access other attributes of the instance  Note that at least for instance variables  you can fake total control by not inserting any values in the instance attribute dictionary  but instead inserting them in another object   See the   getattribute     method below for a way to actually get total control in new-style classes   Note  for this to work  the instance should not have a test attribute  so the line self test 20 should be removed

User · Answer

Actually  I believe you want to use the   getattr   special method instead  Quote from the Python docs     getattr    self  name  Called when an attribute lookup has not found the attribute in the usual places  i e  it is not an instance attribute nor is it found in the class tree for self   name is the attribute name  This method should return the  computed  attribute value or raise an AttributeError exception  Note that if the attribute is found through the normal mechanism    getattr     is not called   This is an intentional asymmetry between   getattr     and   setattr       This is done both for efficiency reasons and because otherwise   setattr     would have no way to access other attributes of the instance  Note that at least for instance variables  you can fake total control by not inserting any values in the instance attribute dictionary  but instead inserting them in another object   See the   getattribute     method below for a way to actually get total control in new-style classes   Note  for this to work  the instance should not have a test attribute  so the line self test 20 should be removed

User · Answer

How is the   getattribute   method used    It is called before the normal dotted lookup  If it raises AttributeError  then we call   getattr     Use of this method is rather rare  There are only two definitions in the standard library     grep -Erl   def   getattribute    self  cpython Lib   grep -v   test   cpython Lib  threading local py cpython Lib importlib util py   Best Practice  The proper way to programmatically control access to a single attribute is with property  Class D should be written as follows  with the setter and deleter optionally to replicate apparent intended behavior    class D object       def   init   self           self test2 21       property     def test self           return 0        test setter     def test self  value              dummy function to avoid AttributeError on setting property          test deleter     def test self              dummy function to avoid AttributeError on deleting property      And usage    gt  gt  gt  o   D    gt  gt  gt  o test 0 0  gt  gt  gt  o test    foo   gt  gt  gt  o test 0 0  gt  gt  gt  del o test  gt  gt  gt  o test 0 0   A property is a data descriptor  thus it is the first thing looked for in the normal dotted lookup algorithm   Options for   getattribute    You several options if you absolutely need to implement lookup for every attribute via   getattribute      raise AttributeError  causing   getattr   to be called  if implemented  return something from it by   using super to call the parent  probably object s  implementation calling   getattr   implementing your own dotted lookup algorithm somehow    For example   class NoisyAttributes object       def   init   self           self test 20         self test2 21     def   getattribute   self  name           print  getting      name          try              return super NoisyAttributes  self    getattribute   name          except AttributeError              print  oh no  AttributeError caught and reraising               raise     def   getattr   self  name              Called if   getattribute   raises AttributeError            return  close but no     name        gt  gt  gt  n   NoisyAttributes    gt  gt  gt  nfoo   n foo getting  foo oh no  AttributeError caught and reraising  gt  gt  gt  nfoo  close but no foo   gt  gt  gt  n test getting  test 20   What you originally wanted   And this example shows how you might do what you originally wanted   class D object       def   init   self           self test 20         self test2 21     def   getattribute   self name           if name   test               return 0          else              return super D  self    getattribute   name    And will behave like this    gt  gt  gt  o   D    gt  gt  gt  o test    foo   gt  gt  gt  o test 0 0  gt  gt  gt  del o test  gt  gt  gt  o test 0 0  gt  gt  gt  del o test  Traceback  most recent call last     File   lt pyshell 216 gt    line 1  in  lt module gt      del o test AttributeError  test   Code review  Your code with comments  You have a dotted lookup on self in   getattribute    This is why you get a recursion error  You could check if name is    dict    and use super to workaround  but that doesn t cover   slots    I ll leave that as an exercise to the reader   class D object       def   init   self           self test 20         self test2 21     def   getattribute   self name           if name   test               return 0          else           v--- Dotted lookup on self in   getattribute               return self   dict   name    gt  gt  gt  print D   test 0 0  gt  gt  gt  print D   test2     RuntimeError  maximum recursion depth exceeded in cmp

User · Answer

How is the   getattribute   method used    It is called before the normal dotted lookup  If it raises AttributeError  then we call   getattr     Use of this method is rather rare  There are only two definitions in the standard library     grep -Erl   def   getattribute    self  cpython Lib   grep -v   test   cpython Lib  threading local py cpython Lib importlib util py   Best Practice  The proper way to programmatically control access to a single attribute is with property  Class D should be written as follows  with the setter and deleter optionally to replicate apparent intended behavior    class D object       def   init   self           self test2 21       property     def test self           return 0        test setter     def test self  value              dummy function to avoid AttributeError on setting property          test deleter     def test self              dummy function to avoid AttributeError on deleting property      And usage    gt  gt  gt  o   D    gt  gt  gt  o test 0 0  gt  gt  gt  o test    foo   gt  gt  gt  o test 0 0  gt  gt  gt  del o test  gt  gt  gt  o test 0 0   A property is a data descriptor  thus it is the first thing looked for in the normal dotted lookup algorithm   Options for   getattribute    You several options if you absolutely need to implement lookup for every attribute via   getattribute      raise AttributeError  causing   getattr   to be called  if implemented  return something from it by   using super to call the parent  probably object s  implementation calling   getattr   implementing your own dotted lookup algorithm somehow    For example   class NoisyAttributes object       def   init   self           self test 20         self test2 21     def   getattribute   self  name           print  getting      name          try              return super NoisyAttributes  self    getattribute   name          except AttributeError              print  oh no  AttributeError caught and reraising               raise     def   getattr   self  name              Called if   getattribute   raises AttributeError            return  close but no     name        gt  gt  gt  n   NoisyAttributes    gt  gt  gt  nfoo   n foo getting  foo oh no  AttributeError caught and reraising  gt  gt  gt  nfoo  close but no foo   gt  gt  gt  n test getting  test 20   What you originally wanted   And this example shows how you might do what you originally wanted   class D object       def   init   self           self test 20         self test2 21     def   getattribute   self name           if name   test               return 0          else              return super D  self    getattribute   name    And will behave like this    gt  gt  gt  o   D    gt  gt  gt  o test    foo   gt  gt  gt  o test 0 0  gt  gt  gt  del o test  gt  gt  gt  o test 0 0  gt  gt  gt  del o test  Traceback  most recent call last     File   lt pyshell 216 gt    line 1  in  lt module gt      del o test AttributeError  test   Code review  Your code with comments  You have a dotted lookup on self in   getattribute    This is why you get a recursion error  You could check if name is    dict    and use super to workaround  but that doesn t cover   slots    I ll leave that as an exercise to the reader   class D object       def   init   self           self test 20         self test2 21     def   getattribute   self name           if name   test               return 0          else           v--- Dotted lookup on self in   getattribute               return self   dict   name    gt  gt  gt  print D   test 0 0  gt  gt  gt  print D   test2     RuntimeError  maximum recursion depth exceeded in cmp

User · Answer

Are you sure you want to use   getattribute     What are you actually trying to achieve   The easiest way to do what you ask is   class D object       def   init   self           self test   20         self test2   21      test   0   or   class D object       def   init   self           self test   20         self test2   21       property     def test self           return 0   Edit  Note that an instance of D would have different values of test in each case   In the first case d test would be 20  in the second it would be 0   I ll leave it to you to work out why   Edit2  Greg pointed out that example 2 will fail because the property is read only and the   init   method tried to set it to 20   A more complete example for that would be   class D object       def   init   self           self test   20         self test2   21       test   0      def get test self           return self  test      def set test self  value           self  test   value      test   property get test  set test    Obviously  as a class this is almost entirely useless  but it gives you an idea to move on from

User · Answer

Actually  I believe you want to use the   getattr   special method instead  Quote from the Python docs     getattr    self  name  Called when an attribute lookup has not found the attribute in the usual places  i e  it is not an instance attribute nor is it found in the class tree for self   name is the attribute name  This method should return the  computed  attribute value or raise an AttributeError exception  Note that if the attribute is found through the normal mechanism    getattr     is not called   This is an intentional asymmetry between   getattr     and   setattr       This is done both for efficiency reasons and because otherwise   setattr     would have no way to access other attributes of the instance  Note that at least for instance variables  you can fake total control by not inserting any values in the instance attribute dictionary  but instead inserting them in another object   See the   getattribute     method below for a way to actually get total control in new-style classes   Note  for this to work  the instance should not have a test attribute  so the line self test 20 should be removed

[python] How do I implement getattribute without an infinite recursion error?

Examples related to python

Examples related to class

Examples related to oop

Examples related to recursion

Examples related to getattr

[python] How do I implement __getattribute__ without an infinite recursion error?