Understanding get and set and Python descriptors

Question

I am trying to understand what Python s descriptors are and what they are useful for  I understand how they work  but here are my doubts  Consider the following code   class Celsius object       def   init   self  value 0 0           self value   float value      def   get   self  instance  owner           return self value     def   set   self  instance  value           self value   float value    class Temperature object       celsius   Celsius      Why do I need the descriptor class  What is instance and owner here   in   get     What is the purpose of these parameters  How would I call use this example

User · Answer

I am trying to understand what Python s descriptors are and what they can be useful for    Descriptors are class attributes  like properties or methods  with any of the following special methods      get    non-data descriptor method  for example on a method function    set    data descriptor method  for example on a property instance    delete    data descriptor method    These descriptor objects can be used as attributes on other object class definitions   That is  they live in the   dict   of the class object    Descriptor objects can be used to programmatically manage the results of a dotted lookup  e g  foo descriptor  in a normal expression  an assignment  and even a deletion    Functions methods  bound methods  property  classmethod  and staticmethod all use these special methods to control how they are accessed via the dotted lookup   A data descriptor  like property  can allow for lazy evaluation of attributes based on a simpler state of the object  allowing instances to use less memory than if you precomputed each possible attribute    Another data descriptor  a member descriptor  created by   slots    allow memory savings by allowing the class to store data in a mutable tuple-like datastructure instead of the more flexible but space-consuming   dict     Non-data descriptors  usually instance  class  and static methods  get their implicit first arguments  usually named cls and self  respectively  from their non-data descriptor method    get     Most users of Python need to learn only the simple usage  and have no need to learn or understand the implementation of descriptors further   In Depth  What Are Descriptors   A descriptor is an object with any of the following methods    get      set    or   delete     intended to be used via dotted-lookup as if it were a typical attribute of an instance  For an owner-object  obj instance  with a descriptor object    obj instance descriptor invokes descriptor   get   self  obj instance  owner class  returning a value This is how all methods and the get on a property work  obj instance descriptor   value invokes descriptor   set   self  obj instance  value  returning None This is how the setter on a property works  del obj instance descriptor invokes descriptor   delete   self  obj instance  returning None This is how the deleter on a property works    obj instance is the instance whose class contains the descriptor object s instance  self is the instance of the descriptor  probably just one for the class of the obj instance   To define this with code  an object is a descriptor if the set of its attributes intersects with any of the required attributes   def has descriptor attrs obj       return set     get        set        delete      intersection dir obj    def is descriptor obj          obj can be instance of descriptor or the descriptor class        return bool has descriptor attrs obj     A Data Descriptor has a   set   and or   delete    A Non-Data-Descriptor has neither   set   nor   delete     def has data descriptor attrs obj       return set     set        delete       amp  set dir obj    def is data descriptor obj       return bool has data descriptor attrs obj     Builtin Descriptor Object Examples    classmethod staticmethod property functions in general   Non-Data Descriptors  We can see that classmethod and staticmethod are Non-Data-Descriptors    gt  gt  gt  is descriptor classmethod   is data descriptor classmethod   True  False   gt  gt  gt  is descriptor staticmethod   is data descriptor staticmethod   True  False    Both only have the   get   method    gt  gt  gt  has descriptor attrs classmethod   has descriptor attrs staticmethod   set     get       set     get         Note that all functions are also Non-Data-Descriptors    gt  gt  gt  def foo    pass       gt  gt  gt  is descriptor foo   is data descriptor foo   True  False    Data Descriptor  property  However  property is a Data-Descriptor    gt  gt  gt  is data descriptor property  True  gt  gt  gt  has descriptor attrs property  set     set        get        delete        Dotted Lookup Order  These are important distinctions  as they affect the lookup order for a dotted lookup    obj instance attribute    First the above looks to see if the attribute is a Data-Descriptor on the class of the instance   If not  it looks to see if the attribute is in the obj instance s   dict    then  it finally falls back to a Non-Data-Descriptor     The consequence of this lookup order is that Non-Data-Descriptors like functions methods can be overridden by instances   Recap and Next Steps  We have learned that descriptors are objects with any of   get      set    or   delete    These descriptor objects can be used as attributes on other object class definitions  Now we will look at how they are used  using your code as an example     Analysis of Code from the Question  Here s your code  followed by your questions and answers to each   class Celsius object       def   init   self  value 0 0           self value   float value      def   get   self  instance  owner           return self value     def   set   self  instance  value           self value   float value   class Temperature object       celsius   Celsius           Why do I need the descriptor class       Your descriptor ensures you always have a float for this class attribute of Temperature  and that you can t use del to delete the attribute    gt  gt  gt  t1   Temperature    gt  gt  gt  del t1 celsius Traceback  most recent call last     File   lt stdin gt    line 1  in  lt module gt  AttributeError    delete     Otherwise  your descriptors ignore the owner-class and instances of the owner  instead  storing state in the descriptor  You could just as easily share state across all instances with a simple class attribute  so long as you always set it as a float to the class and never delete it  or are comfortable with users of your code doing so    class Temperature object       celsius   0 0   This gets you exactly the same behavior as your example  see response to question 3 below   but uses a Pythons builtin  property   and would be considered more idiomatic   class Temperature object        celsius   0 0      property     def celsius self           return type self   celsius      celsius setter     def celsius self  value           type self   celsius   float value          What is instance and owner here   in get   What is the purpose of these parameters       instance is the instance of the owner that is calling the descriptor  The owner is the class in which the descriptor object is used to manage access to the data point  See the descriptions of the special methods that define descriptors next to the first paragraph of this answer for more descriptive variable names         How would I call use this example       Here s a demonstration    gt  gt  gt  t1   Temperature    gt  gt  gt  t1 celsius 0 0  gt  gt  gt  t1 celsius   1  gt  gt  gt    gt  gt  gt  t1 celsius 1 0  gt  gt  gt  t2   Temperature    gt  gt  gt  t2 celsius 1 0   You can t delete the attribute    gt  gt  gt  del t2 celsius Traceback  most recent call last     File   lt stdin gt    line 1  in  lt module gt  AttributeError    delete     And you can t assign a variable that can t be converted to a float    gt  gt  gt  t1 celsius    0x02  Traceback  most recent call last     File   lt stdin gt    line 1  in  lt module gt    File   lt stdin gt    line 7  in   set   ValueError  invalid literal for float    0x02   Otherwise  what you have here is a global state for all instances  that is managed by assigning to any instance     The expected way that most experienced Python programmers would accomplish this outcome would be to use the property decorator  which makes use of the same descriptors under the hood  but brings the behavior into the implementation of the owner class  again  as defined above    class Temperature object        celsius   0 0      property     def celsius self           return type self   celsius      celsius setter     def celsius self  value           type self   celsius   float value    Which has the exact same expected behavior of the original piece of code    gt  gt  gt  t1   Temperature    gt  gt  gt  t2   Temperature    gt  gt  gt  t1 celsius 0 0  gt  gt  gt  t1 celsius   1 0  gt  gt  gt  t2 celsius 1 0  gt  gt  gt  del t1 celsius Traceback  most recent call last     File   lt stdin gt    line 1  in  lt module gt  AttributeError  can t delete attribute  gt  gt  gt  t1 celsius    0x02  Traceback  most recent call last     File   lt stdin gt    line 1  in  lt module gt    File   lt stdin gt    line 8  in celsius ValueError  invalid literal for float    0x02   Conclusion  We ve covered the attributes that define descriptors  the difference between data- and non-data-descriptors  builtin objects that use them  and specific questions about use   So again  how would you use the question s example  I hope you wouldn t  I hope you would start with my first suggestion  a simple class attribute  and move on to the second suggestion  the property decorator  if you feel it is necessary

User · Answer

Why do I need the descriptor class    It gives you extra control over how attributes work  If you re used to getters and setters in Java  for example  then it s Python s way of doing that  One advantage is that it looks to users just like an attribute  there s no change in syntax   So you can start with an ordinary attribute and then  when you need to do something fancy  switch to a descriptor   An attribute is just a mutable value  A descriptor lets you execute arbitrary code when reading or setting  or deleting  a value  So you could imagine using it to map an attribute to a field in a database  for example     a kind of ORM   Another use might be refusing to accept a new value by throwing an exception in   set       effectively making the  attribute  read only      What is instance and owner here   in   get     What is the purpose of these parameters    This is pretty subtle  and the reason I am writing a new answer here - I found this question while wondering the same thing and didn t find the existing answer that great    A descriptor is defined on a class  but is typically called from an instance  When it s called from an instance both instance and owner are set  and you can work out owner from instance so it seems kinda pointless   But when called from a class  only owner is set     which is why it s there   This is only needed for   get   because it s the only one that can be called on a class  If you set the class value you set the descriptor itself  Similarly for deletion  Which is why the owner isn t needed there      How would I call use this example    Well  here s a cool trick using similar classes   class Celsius       def   get   self  instance  owner           return 5    instance fahrenheit - 32    9      def   set   self  instance  value           instance fahrenheit   32   9   value   5   class Temperature       celsius   Celsius        def   init   self  initial f           self fahrenheit   initial f   t   Temperature 212  print t celsius  t celsius   0 print t fahrenheit     I m using Python 3  for python 2 you need to make sure those divisions are   5 0 and   9 0   That gives   100 0 32 0   Now there are other  arguably better ways to achieve the same effect in python  e g  if celsius were a property  which is the same basic mechanism but places all the source inside the Temperature class   but that shows what can be done

User · Answer

The descriptor is how Python s property type is implemented  A descriptor simply implements   get      set    etc  and is then added to another class in its definition  as you did above with the Temperature class   For example   temp Temperature   temp celsius  calls celsius   get     Accessing the property you assigned the descriptor to  celsius in the above example  calls the appropriate descriptor method   instance in   get   is the instance of the class  so above    get   would receive temp  while owner is the class with the descriptor  so it would be Temperature    You need to use a descriptor class to encapsulate the logic that powers it  That way  if the descriptor is used to cache some expensive operation  for example   it could store the value on itself and not its class   An article about descriptors can be found here   EDIT  As jchl pointed out in the comments  if you simply try Temperature celsius  instance will be None

User · Answer

Why do I need the descriptor class    Inspired by Fluent Python by Buciano Ramalho  Imaging you have a class like this  class LineItem       price   10 9      weight   2 1      def   init   self  name  price  weight             self name   name           self price   price           self weight   weight  item   LineItem  apple   2 9  2 1  item price   -0 9    it s price is negative  you need to refund to your customer even you delivered the apple    item weight   -0 8   negative weight  it doesn t make sense   We should validate the weight and price in avoid to assign them a negative number  we can write less code if we use descriptor as a proxy as this  class Quantity object         index   0      def   init   self           self   index   self   class     index         self  storage name    quantity     format self   index          self   class     index    1      def   set   self  instance  value           if value  gt  0              setattr instance  self  storage name  value          else             raise ValueError  value should  gt 0       def   get   self  instance  owner           return getattr instance  self  storage name    then define class LineItem like this   class LineItem object        weight   Quantity        price   Quantity         def   init   self  name  weight  price            self name   name          self weight   weight          self price   price   and we can extend the Quantity class to do more common validating

User · Answer

Before going into the details of descriptors it may be important to know how attribute lookup in Python works  This assumes that the class has no metaclass and that it uses the default implementation of   getattribute    both can be used to  customize  the behavior    The best illustration of attribute lookup  in Python 3 x or for new-style classes in Python 2 x  in this case is from Understanding Python metaclasses  ionel s codelog   The image uses   as substitute for  non-customizable attribute lookup    This represents the lookup of an attribute foobar on an instance of Class     Two conditions are important here    If the class of instance has an entry for the attribute name and it has   get   and   set    If the instance has no entry for the attribute name but the class has one and it has   get      That s where descriptors come into it    Data descriptors which have both   get   and   set    Non-data descriptors which only have   get      In both cases the returned value goes through   get   called with the instance as first argument and the class as second argument   The lookup is even more complicated for class attribute lookup  see for example Class attribute lookup  in the above mentioned blog     Let s move to your specific questions      Why do I need the descriptor class    In most cases you don t need to write descriptor classes  However you re probably a very regular end user  For example functions  Functions are descriptors  that s how functions can be used as methods with self implicitly passed as first argument   def test function self       return self  class TestClass object       def test method self                 If you look up test method on an instance you ll get back a  bound method     gt  gt  gt  instance   TestClass    gt  gt  gt  instance test method  lt bound method TestClass test method of  lt   main   TestClass object at     gt  gt    Similarly you could also bind a function by invoking its   get   method manually  not really recommended  just for illustrative purposes     gt  gt  gt  test function   get   instance  TestClass   lt bound method test function of  lt   main   TestClass object at     gt  gt    You can even call this  self-bound method     gt  gt  gt  test function   get   instance  TestClass     lt   main   TestClass at     gt    Note that I did not provide any arguments and the function did return the instance I had bound   Functions are Non-data descriptors   Some built-in examples of a data-descriptor would be property  Neglecting getter  setter  and deleter the property descriptor is  from Descriptor HowTo Guide  Properties     class Property object       def   init   self  fget None  fset None  fdel None  doc None           self fget   fget         self fset   fset         self fdel   fdel         if doc is None and fget is not None              doc   fget   doc           self   doc     doc      def   get   self  obj  objtype None           if obj is None              return self         if self fget is None              raise AttributeError  unreadable attribute           return self fget obj       def   set   self  obj  value           if self fset is None              raise AttributeError  can t set attribute           self fset obj  value       def   delete   self  obj           if self fdel is None              raise AttributeError  can t delete attribute           self fdel obj    Since it s a data descriptor it s invoked whenever you look up the  name  of the property and it simply delegates to the functions decorated with  property   name setter  and  name deleter  if present    There are several other descriptors in the standard library  for example staticmethod  classmethod   The point of descriptors is easy  although you rarely need them   Abstract common code for attribute access  property is an abstraction for instance variable access  function provides an abstraction for methods  staticmethod provides an abstraction for methods that don t need instance access and classmethod provides an abstraction for methods that need class access rather than instance access  this is a bit simplified    Another example would be a class property   One fun example  using   set name   from Python 3 6  could also be a property that only allows a specific type   class TypedProperty object         slots        name     type       def   init   self  typ           self  type   typ      def   get   self  instance  klass None           if instance is None              return self         return instance   dict   self  name       def   set   self  instance  value           if not isinstance value  self  type               raise TypeError f Expected class  self  type   got  type value             instance   dict   self  name    value      def   delete   self  instance           del instance   dict   self  name       def   set name   self  klass  name           self  name   name   Then you can use the descriptor in a class   class Test object       int prop   TypedProperty int    And playing a bit with it    gt  gt  gt  t   Test    gt  gt  gt  t int prop   10  gt  gt  gt  t int prop 10   gt  gt  gt  t int prop   20 0 TypeError  Expected class  lt class  int  gt   got  lt class  float  gt    Or a  lazy property    class LazyProperty object         slots        fget     name       def   init   self  fget           self  fget   fget      def   get   self  instance  klass None           if instance is None              return self         try              return instance   dict   self  name          except KeyError              value   self  fget instance              instance   dict   self  name    value             return value      def   set name   self  klass  name           self  name   name  class Test object        LazyProperty     def lazy self           print  calculating           return 10   gt  gt  gt  t   Test    gt  gt  gt  t lazy calculating 10  gt  gt  gt  t lazy 10   These are cases where moving the logic into a common descriptor might make sense  however one could also solve them  but maybe with repeating some code  with other means      What is instance and owner here   in   get     What is the purpose of these parameters    It depends on how you look up the attribute  If you look up the attribute on an instance then    the second argument is the instance on which you look up the attribute the third argument is the class of the instance   In case you look up the attribute on the class  assuming the descriptor is defined on the class     the second argument is None the third argument is the class where you look up the attribute   So basically the third argument is necessary if you want to customize the behavior when you do class-level look-up  because the instance is None       How would I call use this example    Your example is basically a property that only allows values that can be converted to float and that is shared between all instances of the class  and on the class - although one can only use  read  access on the class otherwise you would replace the descriptor instance     gt  gt  gt  t1   Temperature    gt  gt  gt  t2   Temperature     gt  gt  gt  t1 celsius   20     setting it on one instance  gt  gt  gt  t2 celsius          looking it up on another instance 20 0   gt  gt  gt  Temperature celsius    looking it up on the class 20 0   That s why descriptors generally use the second argument  instance  to store the value to avoid sharing it  However in some cases sharing a value between instances might be desired  although I cannot think of a scenario at this moment   However it makes practically no sense for a celsius property on a temperature class    except maybe as purely academic exercise

User · Answer

I tried  with minor changes as suggested  the code from Andrew Cooke s answer   I am running python 2 7    The code      usr bin env python class Celsius      def   get   self  instance  owner   return 9    instance fahrenheit   32    5 0     def   set   self  instance  value   instance fahrenheit   32   5   value   9 0  class Temperature      def   init   self  initial f   self fahrenheit   initial f     celsius   Celsius    if   name         main          t   Temperature 212      print t celsius      t celsius   0     print t fahrenheit    The result   C  Users gkuhn Desktop gt python test2 py  lt   main   Celsius instance at 0x02E95A80 gt  212   With Python prior to 3  make sure you subclass from object which will make the descriptor work correctly as the get magic does not work for old style classes

User · Answer

You d see https   docs python org 3 howto descriptor html properties  class Property object        Emulate PyProperty Type   in Objects descrobject c       def   init   self  fget None  fset None  fdel None  doc None           self fget   fget         self fset   fset         self fdel   fdel         if doc is None and fget is not None              doc   fget   doc           self   doc     doc      def   get   self  obj  objtype None           if obj is None              return self         if self fget is None              raise AttributeError  unreadable attribute           return self fget obj       def   set   self  obj  value           if self fset is None              raise AttributeError  can t set attribute           self fset obj  value       def   delete   self  obj           if self fdel is None              raise AttributeError  can t delete attribute           self fdel obj       def getter self  fget           return type self  fget  self fset  self fdel  self   doc         def setter self  fset           return type self  self fget  fset  self fdel  self   doc         def deleter self  fdel           return type self  self fget  self fset  fdel  self   doc

[python] Understanding get and set and Python descriptors

I am trying to understand what Python's descriptors are and what they can be useful for.

In Depth: What Are Descriptors?

Builtin Descriptor Object Examples:

Non-Data Descriptors

Data Descriptor, `property`

Dotted Lookup Order

Recap and Next Steps

Analysis of Code from the Question

Conclusion

Examples related to python

Examples related to descriptor

[python] Understanding __get__ and __set__ and Python descriptors