Accessing dict keys like an attribute

Question

I find it more convenient to access dict keys as obj foo instead of obj  foo    so I wrote this snippet   class AttributeDict dict       def   getattr   self  attr           return self attr      def   setattr   self  attr  value           self attr    value   However  I assume that there must be some reason that Python doesn t provide this functionality out of the box  What would be the caveats and pitfalls of accessing dict keys in this manner

User · Answer

This is what I use  args              batch size   32           workers   4           train dir    train            val dir    val            lr   1e-3           momentum   0 9           weight decay   1e-4       args   namedtuple  Args       join list args keys        args   print  args lr

User · Answer

The best way to do this is  class AttrDict dict       def   init   self   args    kwargs           super AttrDict  self    init    args    kwargs          self   dict     self  Some pros   It actually works  No dictionary class methods are shadowed  e g   keys   work just fine  Unless - of course - you assign some value to them  see below  Attributes and items are always in sync Trying to access non-existent key as an attribute correctly raises AttributeError instead of KeyError Supports  Tab  autocompletion  e g  in jupyter  amp  ipython   Cons   Methods like  keys   will not work just fine if they get overwritten by incoming data Each AttrDict instance actually stores 2 dictionaries  one inherited and another one in   dict   Causes a memory leak in Python  lt  2 7 4   Python3  lt  3 2 3 Pylint goes bananas with E1123 unexpected-keyword-arg  and E1103 maybe-no-member  For the uninitiated it seems like pure magic   A short explanation on how this works  All python objects internally store their attributes in a dictionary that is named   dict    There is no requirement that the internal dictionary   dict   would need to be  quot just a plain dict quot   so we can assign any subclass of dict   to the internal dictionary  In our case we simply assign the AttrDict   instance we are instantiating  as we are in   init     By calling super   s   init     method we made sure that it  already  behaves exactly like a dictionary  since that function calls all the dictionary instantiation code   One reason why Python doesn t provide this functionality out of the box As noted in the  quot cons quot  list  this combines the namespace of stored keys  which may come from arbitrary and or untrusted data   with the namespace of builtin dict method attributes  For example  d   AttrDict   d update   items    quot jacket quot    quot necktie quot    quot trousers quot     for k  v in d items         TypeError   list  object is not callable     print  quot Never reached  quot   Update - 2020 Since this question was asked almost ten years ago  quite a bit has changed in Python itself since then  While this approach is still valid for some cases  e g  legacy projects stuck to older versions of Python and cases where you really need to handle dictionaries with very dynamic string keys - I think that in general the dataclasses introduced in Python 3 7 are the obvious correct solution to vast majority of the use cases of AttrDict

User · Answer

This isn t a  good  answer  but I thought this was nifty  it doesn t handle nested dicts in current form   Simply wrap your dict in a function   def make funcdict d None    kwargs      def funcdict d None    kwargs           if d is not None              funcdict   dict   update d          funcdict   dict   update kwargs          return funcdict   dict       funcdict d    kwargs      return funcdict   Now you have slightly different syntax  To acces the dict items as attributes do f key  To access the dict items  and other dict methods  in the usual manner do f    key   and we can conveniently update the dict by calling f with keyword arguments and or a dictionary  Example  d     name   Henry    age  31  d   make funcdict d   gt  gt  gt  for key in d            print key      age name  gt  gt  gt  print d name     Henry  gt  gt  gt  print d age     31  gt  gt  gt  d   Height   5-11    Job  Carpenter         age   31   name    Henry    Job    Carpenter    Height    5-11     And there it is  I ll be happy if anyone suggests benefits and drawbacks of this method

User · Answer

This doesn t address the original question  but should be useful for people that  like me  end up here when looking for a lib that provides this functionality   Addict it s a great lib for this  https   github com mewwts addict it takes care of many concerns mentioned in previous answers   An example from the docs   body          query              filtered                  query                      match     description    addictive                               filter                      term     created by    Mats                                     With addict   from addict import Dict body   Dict   body query filtered query match description    addictive  body query filtered filter term created by    Mats

User · Answer

How about Prodict  the little Python class that I wrote to rule them all    Plus  you get auto code completion  recursive object instantiations and auto type conversion   You can do exactly what you asked for   p   Prodict   p foo   1 p bar    baz    Example 1  Type hinting  class Country Prodict       name  str     population  int  turkey   Country   turkey name    Turkey  turkey population   79814871     Example 2  Auto type conversion  germany   Country name  Germany   population  82175700   flag colors   black    red    yellow     print germany population     82175700 print type germany population       lt class  int  gt   print germany flag colors       black    red    yellow   print type germany flag colors       lt class  list  gt

User · Answer

The easiest way is to define a class let s call it Namespace  which uses the object dict update   on the dict  Then  the dict will be treated as an object   class Namespace object               helps referencing object in a dictionary as dict key instead of dict  key               def   init   self  adict           self   dict   update adict     Person   Namespace   name    ahmed                         age   30    -- gt  added for edge cls   print Person name

User · Answer

I created this based on the input from this thread  I need to use odict though  so I had to override get and set attr  I think this should work for the majority of special uses   Usage looks like this     Create an ordered dict normally     gt  gt  gt  od   OrderedAttrDict    gt  gt  gt  od  a     1  gt  gt  gt  od  b     2  gt  gt  gt  od OrderedAttrDict    a   1     b   2       Get and set data using attribute access     gt  gt  gt  od a 1  gt  gt  gt  od b   20  gt  gt  gt  od OrderedAttrDict    a   1     b   20       Setting a NEW attribute only creates it on the instance  not the dict     gt  gt  gt  od c   8  gt  gt  gt  od OrderedAttrDict    a   1     b   20     gt  gt  gt  od c 8   The class   class OrderedAttrDict odict OrderedDict               Constructs an odict OrderedDict with attribute access to data       Setting a NEW attribute only creates it on the instance  not the dict      Setting an attribute that is a key in the data will set the dict data but      will not create a new instance attribute             def   getattr   self  attr                       Try to get the data  If attr is not a key  fall-back and get the attr                     if self has key attr               return super OrderedAttrDict  self    getitem   attr          else              return super OrderedAttrDict  self    getattr   attr        def   setattr   self  attr  value                       Try to set the data  If attr is not a key  fall-back and set the attr                     if self has key attr               super OrderedAttrDict  self    setitem   attr  value          else              super OrderedAttrDict  self    setattr   attr  value    This is a pretty cool pattern already mentioned in the thread  but if you just want to take a dict and convert it to an object that works with auto-complete in an IDE  etc   class ObjectFromDict object       def   init   self  d           self   dict     d

User · Answer

Caveat emptor  For some reasons classes like this seem to break the multiprocessing package   I just struggled with this bug for awhile before finding this SO   Finding exception in python multiprocessing

User · Answer

Solution is   DICT RESERVED KEYS   vars dict  keys     class SmartDict dict               A Dict which is accessible via attribute dot notation             def   init   self   args    kwargs                        param args  multiple dicts                       param kwargs  arbitrary keys  value           If   keyerror False   is passed then not found attributes will         always return None                      super SmartDict  self    init             self    keyerror     kwargs pop  keyerror   True           self update arg  for arg in args if isinstance arg  dict           self update kwargs       def   getattr   self  attr           if attr not in DICT RESERVED KEYS              if self    keyerror                    return self attr              else                  return self get attr          return getattr self  attr       def   setattr   self  key  value           if key in DICT RESERVED KEYS              raise AttributeError  You cannot set a reserved name as attribute           self   setitem   key  value       def   copy   self           return self   class   self       def copy self           return self   copy

User · Answer

I found myself wondering what the current state of  dict keys as attr  in the python ecosystem  As several commenters have pointed out  this is probably not something you want to roll your own from scratch  as there are several pitfalls and footguns  some of them very subtle  Also  I would not recommend using Namespace as a base class  I ve been down that road  it isn t pretty    Fortunately  there are several open source packages providing this functionality  ready to pip install  Unfortunately  there are several packages  Here is a synopsis  as of Dec 2019    Contenders  most recent commit to master  commits  contribs coverage      addict    2019-04-28   217   22   100   munch  2019-12-16   160   17       easydict  2018-10-18   51   6       attrdict  2019-02-01   108   5   100   prodict   2019-10-01   65   1         No longer maintained or under-maintained    treedict  2014-03-28   95   2       bunch  2012-03-12   20   2       NeoBunch   I currently recommend munch or addict  They have the most commits  contributors  and releases  suggesting a healthy open-source codebase for each  They have the cleanest-looking readme md  100  coverage  and good looking set of tests    I do not have a dog in this race  for now    besides having rolled my own dict attr code and wasted a ton of time because I was not aware of all these options     I may contribute to addict munch in the future as I would rather see one solid package than a bunch of fragmented ones  If you like them  contribute  In particular  looks like munch could use a codecov badge and addict could use a python version badge    addict pros    recursive initialization  foo a b c    bar    dict-like arguments become addict Dict   addict cons    shadows typing Dict if you from addict import Dict No key checking  Due to allowing recursive init  if you misspell a key  you just create a new attribute  rather than KeyError  thanks  AljoSt    munch pros    unique naming built-in ser de functions for JSON and YAML   munch cons    no recursive init   only can init one attr at a time   Wherein I Editorialize  Many moons ago  when I used text editors to write python  on projects with only myself or one other dev  I liked the style of dict-attrs  the ability to insert keys by just declaring foo bar spam   eggs  Now I work on teams  and use an IDE for everything  and I have drifted away from these sorts of data structures and dynamic typing in general  in favor of static analysis  functional techniques and type hints  I ve started experimenting with this technique  subclassing Pstruct with objects of my own design    class  BasePstruct dict       def   getattr   self  name           if name in self   slots                return self name          return self   getattribute   name       def   setattr   self  key  value           if key in self   slots                self key    value             return         if key in type self    dict                self key    value             return         raise AttributeError               type object      has no attribute       format type self    name    key     class FooPstruct BasePstruct         slots       foo    bar      This gives you an object which still behaves like a dict  but also lets you access keys like attributes  in a much more rigid fashion  The advantage here is I  or the hapless consumers of your code  know exactly what fields can and can t exist  and the IDE can autocomplete fields  Also subclassing vanilla dict means json serialization is easy  I think the next evolution in this idea would be a custom protobuf generator which emits these interfaces  and a nice knock-on is you get cross-language data structures and IPC via gRPC for nearly free   If you do decide to go with attr-dicts  it s essential to document what fields are expected  for your own  and your teammates   sanity    Feel free to edit update this post to keep it recent

User · Answer

You can pull a convenient container class from the standard library   from argparse import Namespace   to avoid having to copy around code bits  No standard dictionary access  but easy to get one back if you really want it  The code in argparse is simple   class Namespace  AttributeHolder          Simple object for storing attributes       Implements equality by attribute names and values  and provides a simple     string representation               def   init   self    kwargs           for name in kwargs              setattr self  name  kwargs name          hash     None      def   eq   self  other           return vars self     vars other       def   ne   self  other           return not  self    other       def   contains   self  key           return key in self   dict

User · Answer

You can have all legal string characters as part of the key if you use array notation  For example  obj        amp

User · Answer

Just to add some variety to the answer  sci-kit learn has this implemented as a Bunch   class Bunch dict                                                                         Scikit Learn s container object                                               Dictionary-like object that exposes its keys as attributes                        gt  gt  gt  b   Bunch a 1  b 2                                                            gt  gt  gt  b  b                                                                         2                                                                                 gt  gt  gt  b b                                                                          2                                                                                 gt  gt  gt  b c   6                                                                       gt  gt  gt  b  c                                                                         6                                                                                                                                                                  def   init   self    kwargs                                                          super Bunch  self    init   kwargs                                            def   setattr   self  key  value                                                     self key    value                                                             def   dir   self                                                                     return self keys                                                              def   getattr   self  key                                                            try                                                                                  return self key                                                              except KeyError                                                                      raise AttributeError key                                                  def   setstate   self  state                                                         pass                          All you need is to get the setattr and getattr methods - the getattr checks for dict keys and the moves on to checking for actual attributes  The setstaet is a fix for fix for pickling unpickling  bunches  - if inerested check https   github com scikit-learn scikit-learn issues 6196

User · Answer

Wherein I Answer the Question That Was Asked  Why doesn t Python offer it out of the box   I suspect that it has to do with the Zen of Python   There should be one -- and preferably only one -- obvious way to do it    This would create two obvious ways to access values from dictionaries  obj  key   and obj key   Caveats and Pitfalls  These include possible lack of clarity and confusion in the code   i e   the following could be confusing to someone else who is going in to maintain your code at a later date  or even to you  if you re not going back into it for awhile   Again  from Zen   Readability counts     gt  gt  gt  KEY    spam   gt  gt  gt  d KEY    1  gt  gt  gt    Several lines of miscellaneous code here        assert d spam    1   If d is instantiated or KEY is defined or d KEY  is assigned far away from where d spam is being used  it can easily lead to confusion about what s being done  since this isn t a commonly-used idiom   I know it would have the potential to confuse me   Additonally  if you change the value of KEY as follows  but miss changing d spam   you now get    gt  gt  gt  KEY    foo   gt  gt  gt  d KEY    1  gt  gt  gt    Several lines of miscellaneous code here        assert d spam    1 Traceback  most recent call last     File   lt stdin gt    line 2  in  lt module gt  AttributeError   C  object has no attribute  spam    IMO  not worth the effort   Other Items  As others have noted  you can use any hashable object  not just a string  as a dict key   For example    gt  gt  gt  d     2  3   True    gt  gt  gt  assert d  2  3   is True  gt  gt  gt     is legal  but   gt  gt  gt  C   type  C    object      2  3   True    gt  gt  gt  d   C    gt  gt  gt  assert d  2  3  is True   File   lt stdin gt    line 1   d  2  3        SyntaxError  invalid syntax  gt  gt  gt  getattr d   2  3   Traceback  most recent call last     File   lt stdin gt    line 1  in  lt module gt  TypeError  getattr    attribute name must be string  gt  gt  gt     is not   This gives you access to the entire range of printable characters or other hashable objects for your dictionary keys  which you do not have when accessing an object attribute   This makes possible such magic as a cached object metaclass  like the recipe from the Python Cookbook  Ch  9    Wherein I Editorialize  I prefer the aesthetics of spam eggs over spam  eggs    I think it looks cleaner   and I really started craving this functionality when I met the namedtuple   But the convenience of being able to do the following trumps it    gt  gt  gt  KEYS    spam eggs ham   gt  gt  gt  VALS    1  2  3   gt  gt  gt  d    k  v for k  v in zip KEYS split       VALS    gt  gt  gt  assert d      spam   1   eggs   2   ham   3   gt  gt  gt    This is a simple example  but I frequently find myself using dicts in different situations than I d use obj key notation  i e   when I need to read prefs in from an XML file    In other cases  where I m tempted to instantiate a dynamic class and slap some attributes on it for aesthetic reasons  I continue to use a dict for consistency in order to enhance readability   I m sure the OP has long-since resolved this to his satisfaction  but if he still wants this functionality  then I suggest he download one of the packages from pypi that provides it    Bunch is the one I m more familiar with   Subclass of dict  so you have all that functionality  AttrDict also looks like it s also pretty good  but I m not as familiar with it and haven t looked through the source in as much detail as I have Bunch  Addict Is actively maintained and provides attr-like access and more  As noted in the comments by Rotareti  Bunch has been deprecated  but there is an active fork called Munch    However  in order to improve readability of his code I strongly recommend that he not mix his notation styles   If he prefers this notation then he should simply instantiate a dynamic object  add his desired attributes to it  and call it a day    gt  gt  gt  C   type  C    object         gt  gt  gt  d   C    gt  gt  gt  d spam   1  gt  gt  gt  d eggs   2  gt  gt  gt  d ham   3  gt  gt  gt  assert d   dict        spam   1   eggs   2   ham   3      Wherein I Update  to Answer a Follow-Up Question in the Comments  In the comments  below   Elmo asks      What if you want to go one deeper    referring to type          While I ve never used this use case  again  I tend to use nested dict  for consistency   the following code works    gt  gt  gt  C   type  C    object         gt  gt  gt  d   C    gt  gt  gt  for x in  spam eggs ham  split            setattr d  x  C            i   1         for y in  one two three  split                setattr getattr d  x   y  i              i    1      gt  gt  gt  assert d spam   dict        one   1   two   2   three   3

User · Answer

It doesn t work in generality   Not all valid dict keys make addressable attributes   the key     So  you ll need to be careful   Python objects are all basically dictionaries   So I doubt there is much performance or other penalty

User · Answer

Here s a short example of immutable records using built-in collections namedtuple   def record name  d       return namedtuple name  d keys      d    and a usage example   rec   record  Model          train op   train op       loss   loss      print rec loss

User · Answer

No need to write your own as setattr   and getattr   already exist   The advantage of class objects probably comes into play in class definition and inheritance

User · Answer

You can use dict to obj https   pypi org project dict-to-obj   It does exactly what you asked for  From dict to obj import DictToObj a      foo   True   b   DictToObj a  b foo True

User · Answer

Let me post another implementation  which builds upon the answer of Kinvais  but integrates ideas from the AttributeDict proposed in http   databio org posts python AttributeDict html   The advantage of this version is that it also works for nested dictionaries   class AttrDict dict               A class to convert a nested Dictionary into an object with key-values     that are accessible using attribute notation  AttrDict attribute  instead of     key notation  Dict  key     This class recursively sets Dicts to objects      allowing you to recurse down nested dicts  like  AttrDict attr attr                 Inspired by        http   stackoverflow com a 14620633 1551810       http   databio org posts python AttributeDict html      def   init   self  iterable    kwargs           super AttrDict  self    init   iterable    kwargs          for key  value in iterable items                if isinstance value  dict                   self   dict   key    AttrDict value              else                  self   dict   key    value

User · Answer

You can do it using this class I just made  With this class you can use the Map object like another dictionary including json serialization  or with the dot notation  I hope help you   class Map dict               Example      m   Map   first name    Eduardo    last name  Pool   age 24  sports   Soccer                def   init   self   args    kwargs           super Map  self    init    args    kwargs          for arg in args              if isinstance arg  dict                   for k  v in arg iteritems                        self k    v          if kwargs              for k  v in kwargs iteritems                    self k    v      def   getattr   self  attr           return self get attr       def   setattr   self  key  value           self   setitem   key  value       def   setitem   self  key  value           super Map  self    setitem   key  value          self   dict   update  key  value        def   delattr   self  item           self   delitem   item       def   delitem   self  key           super Map  self    delitem   key          del self   dict   key    Usage examples   m   Map   first name    Eduardo    last name  Pool   age 24  sports   Soccer      Add new key m new key    Hello world   print m new key print m  new key     Update values m new key    Yay     Or m  new key      Yay     Delete key del m new key   Or del m  new key

User · Answer

tuples can be used dict keys  How would you access tuple in your construct   Also  namedtuple is a convenient structure which can provide values via the attribute access

User · Answer

As noted by Doug there s a Bunch package which you can use to achieve the obj key functionality  Actually there s a newer version called  NeoBunch  It has though a great feature converting your dict to a NeoBunch object through its neobunchify function  I use Mako templates a lot and passing data as NeoBunch objects makes them far more readable  so if you happen to end up using a normal dict in your Python program but want the dot notation in a Mako template you can use it that way   from mako template import Template from neobunch import neobunchify  mako template   Template filename  mako tmpl   strict undefined True  data     tmpl data      key1    value1    key2    value2     with open  out txt    w   as out file      out file write mako template render   neobunchify data      And the Mako template could look like     for d in tmpl data  Column1     Column2   d key1      d key2    endfor

User · Answer

What would be the caveats and pitfalls of accessing dict keys in this manner    As  Henry suggests  one reason dotted-access may not be used in dicts is that it limits dict key names to python-valid variables  thereby restricting all possible names     The following are examples on why dotted-access would not be helpful in general  given a dict  d   Validity  The following attributes would be invalid in Python    d 1 foo                             enumerated names d  bar                              path names d 21 7  d 12 30                     decimals  time d                                   empty strings d john doe  d denny s               spaces  misc punctuation  d 3   x                             expressions     Style  PEP8 conventions would impose a soft constraint on attribute naming   A   Reserved keyword  or builtin function  names    d in d False  d True d max  d min d sum d id      If a function argument s name clashes with a reserved keyword  it is generally better to append a single trailing underscore       B  The case rule on methods and variable names      Variable names follow the same convention as function names    d Firstname d Country      Use the function naming rules  lowercase with words separated by underscores as necessary to improve readability      Sometimes these concerns are raised in libraries like pandas  which permits dotted-access of DataFrame columns by name   The default mechanism to resolve naming restrictions is also array-notation - a string within brackets     If these constraints do not apply to your use case  there are several options on dotted-access data structures

User · Answer

What if you wanted a key which was a method  such as   eq   or   getattr     And you wouldn t be able to have an entry that didn t start with a letter  so using 0343853 as a key is out   And what if you didn t want to use a string

User · Answer

class AttrDict dict         def   init   self              self   dict     self  if   name           main           d   AttrDict        d  ray      hope       d sun    shine    gt  gt  gt  Now we can use this   notation      print d  ray        print d sun

User · Answer

Apparently there is now a library for this - https   pypi python org pypi attrdict - which implements this exact functionality plus recursive merging and json loading  Might be worth a look

User · Answer

From This other SO question there s a great implementation example that simplifies your existing code   How about  class AttributeDict dict         slots               getattr     dict   getitem         setattr     dict   setitem    Much more concise and doesn t leave any room for extra cruft getting into your   getattr   and   setattr   functions in the future

[python] Accessing dict keys like an attribute?

Examples related to python

Examples related to dictionary

Examples related to syntax

Examples related to attributes