In-memory size of a Python structure

Question

Is there a reference for the memory size of Python data stucture on 32- and 64-bit platforms   If not  this would be nice to have it on SO  The more exhaustive the better  So how many bytes are used by the following Python structures  depending on the len and the content type when relevant     int float reference str unicode string tuple list dict set array array numpy array deque new-style classes object old-style classes object     and everything I am forgetting     For containers that keep only references to other objects  we obviously do not want to count the size of the item themselves  since it might be shared    Furthermore  is there a way to get the memory used by an object at runtime  recursively or not

User · Answer

When you use the dir  object   built-in function  you can get the   sizeof   of the built-in function    gt  gt  gt  a   -1  gt  gt  gt  a   sizeof     24

User · Answer

Also you can use guppy module    gt  gt  gt  from guppy import hpy  hp hpy    gt  gt  gt  hp heap   Partition of a set of 25853 objects  Total size   3320992 bytes   Index  Count         Size     Cumulative    Kind  class   dict of class       0  11731  45   929072  28    929072  28 str      1   5832  23   469760  14   1398832  42 tuple      2    324   1   277728   8   1676560  50 dict  no owner       3     70   0   216976   7   1893536  57 dict of module      4    199   1   210856   6   2104392  63 dict of type      5   1627   6   208256   6   2312648  70 types CodeType      6   1592   6   191040   6   2503688  75 function      7    199   1   177008   5   2680696  81 type      8    124   0   135328   4   2816024  85 dict of class      9   1045   4    83600   3   2899624  87   builtin   wrapper descriptor  lt 90 more rows  Type e g     more  to view  gt    And    gt  gt  gt  hp iso 1   1    1    1     1 1   None  Partition of a set of 6 objects  Total size   560 bytes   Index  Count         Size     Cumulative    Kind  class   dict of class       0      1  17      280  50       280  50 dict  no owner       1      1  17      136  24       416  74 list      2      1  17       64  11       480  86 tuple      3      1  17       40   7       520  93 str      4      1  17       24   4       544  97 int      5      1  17       16   3       560 100 types NoneType

User · Answer

I ve been happily using pympler for such tasks  It s compatible with many versions of Python -- the asizeof module in particular goes back to 2 2   For example  using hughdbrown s example but with from pympler import asizeof at the start and print asizeof asizeof v  at the end  I see  system Python 2 5 on MacOSX 10 5      python pymp py  set 120 unicode 32 tuple 32 int 16 decimal 152 float 16 list 40 object 0 dict 144 str 32   Clearly there is some approximation here  but I ve found it very useful for footprint analysis and tuning

User · Answer

These answers all collect shallow size information  I suspect that visitors to this question will end up here looking to answer the question   How big is this complex object in memory    There s a great answer here  https   goshippo com blog measure-real-size-any-python-object   The punchline   import sys  def get size obj  seen None          Recursively finds size of objects        size   sys getsizeof obj      if seen is None          seen   set       obj id   id obj      if obj id in seen          return 0       Important mark as seen  before  entering recursion to gracefully handle       self-referential objects     seen add obj id      if isinstance obj  dict           size    sum  get size v  seen  for v in obj values             size    sum  get size k  seen  for k in obj keys         elif hasattr obj     dict              size    get size obj   dict    seen      elif hasattr obj     iter     and not isinstance obj   str  bytes  bytearray            size    sum  get size i  seen  for i in obj       return size   Used like so   In  1   get size 1  Out 1   24  In  2   get size  1   Out 2   104  In  3   get size   1    Out 3   184   If you want to know Python s memory model more deeply  there s a great article here that has a similar  total size  snippet of code as part of a longer explanation  https   code tutsplus com tutorials understand-how-much-memory-your-python-objects-use--cms-25609

User · Answer

Try memory profiler  memory profiler  Line      Mem usage  Increment   Line Contents                                                     3                            profile      4      5 97 MB    0 00 MB   def my func         5     13 61 MB    7 64 MB       a    1     10    6       6    166 20 MB  152 59 MB       b    2     2   10    7       7     13 61 MB -152 59 MB       del b      8     13 61 MB    0 00 MB       return a

User · Answer

The recommendation from an earlier question on this was to use sys getsizeof    quoting    gt  gt  gt  import sys  gt  gt  gt  x   2  gt  gt  gt  sys getsizeof x  14  gt  gt  gt  sys getsizeof sys getsizeof  32  gt  gt  gt  sys getsizeof  this   38  gt  gt  gt  sys getsizeof  this also   48   You could take this approach    gt  gt  gt  import sys  gt  gt  gt  import decimal  gt  gt  gt    gt  gt  gt  d              int   0           float   0 0           dict   dict             set   set             tuple   tuple             list   list             str    a            unicode   u a            decimal   decimal Decimal 0            object   object           gt  gt  gt  for k  v in sorted d iteritems             print k  sys getsizeof v      decimal 40 dict 140 float 16 int 12 list 36 object 8 set 116 str 25 tuple 28 unicode 28     2012-09-30  python 2 7  linux  32-bit    decimal 36 dict 136 float 16 int 12 list 32 object 8 set 112 str 22 tuple 24 unicode 32   python 3 3  linux  32-bit   decimal 52 dict 144 float 16 int 14 list 32 object 8 set 112 str 26 tuple 24 unicode 26     2016-08-01  OSX  Python 2 7 10  default  Oct 23 2015  19 19 21   GCC 4 2 1 Compatible Apple LLVM 7 0 0  clang-700 0 59 5   on darwin  decimal 80 dict 280 float 24 int 24 list 72 object 16 set 232 str 38 tuple 56 unicode 52

User · Answer

One can also make use of the tracemalloc module from the Python standard library  It seems to work well for objects whose class is implemented in C  unlike Pympler  for instance

[python] In-memory size of a Python structure

Examples related to python

Examples related to memory

Examples related to memory-footprint