Calculating a directory s size using Python

Question

Before I re-invent this particular wheel  has anybody got a nice routine for calculating the size of a directory using Python  It would be very nice if the routine would format the size nicely in Mb Gb etc

User · Answer

One-liner you say    Here is a one liner   sum  sum map lambda fname  os path getsize os path join directory  fname    files   for directory  folders  files in os walk path      Although I would probably split it out and it performs no checks   To convert to kb see Reusable library to get human readable version of file size  and work it in

User · Answer

A little late to the party but in one line provided that you have glob2 and humanize installed  Note that in Python 3  the default iglob has a recursive mode  How to modify the code for Python 3 is left as a trivial exercise for the reader    gt  gt  gt  import os  gt  gt  gt  from humanize import naturalsize  gt  gt  gt  from glob2 import iglob  gt  gt  gt  naturalsize sum os path getsize x  for x in iglob   var          546 2 MB

User · Answer

Using pathlib I came up this one-liner to get the size of a folder   sum file stat   st size for file in Path folder  rglob         And this is what I came up with for a nicely formatted output   from pathlib import Path   def get folder size folder       return ByteSize sum file stat   st size for file in Path folder  rglob          class ByteSize int         kB   1024      suffixes    B    kB    MB    GB    PB       def   new   cls   args    kwargs           return super     new   cls   args    kwargs       def   init   self   args    kwargs           self bytes   self B   int self          self kilobytes   self kB   self   self  kB  1         self megabytes   self MB   self   self  kB  2         self gigabytes   self GB   self   self  kB  3         self petabytes   self PB   self   self  kB  4          suffixes  last   self  suffixes         suffix   next               suffix             for suffix in suffixes             if 1  lt  getattr self  suffix   lt  self  kB            last          self readable   suffix  getattr self  suffix           super     init          def   str   self           return self   format     2f        def   repr   self           return          format self   class     name    super     repr           def   format   self  format spec           suffix  val   self readable         return   val  fmt    suf   format val val  fmt format spec  suf suffix       def   sub   self  other           return self   class   super     sub   other        def   add   self  other           return self   class   super     add   other        def   mul   self  other           return self   class   super     mul   other        def   rsub   self  other           return self   class   super     sub   other        def   radd   self  other           return self   class   super     add   other        def   rmul   self  other           return self   class   super     rmul   other        Usage    gt  gt  gt  size   get folder size  c  users tdavis downloads    gt  gt  gt  print size  5 81 GB  gt  gt  gt  size GB 5 810891855508089  gt  gt  gt  size gigabytes 5 810891855508089  gt  gt  gt  size PB 0 005674699077644618  gt  gt  gt  size MB 5950 353260040283  gt  gt  gt  size ByteSize 6239397620    I also came across this question  which has some more compact and probably more performant strategies for printing file sizes

User · Answer

def recursive dir size path       size   0      for x in os listdir path           if not os path isdir os path join path x                size    os stat os path join path x   st size         else              size    recursive dir size os path join path x        return size   I wrote this function which gives me accurate overall size of a directory  i tried other for loop solutions with os walk but i don t know why the end result was always less than the actual size  on ubuntu 18 env   I must have done something wrong but who cares wrote this one works perfectly fine

User · Answer

Here is a recursive function  it recursively sums up the size of all subfolders and their respective files  which returns exactly the same bytes as when running  du -sb    in linux  where the     means  the current folder     import os  def getFolderSize folder       total size   os path getsize folder      for item in os listdir folder           itempath   os path join folder  item          if os path isfile itempath               total size    os path getsize itempath          elif os path isdir itempath               total size    getFolderSize itempath      return total size  print  Size      str getFolderSize

User · Answer

A solution that works on Python 3 6 using pathlib   from pathlib import Path  sum  f stat   st size for f in Path  path   glob

User · Answer

use library sh  the module du does it    pip install sh  import sh print  sh du  -s          91154728            if you want to pass asterix  use glob as described here   to convert the values in human readables  use humanize   pip install humanize  import humanize print  humanize naturalsize  91157384     91 2 MB

User · Answer

du does not follow symlinks by default  No answer here make use of follow symlinks False   Here is an implementation which follows default behavior of du   def du path  - gt  int      total   0     for entry in os scandir path           if entry is file follow symlinks False               total    entry stat   st size         elif entry is dir follow symlinks False               total    du entry path      return total   Test   class Test unittest TestCase       def test du self           root     tmp du test          subprocess run   rm    -rf   root           test utils mkdir root          test utils create file root   A    1M           test utils create file root   B    1M           sub       join  root   sub            test utils mkdir sub          test utils create file sub   C    1M           test utils create file sub   D    1M           subprocess run   ln    -s     tmp       join  root   link                self assertEqual 4  lt  lt  20  util du root

User · Answer

You can do something like this     import commands    size   commands getoutput  du -sh  path    split   0    in this case I have not tested the result before returning it  if you want you can check it with commands getstatusoutput

User · Answer

Python 3 5 recursive folder size using os scandir  def folder size path           total   0     for entry in os scandir path           if entry is file                total    entry stat   st size         elif entry is dir                total    folder size entry path      return total

User · Answer

If you are in Windows OS you can do   install the module pywin32 by launching   pip install pywin32  and then coding the following   import win32com client as com  def get folder size path      try         fso   com Dispatch  Scripting FileSystemObject          folder   fso GetFolder path         size   str round folder Size   1048576          print  Size      size     MB      except Exception as e         print  Error -- gt      str e

User · Answer

monknut answer is good but it fails on broken symlink  so you also have to check if this path really exists  if os path exists fp       total size    os stat fp  st size

User · Answer

For what it s worth    the tree command does all of this for free   tree -h --du  path to dir    files and dirs tree -h -d --du  path to dir    dirs only   I love Python  but by far the simplest solution to the problem requires no new code

User · Answer

It is handy   import os import stat  size   0 path       def calculate path os environ  SYSTEMROOT         global size  path      size   0     path    path      for x  y  z in os walk path           for i in z              size    os path getsize x   os sep   i   def cevir x       global path      print path   x   Byte       print path   x 1024   Kilobyte       print path   x 1048576   Megabyte       print path   x 1073741824   Gigabyte    calculate  C  Users Jundullah Desktop   cevir size   Output  C  Users Jundullah Desktop 87874712211 Byte C  Users Jundullah Desktop 85815148 64355469 Kilobyte C  Users Jundullah Desktop 83803 85609722137 Megabyte C  Users Jundullah Desktop 81 83970321994275 Gigabyte

User · Answer

for python3 5   from pathlib import Path  def get size path       return sum p stat   st size for p in Path path  rglob

User · Answer

Get directory size  Properties of the solution    returns both  the apparent size  number of bytes in the file  and the actual disk space the files uses  counts hard linked files only once counts symlinks the same way du does does not use recursion uses st st blocks for disk space used  thus works only on Unix-like systems   The code   import os   def du path       if os path islink path           return  os lstat path  st size  0      if os path isfile path           st   os lstat path          return  st st size  st st blocks   512      apparent total bytes   0     total bytes   0     have          for dirpath  dirnames  filenames in os walk path           apparent total bytes    os lstat dirpath  st size         total bytes    os lstat dirpath  st blocks   512         for f in filenames              fp   os path join dirpath  f              if os path islink fp                   apparent total bytes    os lstat fp  st size                 continue             st   os lstat fp              if st st ino in have                  continue    skip hardlinks which were already counted             have append st st ino              apparent total bytes    st st size             total bytes    st st blocks   512         for d in dirnames              dp   os path join dirpath  d              if os path islink dp                   apparent total bytes    os lstat dp  st size     return  apparent total bytes  total bytes    Example usage    gt  gt  gt  du   lib    236425839  244363264     du -sb  lib 236425839    lib   du -sB1  lib 244363264    lib     Human readable file size  Properties of the solution    Supports up to Yottabytes Supports SI Units or IEC Units Support custom suffixes   The code   def humanized size num  suffix  B   si False       if si          units        K   M   G   T   P   E   Z           last unit    Y          div   1000 0     else          units        Ki   Mi   Gi   Ti   Pi   Ei   Zi           last unit    Yi          div   1024 0     for unit in units          if abs num   lt  div              return   3 1f s s     num  unit  suffix          num    div     return    1f s s     num  last unit  suffix    Example usage    gt  gt  gt  humanized size 236425839   225 5MiB   gt  gt  gt  humanized size 236425839  si True   236 4MB   gt  gt  gt  humanized size 236425839  si True  suffix      236 4M

User · Answer

This walks all sub-directories  summing file sizes  import os  def get size start path             total size   0     for dirpath  dirnames  filenames in os walk start path           for f in filenames              fp   os path join dirpath  f                skip if it is symbolic link             if not os path islink fp                   total size    os path getsize fp       return total size  print get size     bytes    And a oneliner for fun using os listdir  Does not include sub-directories   import os sum os path getsize f  for f in os listdir      if os path isfile f    Reference   os path getsize - Gives the size in bytes os walk os path islink  Updated To use os path getsize  this is clearer than using the os stat   st size method  Thanks to ghostdog74 for pointing this out  os stat - st size Gives the size in bytes   Can also be used to get file size and other file related information  import os  nbytes   sum d stat   st size for d in os scandir      if d is file     Update 2018 If you use Python 3 4 or previous then you may consider using the more efficient walk method provided by the third-party scandir package  In Python 3 5 and later  this package has been incorporated into the standard library and os walk has received the corresponding increase in performance  Update 2019 Recently I ve been using pathlib more and more  here s a pathlib solution  from pathlib import Path  root directory   Path      sum f stat   st size for f in root directory glob         if f is file

User · Answer

Chris  answer is good but could be made more idiomatic by using a set to check for seen directories  which also avoids using an exception for control flow   def directory size path       total size   0     seen   set        for dirpath  dirnames  filenames in os walk path           for f in filenames              fp   os path join dirpath  f               try                  stat   os stat fp              except OSError                  continue              if stat st ino in seen                  continue              seen add stat st ino               total size    stat st size      return total size    size in bytes

User · Answer

I m a little late  and new  here but I chose to use the subprocess module and the  du  command line with Linux to retrieve an accurate value for folder size in MB  I had to use if and elif for root folder because otherwise subprocess raises error due to non-zero value returned   import subprocess import os      get folder size   def get size self  path       if os path exists path  and path                 cmd   str subprocess check output   sudo    du    -s   path                  replace  b         replace           split    t   0          return float cmd    1000000     elif os path exists path  and path                 cmd   str subprocess getoutput   sudo du -s                      replace  b         replace           split   n           val   cmd len cmd  - 1  replace          replace                  return float val    1000000     else  raise ValueError

User · Answer

a recursive one-liner   def getFolderSize p      from functools import partial    prepend   partial os path join  p     return sum   os path getsize f  if os path isfile f  else getFolderSize f   for f in map prepend  os listdir p

User · Answer

Admittedly  this is kind of hackish and only works on Unix Linux   It matches du -sb   because in effect this is a Python bash wrapper that runs the du -sb   command   import subprocess  def system command cmd           Function executes cmd parameter as a bash command         p   subprocess Popen cmd                           stdout subprocess PIPE                           stderr subprocess PIPE                           shell True      stdout  stderr   p communicate       return stdout  stderr  size   int system command  du -sb      0  split   0

User · Answer

I m using python 2 7 13 with scandir and here s my one-liner recursive function to get the total size of a folder   from scandir import scandir def getTotFldrSize path       return sum  s stat follow symlinks False  st size for s in scandir path  if s is file follow symlinks False              sum  getTotFldrSize s path  for s in scandir path  if s is dir follow symlinks False      gt  gt  gt  print getTotFldrSize      1203245680   https   pypi python org pypi scandir

User · Answer

Here is a one liner that does it recursively  recursive option available as of Python 3 5    import os import glob print sum os path getsize f  for f in glob glob       recursive True  if os path isfile f    1024 1024

User · Answer

import os def get size path   os getcwd         print  Calculating Size    path      total size   0      if path is directory--     if os path isdir path         print  Path type   Directory Folder         for dirpath  dirnames  filenames in os walk path             for f in filenames                fp   os path join dirpath  f                  skip if it is symbolic link               if not os path islink fp                     total size    os path getsize fp       if path is a file---     elif os path isfile path         print  Path type   File         total size os path getsize path      else        print  Path Type   Special File  Socket  FIFO  Device File           total size 0     bytesize total size     print bytesize   bytes       print bytesize  1024    kilobytes       print bytesize  1024 1024    megabytes       print bytesize  1024 1024 1024    gegabytes       return total size   x get size   content examples     I m sure this helps  For folders and files as well

User · Answer

The accepted answer doesn t take into account hard or soft links  and would count those files twice   You d want to keep track of which inodes you ve seen  and not add the size for those files   import os def get size start path           total size   0     seen          for dirpath  dirnames  filenames in os walk start path           for f in filenames              fp   os path join dirpath  f              try                  stat   os stat fp              except OSError                  continue              try                  seen stat st ino              except KeyError                  seen stat st ino    True             else                  continue              total size    stat st size      return total size  print get size

User · Answer

This script tells you which file is the biggest in the CWD and also tells you in which folder the file is  This script works for me on win8 and python 3 3 3 shell  import os  folder os cwd    number 0 string     for root  dirs  files in os walk folder       for file in files          pathname os path join root file            print  pathname            print  os path getsize pathname  1024 1024          if number  lt  os path getsize pathname               number   os path getsize pathname              string pathname             print      print  string  print    print  number  print   Number in bytes

User · Answer

for getting the size of one file  there is os path getsize     gt  gt  gt  import os  gt  gt  gt  os path getsize   path file   35L   its reported in bytes

User · Answer

import os  def get size path       total size   0     for dirpath  dirnames  filenames in os walk path           for f in filenames              if os path exists fp                   fp   os path join dirpath  f                  total size    os path getsize fp       return total size     in megabytes   Thanks monkut  amp  troex

User · Answer

When size of the sub-directories is computed  it should update its parent s folder size and this will go on till it reaches the root parent   The following function computes the size of the folder and all its sub-folders    import os  def folder size path       parent         path to parent path mapper     folder size         storing the size of directories     folder   os path realpath path       for root     filenames in os walk folder           if root    folder              parent root    -1    the root folder will not have any parent             folder size root    0 0    intializing the size to 0          elif root not in parent              immediate parent path   os path dirname root     extract the immediate parent of the subdirectory             parent root    immediate parent path    store the parent of the subdirectory             folder size root    0 0    initialize the size to 0          total size   0         for filename in filenames              filepath   os path join root  filename              total size    os stat filepath  st size    computing the size of the files under the directory         folder size root    total size    store the updated size          temp path   root    for subdirectories  we need to update the size of the parent till the root parent         while parent temp path     -1              folder size parent temp path      total size             temp path   parent temp path       return folder size folder  1000000 0

User · Answer

Python 3 6  recursive folder file size using os scandir  As powerful as in the answer by   blakev  but shorter and in EAFP python style   import os  def size path     follow symlinks False       try          with os scandir path  as it              return sum size entry  follow symlinks follow symlinks  for entry in it      except NotADirectoryError          return os stat path  follow symlinks follow symlinks  st size

User · Answer

Some of the approaches suggested so far implement a recursion  others employ a shell or will not produce neatly formatted results  When your code is one-off for Linux platforms  you can get formatting as usual  recursion included  as a one-liner  Except for the print in the last line  it will work for current versions of python2 and python3    du py -----    usr bin python3 import subprocess  def du path          disk usage in human readable format  e g   2 1GB          return subprocess check output   du   -sh   path   split   0  decode  utf-8    if   name         main         print du         is simple  efficient and will work for files and multilevel directories     chmod 750 du py     du py 2 9M

User · Answer

The following script prints directory size of all sub-directories for the specified directory  It also tries to benefit  if possible  from caching the calls of a recursive functions  If an argument is omitted  the script will work in the current directory  The output is sorted by the directory size from biggest to smallest ones  So you can adapt it for your needs   PS i ve used recipe 578019 for showing directory size in human-friendly format  http   code activestate com recipes 578019    from   future   import print function import os import sys import operator  def null decorator ob       return ob  if sys version info  gt    3 2 0       import functools     my cache decorator   functools lru cache maxsize 4096  else      my cache decorator   null decorator  start dir   os path normpath os path abspath sys argv 1    if len sys argv   gt  1 else       my cache decorator def get dir size start path             total size   0     if  scandir  in dir os             using fast  os scandir  method  new in version 3 5          for entry in os scandir start path               if entry is dir follow symlinks   False                   total size    get dir size entry path              elif entry is file follow symlinks   False                   total size    entry stat   st size     else            using slow  but compatible  os listdir  method         for entry in os listdir start path               full path   os path abspath os path join start path  entry               if os path isdir full path                   total size    get dir size full path              elif os path isfile full path                   total size    os path getsize full path      return total size  def get dir size walk start path             total size   0     for dirpath  dirnames  filenames in os walk start path           for f in filenames              fp   os path join dirpath  f              total size    os path getsize fp      return total size  def bytes2human n  format    value  0f  symbol s   symbols  customary                 c  http   code activestate com recipes 578019       Convert n bytes into a human readable string based on format      symbols can be either  customary    customary ext    iec  or  iec ext       see  http   goo gl kTQMs         gt  gt  gt  bytes2human 0         0 0 B         gt  gt  gt  bytes2human 0 9         0 0 B         gt  gt  gt  bytes2human 1         1 0 B         gt  gt  gt  bytes2human 1 9         1 0 B         gt  gt  gt  bytes2human 1024         1 0 K         gt  gt  gt  bytes2human 1048576         1 0 M         gt  gt  gt  bytes2human 1099511627776127398123789121         909 5 Y          gt  gt  gt  bytes2human 9856  symbols  customary          9 6 K         gt  gt  gt  bytes2human 9856  symbols  customary ext          9 6 kilo         gt  gt  gt  bytes2human 9856  symbols  iec          9 6 Ki         gt  gt  gt  bytes2human 9856  symbols  iec ext          9 6 kibi          gt  gt  gt  bytes2human 10000     value  1f   symbol s sec          9 8 K sec          gt  gt  gt    precision can be adjusted by playing with  f operator        gt  gt  gt  bytes2human 10000  format    value  5f   symbol s          9 76562 K              SYMBOLS              customary          B    K    M    G    T    P    E    Z    Y             customary ext      byte    kilo    mega    giga    tera    peta    exa                               zetta    iotta             iec                Bi    Ki    Mi    Gi    Ti    Pi    Ei    Zi    Yi             iec ext            byte    kibi    mebi    gibi    tebi    pebi    exbi                               zebi    yobi              n   int n      if n  lt  0          raise ValueError  n  lt  0       symbols   SYMBOLS symbols      prefix          for i  s in enumerate symbols 1             prefix s    1  lt  lt   i 1  10     for symbol in reversed symbols 1             if n  gt   prefix symbol               value   float n    prefix symbol              return format   locals       return format   dict symbol symbols 0   value n                                                                         main                                                                     if   name         main         dir tree              version  that uses  slow   os walk method       get size   get dir size walk         this recursive version can benefit from caching the function calls  functools lru cache      get size   get dir size      for root  dirs  files in os walk start dir           for d in dirs              dir path   os path join root  d              if os path isdir dir path                   dir tree dir path    get size dir path       for d  size in sorted dir tree items    key operator itemgetter 1   reverse True           print   s t s    bytes2human size  format    value  2f  symbol s    d        print  -    80      if sys version info  gt    3 2 0           print get dir size cache info      Sample output   37 61M    subdir b 2 18M     subdir a 2 17M     subdir a subdir a 2 4 41K     subdir a subdir a 1 ---------------------------------------------------------- CacheInfo hits 2  misses 4  maxsize 4096  currsize 4    EDIT  moved null decorator above  as user2233949 recommended

User · Answer

For the second part of the question  def human size        B    B      KB    KB       MB    MB      GB    GB      TB    TB      UNITS    B  KB  MB  GB  TB      HUMANFMT     f  s      HUMANRADIX   1024       for u in UNITS  -1           if size  lt  HUMANRADIX   return HUMANFMT    size  u          size    HUMANRADIX      return HUMANFMT    size   UNITS -1

[python] Calculating a directory's size using Python?

Examples related to python

Examples related to directory