Total memory used by Python process

Question

Is there a way for a Python program to determine how much memory it s currently using  I ve seen discussions about memory usage for a single object  but what I need is total memory usage for the process  so that I can determine when it s necessary to start discarding cached data

User · Answer

For Unix based systems  Linux  Mac OS X  Solaris   you can use the getrusage   function from the standard library module resource  The resulting object has the attribute ru maxrss  which gives the peak memory usage for the calling process    gt  gt  gt  resource getrusage resource RUSAGE SELF  ru maxrss 2656    peak memory usage  kilobytes on Linux  bytes on OS X    The Python docs don t make note of the units  Refer to your specific system s man getrusage 2 page to check the unit for the value  On Ubuntu 18 04  the unit is noted as kilobytes  On Mac OS X  it s bytes   The getrusage   function can also be given resource RUSAGE CHILDREN to get the usage for child processes  and  on some systems  resource RUSAGE BOTH for total  self and child  process usage   If you care only about Linux  you can alternatively read the  proc self status or  proc self statm file as described in other answers for this question and this one too

User · Answer

Here is a useful solution that works for various operating systems  including Linux  Windows  etc   import os  psutil process   psutil Process os getpid    print process memory info   rss     in bytes   With Python 2 7 and psutil 5 6 3  the last line should be print process memory info   0    instead  there was a change in the API later   Note   do pip install psutil if it is not installed yet  handy one-liner if you quickly want to know how many MB your process takes  import os  psutil  print psutil Process os getpid    memory info   rss   1024    2

User · Answer

I like it  thank you for  bayer  I get a specific process count tool  now     Megabyte    ps aux   grep python   awk   sum sum  6   END  print sum 1024   MB    87 9492 MB    Byte    ps aux   grep python   awk   sum sum  6   END  print sum   KB    90064 KB   Attach my process list     ps aux    grep python root       943  0 0  0 1  53252  9524          Ss   Aug19  52 01  usr bin python  usr local bin beaver -c  etc beaver beaver conf -l  var log beaver log -P  var run beaver pid root       950  0 6  0 4 299680 34220          Sl   Aug19 568 52  usr bin python  usr local bin beaver -c  etc beaver beaver conf -l  var log beaver log -P  var run beaver pid root      3803  0 2  0 4 315692 36576          S    12 43   0 54  usr bin python  usr local bin beaver -c  etc beaver beaver conf -l  var log beaver log -P  var run beaver pid jonny    23325  0 0  0 1  47460  9076 pts 0    S    17 40   0 00 python jonny    24651  0 0  0 0  13076   924 pts 4    S    18 06   0 00 grep python   Reference   memory - Linux  find out what process is using all the RAM  - Super User Total memory used by Python process  - Stack Overflow linux - ps aux output meaning - Super User

User · Answer

For Python 3 6 and psutil 5 4 5 it is easier to use memory percent   function listed here   import os import psutil process   psutil Process os getpid    print process memory percent

User · Answer

Using sh and os to get into python bayer s answer   float sh awk sh ps  u   -p  os getpid      sum sum  6   END  print sum 1024       Answer is in megabytes

User · Answer

import os  win32api  win32con  win32process han   win32api OpenProcess win32con PROCESS QUERY INFORMATION win32con PROCESS VM READ  0  os getpid    process memory   int win32process GetProcessMemoryInfo han   WorkingSetSize

User · Answer

On unix  you can use the ps tool to monitor it     ps u -p 1347   awk   sum sum  6   END  print sum 1024     where 1347 is some process id  Also  the result is in MB

User · Answer

Even easier to use than  proc self status   proc self statm  It s just a space delimited list of several statistics  I haven t been able to tell if both files are always present       proc  pid  statm      Provides information about memory usage  measured in pages                  The columns are          size        1  total program size                             same as VmSize in  proc  pid  status    resident    2  resident set size                             same as VmRSS in  proc  pid  status    shared      3  number of resident shared pages  i e   backed by a file                              same as RssFile RssShmem in  proc  pid  status    text        4  text  code    lib         5  library  unused since Linux 2 6  always 0    data        6  data   stack   dt          7  dirty pages  unused since Linux 2 6  always 0       Here s a simple example   from pathlib import Path from resource import getpagesize  PAGESIZE   getpagesize   PATH   Path   proc self statm     def get resident set size   - gt  int         Return the current resident set size in bytes           statm columns are  size resident shared text lib data dt     statm   PATH read text       fields   statm split       return int fields 1     PAGESIZE   data      start memory   get resident set size   for   in range 10       data append  X    100000      print get resident set size   - start memory    That produces a list that looks something like this   0 0 368640 368640 368640 638976 638976 909312 909312 909312   You can see that it jumps by about 300 000 bytes after roughly 3 allocations of 100 000 bytes

User · Answer

On Windows  you can use WMI  home page  cheeseshop     def memory        import os     from wmi import WMI     w   WMI          result   w query  SELECT WorkingSet FROM Win32 PerfRawData PerfProc Process WHERE IDProcess  d    os getpid        return int result 0  WorkingSet    On Linux  from python cookbook http   code activestate com recipes 286222    import os  proc status     proc  d status    os getpid     scale     kB   1024 0   mB   1024 0 1024 0             KB   1024 0   MB   1024 0 1024 0   def  VmB VmKey          Private              global  proc status   scale        get pseudo file   proc  lt pid gt  status     try          t   open  proc status          v   t read           t close       except          return 0 0    non-Linux         get VmKey line e g   VmRSS   9999  kB n          i   v index VmKey      v   v i   split None  3     whitespace     if len v   lt  3          return 0 0    invalid format         convert Vm value to bytes     return float v 1      scale v 2     def memory since 0 0          Return memory usage in bytes              return  VmB  VmSize    - since   def resident since 0 0          Return resident memory usage in bytes              return  VmB  VmRSS    - since   def stacksize since 0 0          Return stack size in bytes              return  VmB  VmStk    - since

User · Answer

Current memory usage of the current process on Linux  for Python 2  Python 3  and pypy  without any imports  def getCurrentMemoryUsage            Memory usage in kB          with open   proc self status   as f          memusage   f read   split  VmRSS    1  split   n   0   -3       return int memusage strip     It reads the status file of the current process  takes everything after VmRSS   then takes everything before the first newline  isolating the value of VmRSS   and finally cuts off the last 3 bytes which are a space and the unit  kB   To return  it strips any whitespace and returns it as a number  Tested on Linux 4 4 and 4 9  but even an early Linux version should work  looking in man proc and searching for the info on the  proc  PID status file  it mentions minimum versions for some fields  like Linux 2 6 10 for  quot VmPTE quot    but the  quot VmRSS quot  field  which I use here  has no such mention  Therefore I assume it has been in there since an early version

User · Answer

For Unix systems command time   usr bin time  gives you that info if you pass -v  See Maximum resident set size below  which is the maximum  peak  real  not virtual  memory that was used during program execution      usr bin time -v ls        Command being timed   ls        User time  seconds   0 00     System time  seconds   0 01     Percent of CPU this job got  250      Elapsed  wall clock  time  h mm ss or m ss   0 00 00     Average shared text size  kbytes   0     Average unshared data size  kbytes   0     Average stack size  kbytes   0     Average total size  kbytes   0     Maximum resident set size  kbytes   0     Average resident set size  kbytes   0     Major  requiring I O  page faults  0     Minor  reclaiming a frame  page faults  315     Voluntary context switches  2     Involuntary context switches  0     Swaps  0     File system inputs  0     File system outputs  0     Socket messages sent  0     Socket messages received  0     Signals delivered  0     Page size  bytes   4096     Exit status  0

User · Answer

Below is my function decorator which allows to track how much memory this process consumed before the function call  how much memory it uses after the function call  and how long the function is executed    import time import os import psutil   def elapsed since start       return time strftime   H  M  S   time gmtime time time   - start     def get process memory        process   psutil Process os getpid        return process memory info   rss   def track func       def wrapper  args    kwargs           mem before   get process memory           start   time time           result   func  args    kwargs          elapsed time   elapsed since start          mem after   get process memory           print      memory before        after        consumed        exec time      format              func   name                mem before  mem after  mem after - mem before              elapsed time           return result     return wrapper   So  when you have some function decorated with it  from utils import track   track def list create n       print  inside list create       return  1    n   You will be able to see this output   inside list create list create  memory before  45 928 448  after  46 211 072  consumed  282 624  exec time  00 00 00

[python] Total memory used by Python process?

Examples related to python

Examples related to memory-management