What is RSS and VSZ in Linux memory management

Question

What are RSS and VSZ in Linux memory management   In a multithreaded environment how can both of these can be managed and tracked

User · Answer

They are not managed  but measured and possibly limited  see getrlimit system call  also on getrlimit 2     RSS means resident set size  the part of your virtual address space sitting in RAM    You can query the virtual address space of process 1234 using proc 5  with cat  proc 1234 maps and its status  including memory consumption  thru cat  proc 1234 status

User · Answer

RSS is the Resident Set Size and is used to show how much memory is allocated to that process and is in RAM   It does not include memory that is swapped out   It does include memory from shared libraries as long as the pages from those libraries are actually in memory   It does include all stack and heap memory   VSZ is the Virtual Memory Size   It includes all memory that the process can access  including memory that is swapped out  memory that is allocated  but not used  and memory that is from shared libraries   So if process A has a 500K binary and is linked to 2500K of shared libraries  has 200K of stack heap allocations of which 100K is actually in memory  rest is swapped or unused   and it has only actually loaded 1000K of the shared libraries and 400K of its own binary then   RSS  400K   1000K   100K   1500K VSZ  500K   2500K   200K   3200K   Since part of the memory is shared  many processes may use it  so if you add up all of the RSS values you can easily end up with more space than your system has   The memory that is allocated also may not be in RSS until it is actually used by the program   So if your program allocated a bunch of memory up front  then uses it over time  you could see RSS going up and VSZ staying the same   There is also PSS  proportional set size    This is a newer measure which tracks the shared memory as a proportion used by the current process   So if there were two processes using the same shared library from before   PSS  400K    1000K 2    100K   400K   500K   100K   1000K   Threads all share the same address space  so the RSS  VSZ and PSS for each thread is identical to all of the other threads in the process   Use ps or top to view this information in linux unix   There is way more to it than this  to learn more check the following references    http   manpages ubuntu com manpages en man1 ps 1 html https   web archive org web 20120520221529 http   emilics com blog article mconsumption html   Also see    A way to determine a process  39 s  quot real quot  memory usage  i e  private dirty RSS

User · Answer

RSS is Resident Set Size  physically resident memory - this is currently occupying space in the machine s physical memory   and VSZ is Virtual Memory Size  address space allocated - this has addresses allocated in the process s memory map  but there isn t necessarily any actual memory behind it all right now    Note that in these days of commonplace virtual machines  physical memory from the machine s view point may not really be actual physical memory

User · Answer

I think much has already been said  about RSS vs VSZ  From an administrator programmer user perspective  when I design code applications I am more concerned about the RSZ   Resident memory   as and when you keep pulling more and more variables  heaped  you will see this value shooting up  Try a simple program to build malloc based space allocation in loop  and make sure you fill data in that malloc d space  RSS keeps moving up  As far as VSZ is concerned  it s more of virtual memory mapping that linux does  and one of its core features derived out of conventional operating system concepts  The VSZ management is done by Virtual memory management of the kernel  for more info on VSZ  see Robert Love s description on mm struct and vm struct  which are part of basic task struct data structure in kernel

User · Answer

Minimal runnable example  For this to make sense  you have to understand the basics of paging  How does x86 paging work  and in particular that the OS can allocate virtual memory via page tables   its internal memory book keeping  VSZ virtual memory  before it actually has a backing storage on RAM or disk  RSS resident memory    Now to observe this in action  let s create a program that    allocates more RAM than our physical memory with mmap writes one byte on each page to ensure that each of those pages goes from virtual only memory  VSZ  to actually used memory  RSS  checks the memory usage of the process with one of the methods mentioned at  Memory usage of current process in C   main c   define  GNU SOURCE  include  lt assert h gt   include  lt inttypes h gt   include  lt stdio h gt   include  lt stdlib h gt   include  lt string h gt   include  lt sys mman h gt   include  lt unistd h gt   typedef struct       unsigned long size resident share text lib data dt    ProcStatm      https   stackoverflow com questions 1558402 memory-usage-of-current-process-in-c 7212248 7212248    void ProcStat init ProcStatm  result        const char  statm path     proc self statm       FILE  f   fopen statm path   r        if  f            perror statm path           abort              if 7    fscanf          f            lu  lu  lu  lu  lu  lu  lu            amp  result- gt size            amp  result- gt resident            amp  result- gt share            amp  result- gt text            amp  result- gt lib            amp  result- gt data            amp  result- gt dt                   perror statm path           abort              fclose f      int main int argc  char   argv        ProcStatm proc statm      char  base   p      char system cmd 1024       long page size      size t i  nbytes  print interval  bytes since last print      int snprintf return          Decide how many ints to allocate         if  argc  lt  2            nbytes   0x10000        else           nbytes   strtoull argv 1   NULL  0             if  argc  lt  3            print interval   0x1000        else           print interval   strtoull argv 2   NULL  0             page size   sysconf  SC PAGESIZE           Allocate the memory         base   mmap          NULL          nbytes          PROT READ   PROT WRITE          MAP SHARED   MAP ANONYMOUS          -1          0            if  base    MAP FAILED            perror  mmap            exit EXIT FAILURE                 Write to all the allocated pages         i   0      p   base      bytes since last print   0         Produce the ps command that lists only our VSZ and RSS         snprintf return   snprintf          system cmd          sizeof system cmd            ps -o pid vsz rss   awk   if  NR    1     1       ju    print              uintmax t getpid              assert snprintf return  gt   0       assert  size t snprintf return  lt  sizeof system cmd        bytes since last print   print interval      do              Modify a byte in the page              p   i          p    page size          bytes since last print    page size             Print process memory usage every print interval bytes             We count memory using a few techniques from             https   stackoverflow com questions 1558402 memory-usage-of-current-process-in-c            if  bytes since last print  gt  print interval                bytes since last print -  print interval              printf  extra memory committed  lu KiB n    i   page size    1024               ProcStat init  amp proc statm                  Check  proc self statm                printf                    proc self statm size resident  lu  lu KiB n                    proc statm size   page size    1024                   proc statm resident   page size    1024                               Check ps                 puts system cmd               system system cmd               puts                        i          while  p  lt  base   nbytes           Cleanup         munmap base  nbytes       return EXIT SUCCESS      GitHub upstream   Compile and run   gcc -ggdb3 -O0 -std c99 -Wall -Wextra -pedantic -o main out main c echo 1   sudo tee  proc sys vm overcommit memory sudo dmesg -c   main out 0x1000000000 0x200000000 echo    sudo dmesg   where    0x1000000000    64GiB  2x my computer s physical RAM of 32GiB 0x200000000    8GiB  print the memory every 8GiB  so we should get 4 prints before the crash at around 32GiB  echo 1   sudo tee  proc sys vm overcommit memory  required for Linux to allow us to make a mmap call larger than physical RAM  maximum memory which malloc can allocate   Program output   extra memory committed 0 KiB  proc self statm size resident 67111332 768 KiB ps -o pid vsz rss   awk   if  NR    1     1     29827   print     PID    VSZ   RSS 29827 67111332 1648  extra memory committed 8388608 KiB  proc self statm size resident 67111332 8390244 KiB ps -o pid vsz rss   awk   if  NR    1     1     29827   print     PID    VSZ   RSS 29827 67111332 8390256  extra memory committed 16777216 KiB  proc self statm size resident 67111332 16778852 KiB ps -o pid vsz rss   awk   if  NR    1     1     29827   print     PID    VSZ   RSS 29827 67111332 16778864  extra memory committed 25165824 KiB  proc self statm size resident 67111332 25167460 KiB ps -o pid vsz rss   awk   if  NR    1     1     29827   print     PID    VSZ   RSS 29827 67111332 25167472  Killed   Exit status   137   which by the 128   signal number rule means we got signal number 9  which man 7 signal says is SIGKILL  which is sent by the Linux out-of-memory killer   Output interpretation    VSZ virtual memory remains constant at printf  0x X n  0x40009A4 KiB    64GiB  ps values are in KiB  after the mmap  RSS  real memory usage  increases lazily only as we touch the pages  For example    on the first print  we have extra memory committed 0  which means we haven t yet touched any pages  RSS is a small 1648 KiB which has been allocated for normal program startup like text area  globals  etc  on the second print  we have written to 8388608 KiB    8GiB worth of pages  As a result  RSS increased by exactly 8GIB to 8390256 KiB    8388608 KiB   1648 KiB RSS continues to increase in 8GiB increments  The last print shows about 24 GiB of memory  and before 32 GiB could be printed  the OOM killer killed the process    See also  https   unix stackexchange com questions 35129 need-explanation-on-resident-set-size-virtual-size  OOM killer logs  Our dmesg commands have shown the OOM killer logs   An exact interpretation of those has been asked at    Understanding the Linux oom-killer  39 s logs but let s have a quick look here  https   serverfault com questions 548736 how-to-read-oom-killer-syslog-messages   The very first line of the log was     7283 479087  mongod invoked oom-killer  gfp mask 0x6200ca GFP HIGHUSER MOVABLE   order 0  oom score adj 0   So we see that interestingly it was the MongoDB daemon that always runs in my laptop on the background that first triggered the OOM killer  presumably when the poor thing was trying to allocate some memory   However  the OOM killer does not necessarily kill the one who awoke it   After the invocation  the kernel prints a table or processes including the oom score     7283 479292     pid      uid  tgid total vm      rss pgtables bytes swapents oom score adj name   7283 479303       496      0   496    16126        6   172032      484             0 systemd-journal   7283 479306       505      0   505     1309        0    45056       52             0 blkmapd   7283 479309       513      0   513    19757        0    57344       55             0 lvmetad   7283 479312       516      0   516     4681        1    61440      444         -1000 systemd-udevd   and further ahead we see that our own little main out actually got killed on the previous invocation     7283 479871  Out of memory  Kill process 15665  main out  score 865 or sacrifice child   7283 479879  Killed process 15665  main out  total-vm 67111332kB  anon-rss 92kB  file-rss 4kB  shmem-rss 30080832kB   7283 479951  oom reaper  reaped process 15665  main out   now anon-rss 0kB  file-rss 0kB  shmem-rss 30080832kB   This log mentions the score 865 which that process had  presumably the highest  worst  OOM killer score as mentioned at  https   unix stackexchange com questions 153585 how-does-the-oom-killer-decide-which-process-to-kill-first  Also interestingly  everything apparently happened so fast that before the freed memory was accounted  the oom was awoken again by the DeadlineMonitor process     7283 481043  DeadlineMonitor invoked oom-killer  gfp mask 0x6200ca GFP HIGHUSER MOVABLE   order 0  oom score adj 0   and this time that killed some Chromium process  which is usually my computers normal memory hog     7283 481773  Out of memory  Kill process 11786  chromium-browse  score 306 or sacrifice child   7283 481833  Killed process 11786  chromium-browse  total-vm 1813576kB  anon-rss 208804kB  file-rss 0kB  shmem-rss 8380kB   7283 497847  oom reaper  reaped process 11786  chromium-browse   now anon-rss 0kB  file-rss 0kB  shmem-rss 8044kB   Tested in Ubuntu 19 04  Linux kernel 5 0 0

[linux] What is RSS and VSZ in Linux memory management

Examples related to linux