How to monitor Java memory usage

Question

We have a j2ee application running on Jboss and we want to monitor its memory usage  Currently we use the following code       System gc        Runtime rt   Runtime getRuntime        long usedMB    rt totalMemory   - rt freeMemory      1024   1024      logger information this   memory usage    usedMB     This code works fine  That means it shows memory curve which corresponds to reality  When we create a big xml file from a DB a curve goes up  after the extraction is finished it goes down       A consultant told us that calling gc   explicitly is wrong   let jvm decide when to run gc   Basically his arguments were the same as disscussed here   But I still don t understand    how can I have my memory usage curve  what is wrong with the explicit gc    I don t care about small performance issues which can happen with explicit gc   and which I would estimate in 1-3   What I need is memory and thread monitor which helps me in analysis of our system on customer site

User · Answer

Explicitly running System gc   on a production system is a terrible idea  If the memory gets to any size at all  the entire system can freeze while a full GC is running  On a multi-gigabyte-sized server  this can easily be very noticeable  depending on how the jvm is configured  and how much headroom it has  etc etc - I ve seen pauses of more than 30 seconds    Another issue is that by explicitly calling GC you re not actually monitoring how the JVM is running the GC  you re actually altering it - depending on how you ve configured the JVM  it s  going to garbage collect when appropriate  and usually incrementally  It doesn t just run a full GC when it runs out of memory   What you ll be printing out will be nothing like what the JVM will do on it s own - for one thing you ll probably see fewer automatic   incremental GC s as you ll be clearing the memory manually    As Nick Holt s post points out  options to print GC activity already exist as JVM flags   You could have a thread that just prints out free and available at reasonable intervals  this will show you actual mem useage

User · Answer

About System gc      I just read in Oracle s documentation the following sentence here     The performance effect of explicit garbage collections can be measured by disabling them using the flag -XX  DisableExplicitGC  which causes the VM to ignore calls to System gc      If your VM vendor and version supports that flag you can run your code with and without it and compare Performance   Also note the previous quoted sentence is preceded by this one      This can force a major collection to be done when it may not be necessary  for example  when a minor collection would suffice   and so in general should be avoided

User · Answer

I would say that the consultant is right in the theory  and you are right in practice  As the saying goes      In theory  theory and practice are the same  In practice  they are not    The Java spec says that System gc suggests to call garbage collection  In practice  it just spawns a thread and runs right away on the Sun JVM   Although in theory you could be messing up some finely tuned JVM implementation of garbage collection  unless you are writing generic code intended to be deployed on any JVM out there  don t worry about it  If it works for you  do it

User · Answer

The problem with system gc  is that the JVM already automatically allocates time to the garbage collector based on memory usage   However  if you are  for instance  working in a very memory limited condition  like a mobile device  System gc allows you to manually allocate more time towards this garbage collection  but at the cost of cpu time  but  as you said  you aren t that concerned about performance issues of gc    Best practice would probably be to only use it where you might be doing large amounts of deallocation  like flushing a large array    All considered  since you are simply concerned about memory usage  feel free to call gc  or  better yet  see if it makes much of a memory difference in your case  and then decide

User · Answer

If you use the JMX provided history of GC runs you can use the same before after numbers  you just dont have to force a GC    You just need to keep in mind that those GC runs  typically one for old and one for new generation  are not on regular intervalls  so you need to extract the starttime as well for plotting  or you plot against a sequence number  for most practical purposes that would be enough for plotting     For example on Oracle HotSpot VM with ParNewGC  there is a JMX MBean called java lang type GarbageCollector name PS Scavenge  it has a attribute LastGCInfo  it returns a CompositeData of the last YG scavenger run  It is recorded with duration  absolute startTime and memoryUsageBefore and memoryUsageAfter   Just use a timer to read that attribute  Whenever a new startTime shows up you know that it describes a new GC event  you extract the memory information and keep polling for the next update   Not sure if a AttributeChangeNotification somehow can be used    Tip  in your timer you might measure the distance to the last GC run  and if that is too long for the resulution of your plotting  you could invoke System gc   conditionally  But I would not do that in a OLTP instance

User · Answer

Have you tried JMX   http   java sun com developer technicalArticles J2SE jconsole html    source  sun com

User · Answer

As has been suggested  try VisualVM to get a basic view   You can also use Eclipse MAT  to do a more detailed memory analysis   It s ok to do a System gc   as long as you dont depend on it  for the correctness of your program

User · Answer

If you use java 1 5  you can look at ManagementFactory getMemoryMXBean   which give you numbers on all kinds of memory  heap and non-heap  perm-gen   A good example can be found there http   www freshblurbs com explaining-java-lang-outofmemoryerror-permgen-space

User · Answer

JavaMelody might be a solution for your need  Developed for Java EE applications  this tool measure and build report about the real operation of your applications on any environments  It s free and open-source and easy to integrate into applications with some history  no database nor profiling  really lightweight

User · Answer

There are tools that let you monitor the VM s memory usage   The VM can expose memory statistics using JMX   You can also print GC statistics to see how the memory is performing over time     Invoking System gc   can harm the GC s performance because objects will be prematurely moved from the new to old generations  and weak references will be cleared prematurely   This can result in decreased memory efficiency  longer GC times  and decreased cache hits  for caches that use weak refs    I agree with your consultant  System gc   is bad   I d go as far as to disable it using the command line switch

User · Answer

You can take a look at stagemonitor  It is a open source java  web  application performance monitor  It captures response time metrics  JVM metrics  request details  including a call stack captured by the request profiler  and more  The overhead is very low   Optionally  you can use the great timeseries database graphite with it to store a long history of datapoints that you can look at with fancy dashboards   Example    Take a look at the project website to see screenshots  feature descriptions and documentation   Note  I am the developer of stagemonitor

User · Answer

If you want to really look at what is going on in the VM memory you should use a good tool like VisualVM   This is Free Software and it s a great way to see what is going on   Nothing is really  wrong  with explicit gc   calls  However  remember that when you call gc   you are  suggesting  that the garbage collector run  There is no guarantee that it will run at the exact time you run that command

User · Answer

If you like a nice way to do this from the command line use jstat   http   java sun com j2se 1 5 0 docs tooldocs share jstat html  It gives raw information at configurable intervals which is very useful for logging and graphing purposes

User · Answer

Take a look at the JVM args  http   java sun com javase technologies hotspot vmoptions jsp DebuggingOptions     XX -PrintGC   Print messages at garbage collection  Manageable       -XX -PrintGCDetails   Print more details at garbage collection    Manageable   Introduced in 1 4 0        -XX -PrintGCTimeStamps    Print timestamps at garbage collection    Manageable  Introduced in 1 4 0        -XX -PrintTenuringDistribution    Print tenuring age information    While you re not going to upset the JVM with explicit calls to System gc   they may not have the effect you are expecting   To really understand what s going on with the memory in a JVM with read anything and everything the Brian Goetz writes

User · Answer

Peek into what is happening inside tomcat through Visual VM  http   www skill-guru com blog 2010 10 05 increasing-permgen-size-in-your-server

[java] How to monitor Java memory usage?

Examples related to java

Examples related to garbage-collection

Examples related to monitoring

Examples related to memory-management