[java] How to monitor Java memory usage?

We have a j2ee application running on Jboss and we want to monitor its memory usage. Currently we use the following code

    System.gc();
    Runtime rt = Runtime.getRuntime();
    long usedMB = (rt.totalMemory() - rt.freeMemory()) / 1024 / 1024;
    logger.information(this, "memory usage" + usedMB);

This code works fine. That means it shows memory curve which corresponds to reality. When we create a big xml file from a DB a curve goes up, after the extraction is finished it goes down.

A consultant told us that calling gc() explicitly is wrong, "let jvm decide when to run gc". Basically his arguments were the same as disscussed here. But I still don't understand:

• how can I have my memory usage curve?
• what is wrong with the explicit gc()? I don't care about small performance issues which can happen with explicit gc() and which I would estimate in 1-3%. What I need is memory and thread monitor which helps me in analysis of our system on customer site.

If you use java 1.5, you can look at ManagementFactory.getMemoryMXBean() which give you numbers on all kinds of memory. heap and non-heap, perm-gen.

A good example can be found there http://www.freshblurbs.com/explaining-java-lang-outofmemoryerror-permgen-space

If you use the JMX provided history of GC runs you can use the same before/after numbers, you just dont have to force a GC.

You just need to keep in mind that those GC runs (typically one for old and one for new generation) are not on regular intervalls, so you need to extract the starttime as well for plotting (or you plot against a sequence number, for most practical purposes that would be enough for plotting).

For example on Oracle HotSpot VM with ParNewGC, there is a JMX MBean called java.lang:type=GarbageCollector,name=PS Scavenge, it has a attribute LastGCInfo, it returns a CompositeData of the last YG scavenger run. It is recorded with duration, absolute startTime and memoryUsageBefore and memoryUsageAfter.

Just use a timer to read that attribute. Whenever a new startTime shows up you know that it describes a new GC event, you extract the memory information and keep polling for the next update. (Not sure if a AttributeChangeNotification somehow can be used.)

Tip: in your timer you might measure the distance to the last GC run, and if that is too long for the resulution of your plotting, you could invoke System.gc() conditionally. But I would not do that in a OLTP instance.

The problem with system.gc, is that the JVM already automatically allocates time to the garbage collector based on memory usage.

However, if you are, for instance, working in a very memory limited condition, like a mobile device, System.gc allows you to manually allocate more time towards this garbage collection, but at the cost of cpu time (but, as you said, you aren't that concerned about performance issues of gc).

Best practice would probably be to only use it where you might be doing large amounts of deallocation (like flushing a large array).

All considered, since you are simply concerned about memory usage, feel free to call gc, or, better yet, see if it makes much of a memory difference in your case, and then decide.

Have you tried JMX?

http://java.sun.com/developer/technicalArticles/J2SE/jconsole.html

_{(source: sun.com)}

JavaMelody might be a solution for your need.

Developed for Java EE applications, this tool measure and build report about the real operation of your applications on any environments. It's free and open-source and easy to integrate into applications with some history, no database nor profiling, really lightweight.

Explicitly running System.gc() on a production system is a terrible idea. If the memory gets to any size at all, the entire system can freeze while a full GC is running. On a multi-gigabyte-sized server, this can easily be very noticeable, depending on how the jvm is configured, and how much headroom it has, etc etc - I've seen pauses of more than 30 seconds.

Another issue is that by explicitly calling GC you're not actually monitoring how the JVM is running the GC, you're actually altering it - depending on how you've configured the JVM, it's going to garbage collect when appropriate, and usually incrementally (It doesn't just run a full GC when it runs out of memory). What you'll be printing out will be nothing like what the JVM will do on it's own - for one thing you'll probably see fewer automatic / incremental GC's as you'll be clearing the memory manually.

As Nick Holt's post points out, options to print GC activity already exist as JVM flags.

You could have a thread that just prints out free and available at reasonable intervals, this will show you actual mem useage.

You can take a look at stagemonitor. It is a open source java (web) application performance monitor. It captures response time metrics, JVM metrics, request details (including a call stack captured by the request profiler) and more. The overhead is very low.

Optionally, you can use the great timeseries database graphite with it to store a long history of datapoints that you can look at with fancy dashboards.

Example:

Take a look at the project website to see screenshots, feature descriptions and documentation.

Note: I am the developer of stagemonitor

About System.gc()… I just read in Oracle's documentation the following sentence here

The performance effect of explicit garbage collections can be measured by disabling them using the flag -XX:+DisableExplicitGC, which causes the VM to ignore calls to System.gc().

If your VM vendor and version supports that flag you can run your code with and without it and compare Performance.

Also note the previous quoted sentence is preceded by this one:

This can force a major collection to be done when it may not be necessary (for example, when a minor collection would suffice), and so in general should be avoided.

Take a look at the JVM args: http://java.sun.com/javase/technologies/hotspot/vmoptions.jsp#DebuggingOptions

XX:-PrintGC Print messages at garbage collection. Manageable.

-XX:-PrintGCDetails Print more details at garbage collection. Manageable. (Introduced in 1.4.0.)

-XX:-PrintGCTimeStamps Print timestamps at garbage collection. Manageable (Introduced in 1.4.0.)

-XX:-PrintTenuringDistribution Print tenuring age information.

While you're not going to upset the JVM with explicit calls to System.gc() they may not have the effect you are expecting. To really understand what's going on with the memory in a JVM with read anything and everything the Brian Goetz writes.

If you like a nice way to do this from the command line use jstat:

http://java.sun.com/j2se/1.5.0/docs/tooldocs/share/jstat.html

It gives raw information at configurable intervals which is very useful for logging and graphing purposes.

Peek into what is happening inside tomcat through Visual VM. http://www.skill-guru.com/blog/2010/10/05/increasing-permgen-size-in-your-server/

I would say that the consultant is right in the theory, and you are right in practice. As the saying goes:

In theory, theory and practice are the same. In practice, they are not.

The Java spec says that System.gc suggests to call garbage collection. In practice, it just spawns a thread and runs right away on the Sun JVM.

Although in theory you could be messing up some finely tuned JVM implementation of garbage collection, unless you are writing generic code intended to be deployed on any JVM out there, don't worry about it. If it works for you, do it.

As has been suggested, try VisualVM to get a basic view.

You can also use Eclipse MAT, to do a more detailed memory analysis.

It's ok to do a System.gc() as long as you dont depend on it, for the correctness of your program.

There are tools that let you monitor the VM's memory usage. The VM can expose memory statistics using JMX. You can also print GC statistics to see how the memory is performing over time.

Invoking System.gc() can harm the GC's performance because objects will be prematurely moved from the new to old generations, and weak references will be cleared prematurely. This can result in decreased memory efficiency, longer GC times, and decreased cache hits (for caches that use weak refs). I agree with your consultant: System.gc() is bad. I'd go as far as to disable it using the command line switch.