Normally, the VM would do a garbage collection automatically before throwing an OutOfMemoryException, so adding an explicit call shouldn't help except in that it perhaps moves the performance hit to an earlier moment in time.
However, I think I encountered a case where it might be relevant. I'm not sure though, as I have yet to test whether it has any effect:
When you memory-map a file, I believe the map() call throws an IOException when a large enough block of memory is not available. A garbage collection just before the map() file might help prevent that, I think. What do you think?