C Out of Memory exception

Question

Today my application threw an OutOfMemoryException  To me this was always almost impossible since I have 4GB RAM and a lot of virtual memory too  The error happened when I tried to add an existing collection to a new list  List lt Vehicle gt  vList   new List lt Vehicle gt  selectedVehicles      To my understanding there isn t much memory allocated here since the vehicles my new list should contain already exist inside the memory  I have to admit Vehicle is a very complex class and I tried to add about 50 000 items to the new list at once  But since all Vehicles in the application come from a database that is only 200MB in size  I have no idea what may cause an OutOfMemoryException at this point

User · Answer

As  Net progresses  so does their ability to add new 32-bit configurations that trips everyone up it seems    If you are on  Net Framework 4 7 2 do the following   Go to Project Properties  Build  Uncheck  prefer 32-bit   Cheers

User · Answer

I know this is an old question but since none of the answers mentioned the large object heap  this might be of use to others who find this question      Any memory allocation in  NET that is over 85 000 bytes comes from the large object heap  LOH  not the normal small object heap  Why does this matter  Because the large object heap is not compacted  Which means that the large object heap gets fragmented and in my experience this inevitably leads to out of memory errors   In the original question the list has 50 000 items in it  Internally a list uses an array  and assuming 32 bit that requires 50 000 x 4bytes   200 000 bytes  or double that if 64 bit   So that memory allocation is coming from the large object heap   So what can you do about it   If you are using a  net version prior to 4 5 1 then about all you can do about it is to be aware of the problem and try to avoid it  So  in this instance  instead of having a list of vehicles you could have a list of lists of vehicles  provided no list ever had more than about 18 000 elements in it  That can lead to some ugly code  but it is viable work around   If you are using  net 4 5 1 or later then the behaviour of the garbage collector has changed subtly  If you add the following line where you are about to make large memory allocations   System Runtime GCSettings LargeObjectHeapCompactionMode   System Runtime GCLargeObjectHeapCompactionMode CompactOnce    it will force the garbage collector to compact the large object heap - next time only   It might not be the best solution but the following has worked for me   int tries   0  while  tries    lt  2      try              some large allocation         return        catch  System OutOfMemoryException          System Runtime GCSettings LargeObjectHeapCompactionMode   System Runtime GCLargeObjectHeapCompactionMode CompactOnce      GC Collect            of course this only helps if you have the physical  or virtual  memory available

User · Answer

You should not try to bring all the list at once  te size of the elements in the database is not the same that the one it takes into memory  If you want to process the elements you should use  a for each loop and take advantage of entity framework lazy loading so you dont bring all the elements into memory at once  In case you want to show the list use pagination   Skip   and  take

User · Answer

OutOfMemoryException  on 32-bit machines  is just as often about Fragmentation as actual hard limits on memory - you ll find lots about this  but here s my first google hit briefly discussing it  http   blogs msdn com b joshwil archive 2005 08 10 450202 aspx    Anthony Pegram is referring to the same problem in his comment above     That said  there is one other possibility that comes to mind for your code above  As you re using the  IEnumerable  constructor to the List  you may not giving the object any hints as to the size of the collection you re passing to the List constructor  If the object you are passing is is not a collection  does not implement the ICollection interface   then behind-the-scenes the List implementation is going to need to grow several  or many  times  each time leaving behind a too-small array that needs to be garbage collected  The garbage collector probably won t get to those discarded arrays fast enough  and you ll get your error    The simplest fix for this would be to use the List int capacity  constructor to tell the framework what backing array size to allocate  even if you re estimating and just guessing  50000  for example   and then use the AddRange IEnumerable collection  method to actually populate your list   So  simplest  Fix  if I m right  replace  List lt Vehicle gt  vList   new List lt Vehicle gt  selectedVehicles     with   List lt Vehicle gt  vList   new List lt Vehicle gt  50000     vList AddRange selectedVehicles     All the other comments and answers still apply in terms of overall design decisions - but this might be a quick fix   Note  as  Alex commented below   this is only an issue if selectedVehicles is not an ICollection

User · Answer

While the GC compacts the small object heap as part of an optimization strategy to eliminate memory holes  the GC never compacts the large object heap for performance reasons   the cost of compaction is too high for large objects  greater than 85KB in size      Hence if you are running a program that uses many large objects in an x86 system  you might encounter OutOfMemory exceptions  If you are running that program in an x64 system  you might have a fragmented heap

User · Answer

Data stored in database compared to memory in your application is very different   There isn t a way to get the exact size of your object but you could do this   GC GetTotalMemory      After a certain amount of objects have been loaded and see how much your memory is changing as you load the list   If it is the list that is causing the excessive memory usage then we can look at ways to minimize it  Such as why do you want 50 000 objects loaded into memory all at once in the first place  Wouldn t it be best to call the DB as you require them   If you take a look here  http   www dotnetperls com array-memory you will also see that objects in  NET are greater than their actual data  A generic list is even more of a memory hog than an array  If you have a generic list inside your object then it will grow even faster

User · Answer

My Development Team resolved this situation   We added the following Post-Build script into the  exe project and compiled again  setting the target to x86 and increasing by 1 5 gb and also x64 Platform target increasing memory using 3 2 gb  Our application is 32 bit   Related URLs    http   www guylangston net blog Article MaxMemory  NET Out Of Memory Exception - Used 1 3GB but have 16GB installed   Script   if exist    DevEnvDir    tools vsvars32 bat        call    DevEnvDir    tools vsvars32 bat      editbin  largeaddressaware    TargetPath

User · Answer

Two points    If you are running a 32 bit Windows  you won t have all the 4GB accessible  only 2GB  Don t forget that the underlying implementation of List is an array  If your memory is heavily fragmented  there may not be enough contiguous space to allocate your List  even though in total you have plenty of free memory

User · Answer

Net4 5 does not have a 2GB limitation for objects any more  Add this lines to App config    lt runtime gt       lt gcAllowVeryLargeObjects enabled  true    gt       lt  runtime gt    and it will be possible to create very large objects without getting OutOfMemoryException   Please note it will work only on x64 OS s

User · Answer

3 years old topic  but I found another working solution  If you re sure you have enough free memory  running 64 bit OS and still getting exceptions  go to Project properties -  Build tab and be sure to set x64 as a Platform target

[c#] C# : Out of Memory exception

Examples related to c#

Examples related to .net

Examples related to out-of-memory