Wednesday, April 28, 2010

Who is this OutOfMemory guy and why does he make my process crash when I have plenty of memory left?

To answer this question, there are a few concepts we need to discuss.

Working on a 32-bit system, you can address 4 GB of memory, out of this 2 GB is typically reserved to the Operating System and 2 GB are allowed for each user mode process, such as w3wp.exe (asp.net) for example. This memory is called virtual memory and the 2 GB’s are 2GB independently of how much RAM you have added to the system.  The amount of RAM simply decides how much paging and swapping you will do, i.e. how fast memory access will be.

When a process allocates memory it does so by first reserving virtual memory and then committing memory out of this chunk (this is the memory actually used). The committed memory is called private bytes.

The virtual address space is used for a number of different items in the process such as:

   *      Dll’s
   *      Native heaps (non .net heaps)
   *      Threads (each thread reserves 1 MB for the stack)
   *      .net heaps (for managed variables)
   *      .net loader heap (for assemblies and related structures)
   *      Virtual allocations made by com components

Virtual memory allocations are not necessarily (or very seldom) lined up nicely in memory. For example, dll’s have preferred loading addresses so gaps are left between them, and virtual allocations that have been de-allocated will also leave gaps.  This means that even though you can address 2 GB of virtual memory you may not be able to use it all since when enough memory is used, the memory will look somewhat like a Swiss cheese and your plug might have a big enough hole to fit in.

This is what happens when you get an OutOfMemory exception.

I will likely talk more about .net memory management later, but for now I’m going to make it very brief since there are several very good blogs about this such as Maoni's CLR Performance blog http://blogs.msdn.com/maoni/  and http://blogs.msdn.com/yunjin .

In the .net framework the garbage collector which is our memory manager reserves virtual memory in heaps. Once these heaps are created and we create a new instance of a .net object this object is stored in these heap segments and memory is committed.

Read more: If broken it is, fix it you should

Posted via email from jasper22's posterous