Issues related to high memory utilization on an IIS application server are common. With .NET there is a little misconception that the Garbage Collector (GC) will clean up objects and therefore the process can never run out of memory. This isn’t true. GC will never clean up an object which is in use. If that was the case, you can imagine the kind of problems it would create.

While debugging memory problems, it is a good idea to capture memory dump when the process memory consumption is at its peak maximum usage. For .NET applications, a System.OutOfMemoryException is thrown when GC fails on a VirtualAlloc().

So how do we capture a memory dump when this Exception is thrown? Here’s how.

For .NET Framework version 1.1

Open the registry path: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\.NETFramework

Key: GCFailFastOnOOM

Type: DWORD

Value: 2

For .NET Framework version 2.0 and above

Open the registry path: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\.NETFramework

Key: GCBreakOnOOM

Type: DWORD

Value: 2

Setting the above key causes a DebugBreak within the process when a System.OutOfMemoryException is encountered. You can then use a tool like DebugDiag or a Debugger like WinDBG/CDB/NTSD to capture a dump on this DebugBreak Exception. Windbg/CDB/NTSD debuggers are for advanced users and DebugDiag is generally preferred due to ease of use and is designed to be used in production environments.

Configuring Debug Diagnostic Tool

  1. Download and install DebugDiag to a drive with at least 4-5 GB of disk space.
  2. Open DebugDiag. If prompted to select a rule, select Crash. Else click on Add Rule button and select Crash.
  3. Click Next & select “A specific process”
  4. Select the process name. For IIS 6.0, this will be w3wp.exe. Click Next
  5. Under Advanced Settings, click on Exceptions, then click on Add Exception
  6. From the list of exceptions, select 80000003 Breakpoint Exception
  7. Set Action Type to Full userdump & Action limit to 1. Click OK
  8. Click Save & Close button
  9. Click Next and provide a name for the rule and location where the dump files must be saved.
  10. Click Next and then Finish button.

NOTE: When the dump is captured, the Userdump count column will be incremented by 1. You can then do post mortem debugging using Windbg and SOS.