Chris Lyon's WebLog : Orcas

New In Orcas Part 3: GC Latency Modes

clyon — Mon, 12 Mar 2007 22:10:00 GMT

As you may know, there are different GC modes to choose from depending on the type of application you’re using: Server GC, Workstation GC, and Concurrent GC (more info). These settings are process-wide, set at the beginning of the process. Once the GC mode is set, it cannot be changed.
In Orcas, we’ve added the concept of GC Latency Modes that while process-wide, can be changed during the lifetime of the process to meet an application’s needs.
The Latency Modes can be accessed as new properties onto the GCSettings class:

System.Runtime.GCLatencyMode System.Runtime.GCSettings.LatencyMode { get; set; }

The values for GCLatencyMode are Batch, Interactive and LowLatency.

Batch: This mode is designed for maximum throughput, at the expense of responsiveness. It is best for applications with no UI or server-side operations and is equivalent to Workstation GC without Concurrent GC. If Concurrent GC is enabled, switching to Batch mode will prevent any further concurrent collections. This is the only valid mode for Server GC.
Interactive: This mode balances responsiveness with throughput. It is designed for applications with UI and is the default Latency Mode, equivalent to Workstation GC with Concurrent GC. This mode is not available on Server GC.
LowLatency: This mode is meant for short-term, time-sensitive operations where interruptions from the GC may be disruptive, like animation rendering or data acquisition functions. This mode is not available on Server GC.

How does LowLatency mode work?

When you set the latency mode to LowLatency, the GC will perform almost no generation 2 collections, nor will it start any new concurrent collections. Since generation 2 is unbounded and can become very large, collecting it can cause your managed threads to pause for short amounts of time. This can be unacceptable for certain scenarios.

To be clear, I’m not talking about real-time application requirements, rather requirements that a short-running block of code run smoothly with minimal interruptions from the runtime. LowLatency mode is not real-time mode.

I mentioned above that in LowLatency mode, the GC will perform almost no most generation 2 collections, but there are situations when it will. As we know, there are three things that cause the GC to perform a collection (more info):

Allocation exceeds the Gen0 threshold – normally, these collections can escalate into full heap collections. With LowLatency, generation 1 is the maximum generation that will be collected, possibly promoting objects to generation 2.
System.GC.Collect is called – this will continue to work as expected. If you specify to collect generation 2, the GC will honor your request regardless of the Latency Mode.
System is in low memory situation – the OS has raised an event telling the runtime that it is low on system memory. In this ase the GC will perform a generation 2 collection to attempt to free memory. The alternative is to allow the OS to begin paging which will generally have worse pause times than a full collection.

How to safely use LowLatency mode

As you might have guessed, since generation 2 is rarely collected, OutOfMemoryExceptions are more likely under LowLatency mode. Here are some guidelines to follow to avoid potential problems:

Keep the amount of time spent in LowLatency as short as possible. Remember, you’re changing the behavior of the GC, which can lead to sub-optimal performance in the long-run.
While in LowLatency mode, minimize the number of allocations you make, in particular allocations onto the Large Object Heap and pinned objects.
Be mindful of other threads that could be allocating. Remember, these settings are process-wide, so you could generate an OutOfMemoryException on any thread that may be allocating.
Wrap the LowLatency code in a CER (more info).
Remember to set the latency mode back to avoid hard-to debug OutOfMemoryExceptions later.

Here’s a code sample of how to use LowLatency mode

// preallocate objects here GCLatencyMode oldMode = GCSettings.LatencyMode; RuntimeHelpers.PrepareConstrainedRegions(); try { GCSettings.LatencyMode = GCLatencyMode.LowLatency; // perform time-sensitive actions here /* minimize: -all allocations, especially LOH allocations -pinning -allocations on other threads */ } catch (ApplicationException) { // catch any exceptions you expect your application to throw // perform cleanup code } finally { // always set the mode back! GCSettings.LatencyMode = oldMode; }

Remember, this mode can cause failures in your application, so please use good judgment when using it.

New In Orcas Part 2: GC Collection Modes

clyon — Thu, 08 Mar 2007 01:14:00 GMT

In Orcas we’ve added an overload to System.GC.Collect():

void System.GC.Collect(int generation, System.GCCollectionMode mode)

Where generation is the highest generation to collect (from 0 to System.GC.MaxGeneration) and mode can be:

Default: the same behavior if you called GC.Collect without specifying the mode. Currently this is the same behavior as Forced, but this is subject to change in future versions of the runtime.
Forced: guarantees a collection occurs for all the generations up to and including generation.
Optimized: this mode will tell the GC to only collect if it determines that a collection will be productive. In this case, “productive” is determined by a number of factors, including amount of memory considered garbage, heap fragmentation, etc. The exact formula is subject to change between CLR releases. If the GC decides a collection will not be productive, then the call will have no effect.

When should we use these new modes?

Calling GC.Collect is generally discouraged, but as Rico points out here, there are legitimate circumstances where you know there is a large number of objects you’ll never need again. For example, at the end of a game level, when a custom Form is closed, or when a web form is finished, there may be a number of long-lived objects in generation 2 that are now dead. By calling an Optimized collection at this point you give the GC a chance to evaluate the heap, and have it decide if a collection will free enough memory to be worth it.

Forced and Default modes should generally only be used for debugging or testing scenarios, where you want to ensure objects are collected at a certain point in your application, or want to compare performance data. In future versions of the CLR, there may be new guidance for using Default, but for now, its use is discouraged.

New In Orcas Part 1: What we’ve been doing

clyon — Tue, 06 Mar 2007 08:14:00 GMT

The Orcas March CTP is out, and what does that mean for the Garbage Collector? The GC team has been concentrating on three areas for this release:

Bug fixes. For Orcas, we’ve fixed several premature Out of Memory bugs, improved stability in certain stressful conditions, and even improved performance in some scenarios.
GC Collection Modes. We’ve added a new overload to System.GC.Collect that takes a System.GCCollectionMode enum, which allows the user to either force a collection or allow the GC to decide if a collection would be productive.
GC Latency Modes. A new GC feature that allows the user to specify blocks of code that are time-sensitive, and the GC will minimize its intrusiveness. This has been implemented as new properties on the System.Runtime.GCSettings class.

In my next few blog entries I’ll go into more detail on our new features, including code samples and best practices.