MSDN Utopia : Performance

Background and Foreground GC in .NET 4

Salva Patuel — Wed, 10 Jun 2009 14:33:00 GMT

Another interesting new feature of the CLR 4 comes from the Garbage collection team. On this version, they are adding some performance enhancements on the memory allocation process. The feature is commonly called “Background GC”. But what does it actually mean?

As applications are starting to consume more memory and some of them moving to wider memory spaces under 64bits processes we have started to see some latency issues while allocating memory when the full GC is running. As you may remember, for workstation version of the CLR we use a concurrent GC. This means that the GC thread will run in parallel without blocking the application execution (well, we try to minimize the blocking time). This thread will scan Gen 2 in order to mark dead objects. This operation can take some time if the memory allocation is quite large and this prevents ephemeral collections while is running. Ok, now you may be asking what does it mean?, let me explain it with some graphics.

Let’s analyze how the current concurrency GC works and the scenario that we are improving:

Now, our application needs to perform a full GC, for this it will scan Generation 2 and try to mark the dead objects as free objects. This is executed by the GC thread, the simplified steps that it will take are the following ones:

1) It will start marking the objects, checking the stacks and the GC roots. This operation will allow further allocations, this means that your application may create a new object and this will be allocated in generation 0.

2) Now there are further allocations that the GC needs to suspend the EE (Execution engine) and this will stop all threads on your application. At this stage no allocation is allowed and your application may suffer some latency.

3) The EE is resumed in order to continue working on the heap and other bits and pieces that the GC needs to handle; at this stage the allocation is allowed. But what happen if our ephemeral segment is full while this collection happens?

4) At this stage the ephemeral collection cannot swap segments and the allocation will be delayed, adding latency to your application.

As you can see, the problem is that a single GC thread cannot cope with those two operations at the same time. The current ephemeral segment is 16mb (note that this may change in the future so don’t relay on this value!). This means that you can only allocate up to 16mb or whatever is available at the time of allocation, and as you have seen on the example this space may run out before the GC collection finishes! I hope now you understand why we don’t recommend you to call GC.Collect() without a good reason J.

Ok, now let me introduce you to the background GC. This model has been optimized to reduce the latency introduced by the scenario described above. The solution came from the idea of creating a background GC that works as it has described above and a foreground GC that will be only triggered when the ephemeral segment needs to be collected while performing a generation 2 collection.

Now, if we repeat the scenario above and we try to allocate memory on the ephemeral segment while the background GC is marking the foreground GC will execute an ephemeral collection:

The ephemeral foreground thread will mark the dead objects and will swap segments (as this is more efficient rather than copying the objects to generation 2. The ephemeral segment with the free and allocated objects becomes a generation 2 segment.

As you can see now the allocation is allowed and your application will not need to wait for the full GC to finish before allowing you the allocation.

Note that this enhancement is only available on workstation, as the server version is a blocking GC per core and we didn’t have enough time to port these enhancements into it, but is definitely in our plans in the near future. We have tested this solution in 64 cores but our future objective is to hit the 128 core mark as the SQL team has provenJ.

I hope this blog post makes a bit clearer how the GC works and what kind of enhancements we are including in .NET 4.

Working with memory mapped files in .NET 4

Salva Patuel — Mon, 08 Jun 2009 19:47:00 GMT

I have been exploring different new features that come with the .NET 4, beyond the most popular ones like dynamic types and covariance; I was interested in performance enhancements. For this reason I am going to publish a couple of blog entries were I explore these different features.

Memory mapped files may sounds alien to the managed code developer but it has been around for years, what is more, it is so intrinsic in the OS that practically any communication model that wants to share data uses it behind the scenes.

So what is it? A memory mapped file allows you to reserve a region of address space and commit physical storage to a region (hmmm, sounds like virtual memory, isn’t it?) but the main difference is that the physical storage comes from a file that is already on the disk instead of the memory manager. I will say that it has two main purposes:

· It is ideal to access a data file on disk without performing file I/O operations and from buffering the file’s content. This works great when you deal with large data files.

· You can use memory mapped files to allow multiple processes running on the same machine to share data with each other.

The memory mapped file is the most efficient way for multiple processes on a single machine to communicate with each other. What is more, if we check other IPC methods we can see the following architecture:

Impressive, isn’t it? Now you have the power of this technology available on the System.IO namespace (instead of using the Pinvoke approach).

Now let’s quickly explore how it works. We have two types of memory mapped files models, one model is using a custom file, this can be any file that the application accesses it and the other one using the page file that we are going to share it with the memory manager (this is the model that most of the technologies above use).

Let’s explore the custom file model. The first thing that we need to do is to create a FileStream to the file that we are going to use, this can be an existing file or a new file (keep in mind that you should open this file as shared, otherwise no other process will be able to access it!). With the stream in place, we can now create the memory mapped file. Let’s see an example:

using System.IO.MemoryMappedFile;

MemoryMappedFile MemoryMapped = MemoryMappedFile.CreateFromFile(

new FileStream(@"C:\temp\Map.mp", FileMode.Create), // Any stream will do it

"MyMemMapFile", // Name

1024 * 1024, // Size in bytes

MemoryMappedFileAccess.ReadWrite); // Access type

I have use one of the simplest constructor, we define the stream to use and we provide a name. The object needs to know the size of it in bytes and the type of access that we need. This will create the memory mapped file but to start using it we will need a map view. We can create one using the following syntax:

MemoryMappedViewAccessor FileMapView = MemoryMapped.CreateViewAccessor();

This map covers the file from the first byte until the end. If we need now to write or read information from it we just call the map view methods with the correct offset.

int Number = 1234;

FileMapView.Write(0, Number);

FileMapView.Write<Container>(4, ref MyContainer);

We can see that we can write built in types or custom types with the generic version. The good thing about the memory mapped files is persistence, as soon as you close it the contents will be dumped on the disk, this is a great scenario for sharing cached information between applications.

Now if we want to read from it, the other process needs also to create a memory mapped file, we can use the other static initialize that opens an existing one or creates one if it does not exist.

MemoryMappedFile MemoryMapped = MemoryMappedFile.CreateOrOpen(

new FileStream(@"C:\temp\Map.mp", FileMode.Create), // Any stream will do it
"MyMemMapFile", // Name

1024 * 1024, // Size in bytes

MemoryMappedFileAccess.ReadWrite); // Access type

Create the map view and read it:

using (MemoryMappedViewAccessor FileMap = MemoryMapped.CreateViewAccessor())

{

Container NewContainer = new Container();

FileMap.Read<Container>(4, out NewContainer);

}

That’s it, really simple isn’t it? Now, there is a small drawback with this approach and is related to the size of the memory mapped file. If you don’t know in advance you will need to create a large file
“just in case”, this can be a very large file with a lot of wasted space, it would be nice to have the ability to reserve the space instead of committing all of it, isn’t it?

In order to solve this issue you can use the page file, this has the advantage of allowing you to commit data on the fly but introduces another issue: you don’t own the file and the map will last until the last handle is destroyed. But think about it, for certain scenarios this is absolutely valid.

Now if we revisit the sample we will need to change some initialization parameters, for this particular one I will use the full constructor so I can introduce some other features that are also applicable to the custom files one.

MemoryMappedFileSecurity CustomSecurity = new MemoryMappedFileSecurity();

MemoryMappedFile PagedMemoryMapped = MemoryMappedFile.CreateNew(

@"Salvador", // Name

1024 * 1024, // Size

MemoryMappedFileAccess.ReadWrite, // Access type

MemoryMappedFileOptions.DelayAllocatePages, // Pseudo reserve/commit

CustomSecurity, // You can customize the security

HandleInheritability.Inheritable); // Inherit to child process

The memory mapped file security allows you to customize who or which process can have access to the resource, this can be quite important when you want to protect sensitive information and you don’t want other processes changing the file map. You can explore that object and see all the different settings that you can change. The reference is here. If we explore the construction of the memory mapped file we can see that there is no stream, we just name the resource. This will create an association between a section of the file based on the size and the map name, this is how both processes will access the file. Note as well that I am setting the property “DelayAllocatePages”, this implements a pseudo reserve/commit model where we will use the space once is needed. Finally, the other interesting parameter is the handle inheritance model, this will allow to share this resource with a child process if is required.

The access to the file uses the same syntax as the previous example, remember that if you close the memory mapped file this will be non accessible, this issue catches many developer.

Finally, another interesting area is the creation of multiple map views, these can work on the same memory mapped file accessing different areas of the files. This will allow you to properly protect the content and allowing you to synchronize the access.

You can do this defining different offsets and lengths when you create your map view.

MemoryMappedViewAccessor WriteMap = MemoryMapped.CreateViewAccessor(0, 1024,
MemoryMappedFileAccess.ReadWrite);

MemoryMappedViewAccessor ReadMap = MemoryMapped.CreateViewAccessor(1025, 1024,
MemoryMappedFileAccess.Read);

Now you can enjoy the power of this high performance data sharing model on your applications when you compile them with .NET 4.0! Note that this blog is based on beta 1 information, I will check the post once is released.