Maoni's WebLog : Performance

So, what’s new in the CLR 4.0 GC?

maoni — Thu, 20 Nov 2008 04:02:00 GMT

PDC 2008 happened not long ago so I get to write another “what’s new in GC” blog entry. For quite a while now I’ve been working on a new concurrent GC that replaces the existing one. And this new concurrent GC is called “background GC”.

First of all let me apologize for having not written anything for so long. It’s been quite busy working on the new GC and other things.

Let me refresh your memory on concurrent GC. Concurrent GC has existed since CLR V1.0. For a blocking GC, ie, a non concurrent GC we always suspend managed threads, do the GC work then resume managed threads. Concurrent GC, on the other hand, runs concurrently with the managed threads to the following extend:

§ It allows you to allocate while a concurrent GC is in progress.

However you can only allocate so much – for small objects you can allocate at most up to end of the ephemeral segment. Remember if we don’t do an ephemeral GC, the total space occupied by ephemeral generations can be as big as a full segment allows so as soon as you reached the end of the segment you will need to wait for the concurrent GC to finish so managed threads that need to make small object allocations are suspended.

§ It still needs to stop managed threads a couple of times during a concurrent GC.

During a concurrent GC we need to suspend managed threads twice to do some phases of the GC. These phases could possibly take a while to finish.

We only do concurrent GCs for full GCs. A full GC can be either a concurrent GC or a blocking GC. Ephemeral GCs (ie, gen0 or gen1 GCs) are always blocking.

Concurrent GC is only available for workstation GC. In server GC we always do blocking GCs for any GCs.

Concurrent GC is done on a dedicated GC thread. This thread times out if no concurrent GC has happened for a while and gets recreated next time we need to do concurrent GC.

When the program activity (including making allocations and modifying references) is not really high and the heap is not very large concurrent GC works well – the latency caused by the GC is reasonable. But as people start writing larger applications with larger heaps that handle more stressful situations, the latency can be unacceptable.

Background GC is an evolution to concurrent GC. The significance of background GC is we can do ephemeral GCs while a background GC is in progress if needed. As with concurrent GC, background GC is also only applicable to full GCs and ephemeral GCs are always done as blocking GCs, and a background GC is also done on its dediated GC thread. The ephemeral GCs done while a background GC is in progress are called foreground GCs.

So when a background GC is in progress and you’ve allocated enough in gen0, we will trigger a gen0 GC (which may stay as a gen0 GC or get elevated as a gen1 GC depending on GC’s internal tuning). The background GC thread will check at frequent safe points (ie, when we can allow a foreground GC to happen) and see if there’s a request for a foreground GC. If so it will suspend itself and a foreground GC can happen. After this foreground GC is finished, the background GC thread and the user threads can resume their work.

Not only does this allow us to get rid of dead objects in young generations, it also lifts the restriction of having to stay in the ephemeral segment – if we need to expand the heap while a background GC is going on, we can do so in a gen1 GC.

We also made some performance improvement in background GC which does better at doing more things concurrently so the time we need to suspend managed threads is also shorter.

We are not offering background GC for server GC in V4.0. It’s under consideration – we recognize how important it is for server applications (which usually have much larger heaps than client apps) to benefit from smaller latency but the work did not fit in our V4.0 timeframe. For now for server applications, I would recommend you to look at the full GC notification feature we added in .NET 3.5 SP1. It’s explained here: http://msdn.microsoft.com/en-us/library/cc713687.aspx. Basically you register to get notified when a full GC is approaching and when it’s finished. This allows you to do software load balancing between different server instances – when a full GC is about to happen in one of the server instances, you can redirect new requests to other instances.

64-bit vs 32-bit

maoni — Tue, 15 May 2007 16:40:00 GMT

As 64-bit machines become more common, the problems we need to solve also evolve. In this post I’d like to talk about what it means for the GC and the applications’ memory usage when we move from 32-bit to 64-bit.

One big limitation of 32-bit is the virtual memory address space - as a user mode process you get 2GB, and if you use large address aware you get 3GB. A few years these seemed like giant numbers but I've seen as more and more people start using .NET framework, the sizes of the managed heap go up at a quite high rate. I remember when I first started working on GC (which was late 2004 I think) we were talking about hundreds of MBs of heaps - 300MB seemed like a lot. Today I am seeing managed heaps easily of GBs in size - and yes, some of them (and more and more of them) are on 64-bit - 2 or 3GB is just not enough anymore.

And along with this, we are shifting to solving a different set of problems. In CLR 2.0 we concentrated heavily on using the VM space efficiently. We tried very hard to reduce the fragmentation on the managed heap so when you get a hold of a chunk of virtual memory you can make very efficient use of it. So people don't see problems like they have N managed heap segments, are running out of VM, yet many of these segments are quite empty (meaning having a lot of free space on them).

Then you switch to 64-bit. Now suddenly you don't need to worry about VM anymore - you get plenty there. Practically unlimited for many applications (of course it’s still limited – for example if you are running out physical memory to even allocate the datastructures for virtual pages then you still can’t reserve those pages). What kind of differences will you see in your managed memory usage?

First of all, your process consumes more memory - I am sure all of you are already aware of this - the pointer size is bigger - it's doubled on 64-bit so if you don't change anything at all, now your managed heap (which undoubtly contains references) is bigger. Of course being able to manipulate memory in QWORDs instead of DWORDs can also be beneficial –our measurements show that the raw allocation speed is slightly higher on 64-bit than on 32-bit that can be attributed to this.

There are other factors that could make your process consume more memory - for example the module size is bigger (mscorwks.dll is about 5MB on x86, 10MB on x64 and 20MB on ia64), instructions are bigger on 64-bit and what have you.

Another thing you may notice - if you have looked at the performance counters under .NET CLR Memory - is that you are now doing a lot fewer GCs on 64-bit than what you used to see on 32-bit.

The curious minds might have already noticed one thing - the managed heap segments are much bigger in size on 64-bit. If you do !SOS.eeheap -gc you will now see way bigger segments.

Why did we make the segment size so much bigger on 64-bit? Well, remember we talked about in Using GC Efficiently Part 2 how we have a budget for gen0 and when you've allocated more than this budget a GC will be triggered. When you have a bigger budget it means you’ll need to do fewer GCs which means your code will get more chance to run. From this perspective you should get a performance gain when you move to 64-bit - I want to emphasize the “this perspective” part because in general things tend to run slower on 64-bit. The perf benefit you get because of GC may very well be obscured by other perf degrades. In reality many people are not expecting perf gain when they move to 64-bit but rather they are happy with being able to use more memory to handle more work load.

Of course we also don’t want to wait for too long before we collect – we strive for the right balance between memory (how much memory your app consumes) and CPU (how often user threads run).

Difference Between Perf Data Reported by Different Tools – 4

maoni — Fri, 12 Jan 2007 08:03:00 GMT

.NET CLR Memory\% Time in GC counter and !runaway on thread(s) doing GC.

The 2 common ways people use to look at the time spent in GC are the % Time in GC performance counter under .NET CLR Memory, and the CPU time displayed by the !runaway debugger command in cdb/windbg. What do they mean exactly? % Time in GC is calculated like this:

When the n^thGC starts (ie, after the managed threads are suspended and before the GC work starts), we record the timestamp at that time. Let’s call this TA(n);

When n^thGC ends (ie, after the GC work is done and before we resume the managed threads), we record another timestamp. Let’s call this TB(n);

So Time spent in this GC is TB(n) – TA(n). And the time since the last GC ended is TB(n) – TB(n-1). So % Time in GC is (TB(n) – TA(n)) / (TB(n) – TB(n-1)).

Since we only record the timestamps we don’t actually discount the time when the thread was switched out – for example, if you are on a single proc machine and another process has a thread of the same priority (as the thread that’s doing GC) that’s also ready to run it may take away some time from the thread that’s doing the GC. None the less it’s a good approximation.

One common scenario where it’s not a good approximation is when paging occurs. In this case you will see a very high % Time in GC but really the time that’s actually spent doing GC work is low ‘cause most of the time is spent doing IO. To verify if you hit this case you can look at the Memory\Pages/sec to see how much it’s paging.

!runaway is more accurate in the sense that it does record the actual time spent on the threads. However I did observe 2 common mistakes when using !runaway to look at time spent in GC.

1) mistake “time spent on the GC thread(s)” as “time spent in GC”.

Let’s take Server GC as an example. When a GC is needed the GC threads first perform the suspension work. Obviously this takes time. Sometimes it can take quite a bit time if you have many threads or some threads get stuck and it takes a long time to suspend them.

2) using the current !runaway output to judge how much time GC has taken without taking into count that some user threads have died. Besides looking at the User/Kernel Mode time you may want to also look at Elapsed Time. The following output is from one of the issues I looked at:

Elapsed Time

Thread Time

0:2094 0 days 11:57:10.406

1:2098 0 days 11:57:10.152

2:2044 0 days 11:57:09.898

3:27cc 0 days 11:57:09.882

4:20c0 0 days 11:57:09.597

13:810 0 days 11:57:09.565

12:21d4 0 days 11:57:09.565

11:2308 0 days 11:57:09.565

10:1d24 0 days 11:57:09.565

9:23e0 0 days 11:57:09.565

8:24d0 0 days 11:57:09.565

7:2168 0 days 11:57:09.565

6:2134 0 days 11:57:09.565

5:2124 0 days 11:57:09.565

14:570 0 days 11:57:08.582

15:10fc 0 days 11:57:00.000

16:1ac4 0 days 11:56:58.256

17:1900 0 days 11:56:58.176

18:1624 0 days 11:56:57.494

19:11f8 0 days 10:35:01.881

21:1620 0 days 0:41:45.017

22:1cc8 0 days 0:37:30.496

23:2de0 0 days 0:29:01.820

24:2e10 0 days 0:29:01.711

25:2d88 0 days 0:22:26.988

26:1668 0 days 0:19:26.175

27:2cb0 0 days 0:16:16.814

29:2d1c 0 days 0:11:53.779

28:1de8 0 days 0:11:53.779

30:2ed4 0 days 0:11:35.466

32:2030 0 days 0:11:22.544

31:2084 0 days 0:11:22.544

33:12b0 0 days 0:07:53.510

35:28e4 0 days 0:03:13.801

34:2654 0 days 0:03:13.801

36:2d68 0 days 0:02:31.988

The orange lines are GC threads. They were created about 12 hours ago. The green lines are user threads and they were created 2 mins to 40mins ago. So naturally at the current state these user threads couldn’t’ve spent much time.

Difference Between Perf Data Reported by Different Tools – 3

maoni — Wed, 20 Dec 2006 08:51:00 GMT

Both the !SOS.gchandles command (added in CLR 2.0) and the .NET CLR Memory\# GC Handles counter show you the number of GC handles you have in your process.

The # GC Handles counter is one of the rare counters in the .NET CLR Memory category that doesn’t get updated at the end of each GC. Rather we update it in the handle table code, for example, when some code in the CLR calls the function to create a GC handle (possibly because the user has requested to create a handle via managed code), we increase this counter value by one. For performance reasons we don’t use interlocked operations when we need to increase or decrease this value. This means the value can get changed concurrently by multiple threads. For this reason you should always trust the value returned by the !SOS.gchandles command if you ever doubt the counter value. The SOS command is accurate because we always walk the handle table when you issue the command so it returns the # of handles truthfully.

Difference Between Perf Data Reported by Different Tools - 1

maoni — Tue, 12 Dec 2006 10:16:00 GMT

So, there are many perf tools and some of them report either the same or the same type of data. I want to talk about various differences between the ones related to managed heap investigation. This is not supposed to cover everything..just the ones I think people use frequently.

Managed Heap Size

We have both .NET CLR Memory perf counters and SoS extensions that report manged heap size related data.

Difference 1

The values reported by the .NET CLR Memory perf counters are affected by

1) how often they are collected - meaning however often you tell the tool you use (most people use perfmon, internally I know many groups' test teams collect them as part of the automation). The most frequent interval perfmon gives you is once per second.

2) how often they are updated. Most .NET CLR Memory perf counters are updated only at the end of each GC.

So for example, assuming you are using perfmon to collect at one sample per second and if more than one GC happened in the past second, you'll be missing the intermediate values.

And since these perf counters are updated only every so often, before a value is refreshed it stays the same value as it was updated last time. For example, if you are looking at % Time in GC and say the last value was 80% and if no GC happens for 10 seconds, this counter will stay at 80% but really during the past 10 seconds no GCs were happening.

On the other hand, SOS data is only gotten when you request, which means you get the value at the time you did the SOS command (unless of course the command specifically tells you it reflects a value updated at some specific point in the past).

My application seems to hang. What do I do? – Part 2

maoni — Tue, 14 Nov 2006 11:58:00 GMT

Last time I talked about the hang scenario where your process is taking 0 CPU and the CPU is taking by other process(es) on the same machine.

The next scenario is your process is taking 0 CPU and the CPU is barely used by other processes.

As one of the readers correctly pointed out, this is very likely because you have a deadlock. Usually debugging deadlocks is relatively straightforward – you look at what the threads are waiting on and figure out which other threads are holding the lock(s). And there are plenty of online resources that talk about debugging deadlocks. If you use the Windows Debugger package there are built in debugger extension dlls that help you with this like !locks and etc. If you are debugging a managed app the SoS debugger extension has commands that will aid you - !SyncBlk shows you managed locks (for CLR 2.0 there’s also !Dumpheap –thinlock for objects locked with ThinLocks instead of SyncBlk’s).

Another possibility is your process is not doing any CPU related activities. A common activity is IO – for example if the process is heavily paging you will see almost 0 CPU usage but it appears hang because the memory it needs is getting loaded from the disk which is really slow. A very useful tool that shows you what processes are doing is Process Monitor. Yesterday a program on my machine paused periodically – very annoying. So I used process monitor which showed me that this program periodically checks if I am logged onto my account in another program and since I am not, it would log me on, does a little bit stuff then log me off. And the hang was due to waiting on network IO. So to make it happy I logged myself on then the annoying periodic hang disappeared.

Now if your process is indeed taking CPU it can also appear to hang – as I mentioned last time this means different things for different people. If you have a UI app this can mean the UI is not getting drawn; if you have a server app this can mean your app is not processing requests. So you’ll have to define what hang means to you. I will use server app not processing requests as an example. Usually server applications run on dedicated machines. So let’s assume that’s the case here – you run a server on a machine and the server could consist of multiple processes. You measure the server performance by throughput. One scenario is the CPU usage is high (perhaps even higher than usual) but the throughput is lower than usual.

The easiest case, as one of the readers pointed out, is an infinite loop – very easy to debug. You break into the debugger a few times and see a thread is taking all the CPU and that thread can not exit some function – so there goes your infinite loop. And if your process is pretty much the only process that uses CPU at the time this is super obvious. It gets a bit more complicated if you have multiple CPUs and other processes are also using CPUs. But still since it’s an infinite loop the nice thing is it will always be executing if you don’t interfere so it’s always available to you to investigate as soon as it happens.

It becomes hard if the hang only reproes sporadically and when it reproes it only lasts for a little while. Time to whip out a CPU profiler. As another reader pointed out, Process Explorer is a useful tool to get you started. It shows you which processes are “active” – meaning it’s using CPUs. Personally I start with collecting appropriate performance counters, partially because pretty much all test teams in the product groups at Microsoft have some sort of automated testing procedure that collects perf counters so requesting them is easy. And because of the low overhead you can collect them for a long period of time so you have a histogram.

These are the counters I usually request (comments in []’s):

Processor\% Processor Time for _Total and all processors

[This is so I have an idea what kind of CPU usage I am looking at and if there are paticular processors that get used more than the rest]

Process\% Processor Time for all processes or less the ones that you already know can not be the problem

Thread\% Processor Time for all processes or less the ones that you already know can not be the problem

[The above counters will tell you which threads are using the CPU so you know which threads to look at]

[Since I usually look at GC related issues I request all counters under .NET CLR Memory]

.NET CLR Memory counters for all managed processes or less the ones that you already know can not be the problem

[If you are looking at other things you should add appropriate counters – for example, ASP.NET counters for apps that use ASP.NET]

[If you know the kind of activities your processes do you can add appropriate counters for them. For me I often request memory related counters like:]

Memory\% Committed Bytes In Use

Memory\Available Bytes

Memory\Pages/sec

Process\Private Bytes for processes I am interested in

…

At this point I can look at the results and concentrate on the interesting parts – for example when the CPU is usually high. I will have an idea which threads in what processes are consuming the CPU and the aspects of them that interest me (usually GC and other memory activities). Then I can request more detailed data on those processes/threads. For example I can ask the person to use a sampling profiler so I can see what functions are executing in the part I am interested in (along with other info – this depends on what the profiler you are using is capable of).

Some people prefer to take memory dumps when the process hangs, sometimes this doesn’t necessarily work (when it works it’s great) because if the hang is related to timing/how threads are scheduled the threads can easily behave differently when you interrupt it in order to take memory dumps so the hang may not repro anymore. If you do have consecutive dumps from one hang then you can use the !runaway command to see which threads have been consuming CPU. One dump is hardly useful for debugging hangs because it only gives you info at one point in time how the process behaves.

New MSDN Article - Investigating Memory Issues

maoni — Mon, 23 Oct 2006 08:35:00 GMT

We have a new MSDN article out in the November issue that talks about investigating managed memory issues.

http://msdn.microsoft.com/msdnmag/issues/06/11/CLRInsideOut/default.aspx?loc=en

Take a look and let me know what you think.

Oh, and it's also in 6 other languages (German, Spanish, French, Russian, Portuguese and Chinese) for readers that prefer one of those languages.

My application seems to hang. What do I do? – Part 1

maoni — Thu, 28 Sep 2006 11:29:00 GMT

Defining “hang” is a good place to start.

When people say “hang” they could mean all sorts of things. When I say “hang” I mean the process is not making progress – the threads in the process are either blocked (eg. deadlocked, or not scheduled because of threads from other processes) or executing code (madly) but not doing useful work (eg. infinite loop, or busy spinning for a long time without doing useful work). The former uses no CPU while the later using 100% CPU. When a UI developer says “hang” he could mean “the UI is not getting drawn” so essentially they mean the UI threads are not working – other threads in their process could be doing lots of work but since the UI is not getting updated it appears “hang”. So clarifying what you mean when you say “hang”, which requires you to look at your process and its threads, is the first step.

If you start Task Manager (taskmgr.exe) it shows you how much CPU each process is using currently. If you don’t see a CPU column you can add it by clicking View\Select Columns and check the “CPU Usage” checkbox.

Note that if you have multiple CPUs, the CPU usage is at most 100. Let’s say you have 4 CPUs and your process has one thread that’s running and taking all the CPU it can you will see the CPU column for your process 25 – since your process can only use one CPU (at most to its full) at any given time.

The CPU usage for a process is calculated as the CPU usage used by all the threads that belong to the process. Threads are what get to run on the CPUs. They get scheduled by the OS scheduler which decides when to run what thread on which processor. I won’t cover the details here – the Windows Internals book by Russinovich and Solomon covers it.

If you see your process is taking 0 CPU, that would explain why it’s hung (for the period of time when the CPU keeps being 0) – no threads are getting to run in your process! The next thing to look at is the CPU usage of other processes. If you see one or multiple other processes that take up all the CPU that means the threads in your process simply don’t get a chance to run – this is because the threads in those other processes are of higher priorities (or temporarily of higher priorities due to priority boosting) than the threads in your process. The possible causes are:

1) there are threads that are marked as low priority which acquired locks that other threads in your process need in order to run. And the low priority threads are preempted by other (normal or high) prority threads from those other processes. This happens when people mistakenly use low priority threads to do “unimportant work” or “work that doesn’t need to be done in a timely fashion” without realizing that it’s nearly impossible to avoid taking locks on those threads. I’ve heard of many people say “but I am not taking a lock on my low priority threads” which is not a valid argument because the APIs you call or the OS services you use can take locks in order to run your code – allocating on native NT heap can take locks; even triggering a page fault can take locks (which is not something an application developer can control in his code).

2) the threads in your process are of normal priority but those other processes have high priority threads – this should be relatively easy to diagnose (and unless some process is simply bad citizens this rarely happens) – you can take a look at what those processes are doing (again looking at their threads’ callstacks is a good place to start).

That’s all for today. Next time I will talk about other hang scenarios and techniques to debug them.

Understand the problem before you try to find a solution

maoni — Fri, 01 Sep 2006 10:15:00 GMT

So far I’ve never written a blog entry that gives out philosophical advices on doing performance work. But lately I thought perhaps it’s time to write such an entry because I’ve seen enough people who looked really hard at some performance counters (often not correct ones) or some other data and asked tons of questions such as “is this allocation rate too high? It looks too high to me.” or “my gen1 size is too big, right? It seems big…”, before they have enough evidence to even justify such an investigation and questions.

Now, if you are just asking questions to satisfy your curious mind, that’s great. I am happy to answer questions or point you at documents to read. But for people who are required to investigate performance related issues, especially when the deadline is close, my advice is “understand the problem before you try to find a solution”. Determine what to look at based on evidence, not based on your lack of knowledge in the area unless you already exhausted areas that you do know about. Before you ask questions related to GC, ask yourself if you think GC is actually the problem. If you can’t answer that question it really is not a good use of your time to ask questions related to GC.

I’ve seen many cases when something seems to go wrong in a managed application, people immediately suspect GC without any evidence to support that suspicion. Then they start to ask questions – often very random – hoping that they would somehow find the solution to fix their problem without understand what the problem is. That’s not logical is it? So stop doing it!

So how do you know the right problems to solve, I would recommend the following:

1) Having knowledge about fundamentals really helps.

What are fundamentals? Performance in general comes down to 2 things – memory and CPU. Knowing basics of these 2 areas helps a lot in determining which area to look at. Obviously this involves a lot of reading and experimenting. I will list some memory fundamentals to get you started:

Some fundamentals of memory

· Each process has its own and separated virtual address space; all processes on the same machine share the physical memory (plus the page file if you have one). On 32-bit each process has a 2GB user mode virtual address space by default.

· You, as an application author, work with virtual address space – you don’t ever manipulate physical memory directly. If you are writing native code usually you use the virtual address space via some kind of win32 heap APIs (crt heap or the process heap or the heap you create) – these heap APIs will allocate and free virtual memory on your behalf; if you are writing managed code, GC is the one who allocates/frees virtual memory on your behalf.

· Virtual address space can get fragmented – in other words, there can be “holes” (free blocks) in the address space. When a VM allocation is requested, the VM manager needs to find one free block that’s big enough to satisfy that allocation request – if you only got a few free blocks whose sum is big enough it won’t work. This means even though you got 2GB, you don’t necessarily see all 2GB used.

· VM can be in different states – free, reserved and committed. Free is easy. The difference between reserved and committed is what confused people sometimes. First of all, you need to recognized that they are different states. Reserved is saying “I want to make this region of memory available for my own use”. After you reserve a block of VM that block can not be used to satisfy other reserve requests. At this point you can not store any of your data in that block of memory yet – to be able to do that you have to commit it, which means you have to back it up with some physical storage so you can store stuff in it. When you are looking at the memory via perf counters and what not, make sure you are looking at the right things. You can get out of memory if you are running out space to reserve, or space to commit.

· If you have a page file (by default you do) you can be using it even if your physical memory pressure is very low. What happens is the first time your physical memory pressure gets high and the OS needs to make room in the physical memory to store other data, it will back up some data that’s currently in physical memory in the page file. And that data will not be paged in until it’s needed so you can get into situations where the physical memory load is very low yet you are observing paging.

2) Knowing what your performance requirements are is a must.

If you are writing a server application, it’s very likely that you want to use all the memory and CPU that’s available because people delicate the machine completely to run your app so why waste resources? If you are writing a client application, totally different story – you’ll have to know how to cope with other applications running on the same machine. There’re no rules such as “you have to make your app use as little memory as possible”.

When you’ve decided that there is a problem, dig into it instead of guess what might be wrong. If your app is using too much memory, look at who is using the memory. If you’ve decided that the managed heap is using too much memory, look at why. Managed heap using too much memory generally means you survive too much in your app. Look at what is holding on to those survivors.

When memory is running low…

maoni — Tue, 06 Jun 2006 22:49:00 GMT

When I say memory I mean physical memory. Let’s assume that you have enough virtual memory space. When the physical memory gets low you may start getting OOMs or start paging.

You can experiment and see how you can avoid getting into the low memory situation but sometimes it’s hard to predict and hard to test all scenarios your app can get into. For example, if your app handles requests where the amount of memory each request consumes can vary dramatically, you may be able to handle 1000 concurrent requests some times before running out of memory but only 800 other times.

First of all, let’s talk about how you know when the memory is low on the machine. You can do this by either calling the Win32 API GlobalMemoryStatusEx, or monitor memory performance counters. Currently there’s no notification sent by the GC for low memory – one reason being the memory is used not only by the managed heap but also by native heaps/modules loaded/etc.

You can get the process info via either Win32 APIs, perf counters or the managed Process class.

To get the amount of memory that the managed heap is using, the most accurate way is to monitor the “# Total committed bytes” performance counter under .NET CLR Memory.

Something that server throttling code usually does is to monitor the memory usage and when the memory consumption gets high it stops accepting new requests and waits for the current requests to drain. When the memory load gets to a reasonably low point it starts accepting new requests again.

But the problem is, if the current requests don’t make any allocations or don’t make enough allocations to trigger a full GC your managed heap doesn’t have a chance to collect dead objects so after the current requests drain the memory consumption does not go down.

What do you do in this case? You can call GC.Collect. This is a legitimate case for calling GC.Collect yourself – but be careful how you call it! There are a few things you should keep in mind when calling GC.Collect:

Don’t call it too often -

you want to set a minimum interval of how often you call it. This would depend on your application. Let’s say you have 500MB worth of live objects to collect (meaning the resulting heap size after GC is about 500MB, not counting the garbage), perhaps you want to not call GC.Collect more often than every 5 seconds. You can determine this by doing a simple test on the type of machines that your app usually runs on and see how long it takes to collect certain number of bytes.
GC happens on its own schedule so you want to account for that. When it’s the time for you to induce a GC a GC may have already happened during the last interval so it’s not necessary for you to induce the GC. You can check for this by calling GC.CollectionCount(GC.MaxGeneration) after the interval elapses and if the collection count has already increased from the last time you induced a GC you don’t need to induce another one.

Don’t call it when it’s not productive – if after you call GC.Collect and the memory doesn’t drop by much, you want to call it less often. It’s a good measure to check say, if the memory hasn’t dropped by 1% of the total memory you got on the machine. If that’s the case perhaps you want to increase the interval by some percentage.

Recently I worked with a team that implemented just that and got good results.

Some people have asked why we don’t just give you a way to limit the amount of memory that a process is allowed to use. Imagine if we did, how would you use it? Do you set the amount of memory that your app is allowed to use to 90% of the physical memory? 90% sounds good… now, if everyone was to take this suggestion and set their app to use 90% of the memory we are back to square one. If everyone sets it to 50% and if your app is the only active app on the machine you’d be wasting half of the memory.

If your app is running in a very controlled environment, in other words, you are probably using the hosting APIs to gain more control in a managed app, there are hosting APIs you can use to limit the amount of memory that managed heap can use. You can use IGCHostControl::RequestVirtualMemLimit to deny the request when GC needs to acquire a new segment. So you can check for the amount of memory that managed heap uses and if it exceeds the limit you set, you use this API to tell GC to not add new segments. The result of this is there will be a GC trigger trying to get back more memory. If after GC is done there’s still no space to satisfy your allocation request you will get an OOM. Note that this means you need to be prepared to handle OOMs which can be very difficult or even impossible. So to be able to use this API in an appropriate way generally means you’ll need to use other hosting APIs to do things like communicating the memory load to GC (memory load that you defined), tracking allocations and terminating tasks or unloading app domains. Refer to the hosting API documentation for details.

BTW, if you really want to limit the memory for a process, you even have a way to do it via Win32 APIs. You can create a job object via the CreateJobObject call that you associate with your process, and specify the amount of memory the process is allowed to use (or if you have only one process associated you can set the memory limit for the job) via SetInformationJobObject.

Now, if your application simply needs to use a lot more memory than the amount of physical memory, what do you do? Currently (meaning, on CLR 2.0 or earlier) if you need to constantly churning through all this data, it would mean you will incur full GCs often. If a large portion of the managed heap is stored in the page file and would need to be paged in to do a GC, you will take a pretty serious performance hit. It may be worse of the alternative which is storing it as native data and only page in what you need. You will still need to go to the page file very often but you don’t need to pay the price of paging in everything to do a GC. So right now doing caching in native code (in memory or in a file) could be a better choice, provided that the cache implementation has very low fragmentation. Sorry I don’t have a better answer but we are perfectly aware of this problem though and are working hard on it.

Large Object Heap

maoni — Wed, 19 Apr 2006 07:51:00 GMT

LOH (Large Object Heap) contains objects that are 85,000 bytes or bigger (there’s also some objects that are less than 85,000 bytes that are allocated on the LOH by the runtime itself but usually they are very small and we’ll ignore them for this discussion).

The way LOH is implemented changed dramatically from 1.0 to 1.1. In 1.0 we used a malloc type of allocator for allocating large objects; in 1.1 and beyond we use the same allocator for both large and small objects: we acquire memory from the OS by heap segments and commit on a segment as needed. There’s very little difference between 1.1 and 2.0 for LOH.

LOH is not compacted – however if you want a large object to be pinned you should make sure to pin it because whether LOH is compacted or not is an implementation detail that could be changed in the future. Free blocks between live large objects are threaded into a free list which will be used to satisfy large object allocation requests. Note that this is a real advantage for managed heap – we are able to be very efficient to manage fragmentation because we can coalesce adjacent free objects into a big free block. If a free block is too large we call MEM_RESET on it to tell the OS to not bother backing it up. So far I haven’t heard of any fragmentation problems from production code (of course you write a program that specifically fragments the LOH) – I think one reason is we are doing a good job controlling fragmentation; the other reason is people usually don’t churn LOH too much – one typical pattern I’ve seen people use LOH is to allocate some large arrays that hold on to small objects.

When I talked about weak references in Using GC Efficiently – Part 3, I mentioned a performance implication of using weak refs which is when you use them to refer to really small objects that are comparable in size. Make sure that you are fine with the ratio of the space that weak ref objects take up over the size of objects that they refer to. There’s no difference between using a weak reference to refer to a large and a small object aside from the fact that by definition this ratio is naturally smaller for large objects because of their sizes.

In this comment, it was asked why “explicitly setting a reference to null would prevent the LOH from growing faster than the collection rate”. Actually I think what likely happened was the large objects were held live by something therefore GC was not reclaiming the memory. By setting these objects to null just lets GC know that those objects are garbage now and can be reclaimed. You can use either CLRProfiler or the !SOS.gcroot command to verify this.

Workstation GC for server applications?

maoni — Wed, 01 Mar 2006 09:43:00 GMT

In Using GC Efficiently – Part 2 I talked about different flavors of GC that exist in the CLR and how you choose which flavor is good for your applications, and I said that the Server GC flavor is designed for server applications. As with any performance tuning there are always exceptions – there’s no one-rule-fits-all. Recently I worked with an internal group at MS on their 64-bit server scenario and it’s one of those situations where you do want to try Workstation GC for server applications so I thought it’d be interesting to talk about it here. The scenario is they are running 12 ASP.NET server processes on a 4-proc machine. Since they are using ASP.NET they get Server GC by default.

First of all, let’s review how the server GC works. It creates a GC thread for each CPU you have on the machine and each GC thread collects parts of the GC heap. This means when you have 12 processes you get 48 GC threads. Imagine if all of the processes start doing GCs, there are 48 threads to schedule. In a managed process, the GC threads need to rendezvous at some points. Imagine if in the first process, one of the 4 threads happens to not be scheduled till the GC threads from the other 11 processes are scheduled, the first process will need to wait 11*4=44 quatums.

Keep in mind that it’s very common to run only one server process on a dedicated machine so you don’t get into this situation. But there are legitimate reasons when you do want to run multiple server processes on the same machine. For example, finacial cost for multiple machine, adminitrative cost to manage multiple machines or all the servers need to share the same IP address (if they all server the same web site).

So when would you get into the situation where all processes start doing GCs at the same time? One of the common attributes of server applications is they try to use as much resource as possible on the machine. This is something that many people find hard to accept ‘cause we normally talk about how to make your app use less resource. When you dedicate a machine to running a specific application (+ the normal processes that the OS runs), you do want that app to use as much resource as possible otherwise the rest of the resource would be wasted. So normally server applications try to maximize their memory usage. As I mentioned in Using GC Efficiently – Part 1 one of reasons a GC would be triggered is when the machine is in low memory situation. So if you run multiple managed server applications they could all start to GC when the memory gets low. Note that this is not a specific problem to managed applications – native apps share the same problem because when the memory gets low they all start cleaning up processs which means they will all compete for CPU.

An absolute cricial thing to developing a server application is throttling. Unfortunately this is not known to everyone who develops server apps. Especially when developing managed server apps, some people think that GC should just take care of that aspect not realizing that GC manages memory, not requests. If your machine has 2GB of memory and each request takes 1MB of memory, it can’t handle more than 2048 requests concurrently. If you keep accepting requests as fast as they come in without any kind of throttling you will run out of memory when the number of requests exceeds what your machine can handle.

Some people think Server GC collects faster due to multiple GC threads working together so that would be one reason to choose Server GC over Workstation GC. This is true when you are collecting the same size of heap. Without throttling, however, you could have a bigger heap to collect because Server GC is more efficient (scalable on MP) so more time is spent accepting new requests if you accept much more concurrent requests you could end up using a lot more memory so you may not observe less time in GC simply because now GC has a lot more work to do.

Workstation GC with Concurrent GC off, on the other hand, runs on the user thread that triggered the GC and doesn’t create addition threads. When you run many processes on the same machine this can dramatically cut down the number of threads. As a matter of fact, our internal testing showed with many ASP.NET server processes running on one machine Workstation GC (no Concurrent GC) proved to be a good choice.

So, what’s new in the CLR 2.0 GC?

maoni — Tue, 04 Oct 2005 09:30:00 GMT

Certainly that’s one of the most frequently asked questions I get (at the PDC too!). So since PDC already happened I can tell the rest of you about the new stuff happened in GC in CLR 2.0. The slides can be downloaded here. And I will be referring to some of the slides. I must apologize for your having to click on the link to see the slide each time I refer to one since I don’t have a separated site where I can store pictures.

Most of what I am going to talk about was covered in my PDC GC talk. BTW, just a few words about the 2005 PDC (skip this paragraph if you are interested in only technical stuff :-)). I was very pleased with the way it went. I got to talk to lots of customers who told me how awesome our GC was which certainly made me feel very good! There were a couple of customers who did tell us about the problems they had with our GC. One problem we already addressed in Whidbey and the other one was something we were perfectly aware of and had put on our todo list. So no surprises there. I was happy to see the number of people at my talk considering it was the latest possible and lots of people had already left the conference before that. So all in all it was great.

Some terminology before we start:

Ephemeral generations – gen0 and gen1.

Ephemeral segment – the segment that gen0 and gen1 live in (they always live in one segment) and there can only be one ephemeral segment for each heap. So for server GC, if we have 4 heaps we have 4 ephemeral segments (refer to my Using GC Efficiently – Part 2 for explaination for different flavors of GCs).

Small object heap – you know LOH (if you don’t, it’s covered in Using GC Efficiently – Part 1), so the rest of the managed heap is called small object heap :-)

Fragmentation control

If you ask me what was the most significant improvement in the CLR 2.0 GC from the 1.1 GC, I’d have to say it’s the work we did in reducing fragmentation caused by pinning.

One of the biggest problems we faced for managed applications was wasted free space on the managed heap. And sometimes you get OOM while there’s plenty of free space on the managed heap. So reducing fragmentation, in other words, having less space wasted on the managed heap, was crucial. It was observed often in applications that perform IO which needs to pin the buffers for the native APIs to read or write to them. As such, we’ve done a lot of work in CLR 2.0 to help solve this

problem.

Slide 25 demonstrates the fragmentation problem caused by pinning. Let’s say we have an application where each request performs some type of socket or file IO. Before GC number X we have a heap that has one segment, which is our ephemeral segment, has bunch of stuff allocated in gen0, including some pinned objects. Now a GC is triggered. After the collection what survived from gen1 was promoted to gen2. For a simplistic view let’s just say nothing from gen0 except the pins survived. In reality what survived is the pinned objects and the other data associated with the requests that pinned those objects for this scenario. So after the collection, gen1 just has the pins ‘cause that’s all that survived from gen0. Gen0 then starts right after the 2^nd pin.

You can imagine that if this process keeps repeating itself, and let’s say that the pins survived N more GCs, after these GCs we may have a situation where we have expanded the heap and the old ephemeral segment now is a gen2 segment and we acquired a new ephemeral segment. Again we start the new gen0 after the pins in the ephemeral segment.

This is a really bad situation because we now have lots of wasted space on our heap. Both our gen2 and gen1 look quite empty. Obviously the less wasted space we can make it, the better.

The fragmentation reduction work can be categorized into 2 things: one is called demotion which is to prevent fragmentation from getting into higher generation as much as we can; the other one is reusing existing gen2 segments so we can reuse the free space that has already made into gen2. Let’s talk about them in more details.

Demotion, as the opposite of promotion, means the object doesn’t end up in a generation that it’s supposed to be in. Imagine if after the compaction, there’s plenty of space between some pinned objects at the end of the ephemeral segment, it’s more productive to leave them in gen0 instead of promoting them to gen1 because when allocation requests come in the free space before the pins can be used to satisfy allocations. This is demonstrated with slide 26.

Segment reuse is relatively straightforward. It’s to take advantage of the existing gen2 segments that have a lot free space but not yet empty (because if they were empty they would have been deleted). Slide 27 demonstrates the problem without segment reuse. So before GC we are running out of space in the ephemeral segment and we need to expand the heap. We have 2 pinned objects. And I didn’t mark their generations because it’s not significant to indicate their generations - they can not be moved so they will stay in this segment and become gen2 objects by definition. There might be pinning on the gen2 segments as well but since that doesn’t affect illustrating the point I preferred to keep the picture simple and left them out.

So after GC we allocate a new segment. The old gen1 in the old ephemeral segment was promoted to gen2 and the old gen0 now lives in the new ephemeral segment as the new gen1 and we will start allocating after gen1 on this segment. The pinned objects are left in the old ephemeral seg since they can not be moved. They are part of gen2 now because they live in a gen2 segment.

Slide 28 demonstrates segment reuse. Same picture for before GC. During GC we found that segment 3 is empty enough so instead of allocating a new segment we decide to reuse this segment as the new ephemeral seg. The old ephemeral segment becomes a gen2 segment – as we said there can only be one ephemeral segment – and seg3 becomes the new ephemeral segment. The old gen0 now lives in this segment as the new gen1 and again, we start allocating in gen0 after that.

Fixed premature OOM bugs

Premature OOM means you have lots of free space yet you are getting an OOM exception. As I said above, having fragmentation can be a form of premature OOM because you can get OOM while you still have lots of free space on your managed heap. Besides that we also fixed some other premature OOM bugs so if you were getting premature OOM I say definitely try CLR 2.0 out if possible. The premature OOM bugs included bugs for both large and small object heap.

VM hoarding

First of all, this is a feature I would recommand you to not use unless absolutely necessary. The reason we added this feature was for 2 situations – one is when segments are created and deleted very frequently. If you simply can not avoid this, you can specify the VM hoarding feature which instead of releasing the memory back to the OS it puts the segment on a standby list. Note that we don’t do this for the segments that are larger than the normal segment size (generally this is 16MB, for server GC the segments are larger). We will use these segments later to satisfy new segment requests. So next time we need a new segment we will use one from this standby list if we can find one that’s big enough.

This feature is also useful for apps that worry about fragmenting the VM space too much and want to hold onto the segments that they already acquired, like some server apps that need to frequently load and unload small DLLs and they want to keep what they already reserved so the DLLs don’t fragment VM all over the place.

Since the feature should be used very conservatively it’s only available via hosting API – you can turn it on by specifying the STARTUP_HOARD_GC_VM flag.

That’s all I wanted to talk about for this blog entry. There were of course other changes and it’s not possible to list them all but I think those are (mainly the first two) what affect users the most.

GC talk at the 2005 PDC

maoni — Thu, 28 Jul 2005 01:31:00 GMT

I will be giving a GC talk at the PDC this September. This talk is to give you a close up view of the CLR GC so I hope to see all you hard core .NET developers there!

I will talk about some internal details of generations, allocations, different flavors of GC and fragmentation (what we have done in the GC and what you can do in your applications to reduce it). I'll also mention some advanced programming techniques to improve your managed memory usage.

Using GC Efficiently – Part 4

maoni — Sat, 07 May 2005 06:15:00 GMT

In this article I’ll talk about things you want to look for when you look at the managed heap in your applications to determine if you have a healthy heap. I’ll touch on some topics related to large heaps and the implications you want to be aware of when you have an application that maintains or has potential for the need to maintain a managed heap of large sizes. For many people, it’s always a concern when they will get the Out of Memory exception when they have a heap that’s too big.

Lately I’ve been working on Whidbey Beta2 and Beta2+ (which was one reason why I haven’t posted in a long time) and for the past couple of months I’ve looked at many many memory dumps from various internal teams. Many teams needed help on identifying how healthy their heaps were and what factors would affect the heap usage so I thought I’d share some of this info with you.

How GC is concerned with virtual Memory and physical memory

Some of you probably are already perfectly familiar with this so you can just skim through this section.

GC needs to make allocations for its segments. For an explanation on segments please see Using GC Efficiently – Part 1. When we decide that we need to allocate a new segment, we call VirtualAlloc to allocate space for this segment. This means if there isn’t a contiguous free block in the virtual memory in your process’s address space that’s large enough for a segment, we will fail the allocation for the segment therefore fail the allocation request. This is one of the very few legitimate situations for GC to throw OOM at you (to be accurate, GC actually doesn’t throw any exceptions – it’s the execution engine that makes the allocation request on your behalf that throws the exception – GC just returns NULL to the allocation request).

I often get asked this question, “why do I get OOM when my heap is only X MB??” where X is a lot smaller than 2GB. Note this is on x86 and of course by “heap” they mean managed heap (I should be proud that many people are so customized to managed apps these days that they just use “heap” to refer to only managed heap J).

Remember that there are always allocations that are made not by GC. GC competes for the VM space just like anything else. Modules you load in your process need to take up VM space; some modules in your process could be making native allocations which also take up VM space (VirtualAlloc, HeapAlloc, new and whatnot). CLR itself makes native allocations as well for jitted code, datastructures the CLR needs (including the ones that GC needs to do its work) and etc. Usually the allocations CLR makes should be pretty small. You can use the !eeheap command from the SOS debugger extension dll to look at various categories of allocations that the CLR makes. Following is a simplified sample output with some comments in []’s:

The app is getting OOM at this point. Let’s look at the free VM blocks. You can achieve this by using the !vadump command or any other tool that analysises the VM space. My favorite is the !address command which nicely prints out the largest free region:

0:119> !eeheap
Loader Heap:
--------------------------------------
System Domain: 79bbd970
LowFrequencyHeap: Size: 0x0(0)bytes.
HighFrequencyHeap: 007e0000(10000:2000) Size: 0x2000(8192)bytes.
StubHeap: 007d0000(10000:7000) Size: 0x7000(28672)bytes.
Virtual Call Stub Heap:
IndcellHeap: Size: 0x0(0)bytes.
LookupHeap: Size: 0x0(0)bytes.
ResolveHeap: Size: 0x0(0)bytes.
DispatchHeap: Size: 0x0(0)bytes.
CacheEntryHeap: Size: 0x0(0)bytes.
Total size: 0x9000(36864)bytes
--------------------------------------
Shared Domain: 79bbdf18
...
Total size: 0xe000(57344)bytes
--------------------------------------
Domain 1: 151a18
...
Total size: 0x147000(1339392)bytes
--------------------------------------
Jit code heap:
LoaderCodeHeap: 23980000(10000:7000) Size: 0x7000(28672)bytes.
...
Total size: 0x87000(552960)bytes
[jited code takes very little space – this is a fairly large application]
--------------------------------------
Module Thunk heaps:
Module 78c40000: Size: 0x0(0)bytes.
...
Total size: 0x0(0)bytes
--------------------------------------
Module Lookup Table heaps:
Module 78c40000: Size: 0x0(0)bytes.
...
Total size: 0x0(0)bytes
--------------------------------------
Total LoaderHeap size: 0x1e5000(1986560)bytes [total Loader heap takes < 2MB]
=======================================
Number of GC Heaps: 4
------------------------------
Heap 0 (0015ad08)
generation 0 starts at 0x49521f8c
generation 1 starts at 0x494d7f64
generation 2 starts at 0x007f0038
ephemeral segment allocation context: none
[The first 2 segments are read only segments for frozen strings which is why they look a bit odd compared to other segments. The addresses for begin and segment are very different and usually they are tiny segments (unless you have tons and tons of frozen strings)]
segment begin allocated size
00178250 7a80d84c 7a82f1cc 0x00021980(137600)
00161918 78c50e40 78c7056c 0x0001f72c(128812)
007f0000 007f0038 047eed28 0x03ffecf0(67103984)
3a120000 3a120038 3a3e84f8 0x002c84c0(2917568)
46120000 46120038 49e05d04 0x03ce5ccc(63855820)
Large object heap starts at 0x107f0038
segment begin allocated size
107f0000 107f0038 11ad0008 0x012dffd0(19791824)
20960000 20960038 224f7970 0x01b97938(28932408)
Heap Size 0xae65830(182868016)
------------------------------
Heap 1 (0015b688)
...
Heap Size 0x7f213bc(133305276)
------------------------------
Heap 2 (0015c008)
...
Heap Size 0x7ada9ac(128821676)
------------------------------
Heap 3 (0015cdc8)
...
Heap Size 0x764c214(124043796)
------------------------------
GC Heap Size 0x21ead7ac(569038764) [total managed heap takes ~540MB]

0:119> !address
...
Largest free region: Base 54000000 - Size 03b60000

0:119> ? 03b60000
Evaluate expression: 62259200 = 03b60000

The largest free region is <64MB which explains why we are getting OOM exception – we were trying to allocate another GC segment but failed to do so (this is on Server GC and segment size is bigger).

The VM space can get badly fragmented if you load and unload variable sized modules often. A common scenario is if you have COM DLLs that get unloaded after the components in them are not in use anymore for a certain period of time (10mins I think?). Other common scenarios are tons of interop assemblies or tons of tiny DLLs for a web server (if you have a 10k DLL it’ll consume 64k in VM).

Besides allocating VM space for segments there’s really nothing else about VM that GC is concerned with. Physical memory, on the other hand, is a totally different story. As I mentioned in Using GC Efficiently – Part 1, one of the 3 situations where a GC can happen is if your machine is low on physical memory. GC becomes more aggressive. So if you are looking at the GC performance counters you might notice that you have a different gen0, gen1 and gen2 ratio when your machine is low on memory. If GC fails to commit memory it needs to satisfy your allocation request you will also get OOM. The available physical memory can be obtained via the Memory\Available Bytes perfmon counter.

A few things worth mentioning are:

1) Nowadays it’s not uncommon to have a machine with more than 2GB of physical memory. So not fragmenting VM too badly is really important. However if you are running on 64-bit where the VM space is huge physical memory becomes the limiting factor.

2) You can use the Whidbey hosting API to communicate to GC when you want it to think you are in a low memory situation even though you really are not because either you want to only allocate part of the physical memory to this particular app, or you want to be on the safe side and not worry about getting to the point where you might start getting OOMs (many apps cannot handle OOMs).

Looking at a managed heap

Before we get into that there are a few things I’d like to mention about taking the perf data for your heap:

Getting Perfmon logs

I recommend when you take GC perfmon counters you use the shortest interval possible (right now it’s 1 second) and capture the perfmon log for a few minutes. Usually it gives you much more insight than capturing a log for 2 hours with 5 or 10 seconds interval (which I often see people do when they send me perfmon logs) because if something is misbehaving it’s usually enough to see the pattern after a few minutes. However if the problem repros randomly of course you have no choice but to log for a possibly long time but it’s best to keep the interval short because GC usually happens frequent enough that you need a short interval to make more sense out of the log.

Taking memory dumps

If you take a memory dump for your process, take a full memory dump. A mini dump for analysing the managed heap is usually useless.

Getting CLRProfiler logs

CLRProfiler is very heavy weight and it can’t attach to a process but you can uncheck the Profiler Active checkbox so it’s not on when the app is not running to the point that you are interested in gathering data. You can also uncheck the Calls checkbox to gather less data if just profiling Allocations is enough for you.

-----------------------------------------------------------

So what kind of things do you want to look for when you look at a managed heap?

Time in GC

If the % Time in GC is low, it’s not useful to look for allocation related issues. When % Time in GC is high it usually means you are doing too many Gen2 collections or each Gen2 collection takes too long. Then it’s time to start looking at your heap in more details to determine why you are doing so many Gen2 collections and how you can change your allocation pattern to spend less time on Gen2 collections.

The heap growth pattern

If you have a server app that needs to stay running for a long time, and if you observe that the heap is growing unbounded overtime that’s definitely a bad sign. Usually server apps should clean up all data related to a request after the request is finished. If that’s not the case you’ve got a memory leak that you must look at – otherwise you will get OOM. This is usually the 1st thing we ask app developers (especially server app devs) to fix.

CLRProfiler gives you the most extensive info. If you can run your app with it, great; if you can’t, because it’s too heavyweight, you could take some memory dumps (if you don’t have the luxury to debug the app live) and compare the heaps. I would suggest you to take the dumps right after a GC so you get more acccurate comparision. Otherwise you could be looking at one heap where most objects haven’t been cleaned up yet and another one where no dead objects exist because it just did a full GC.

Perfmon can also be really useful to pinpoint some obvious things such as: if the # of GC Handles keeps growing, it indicates you may have a handle leak. For more info on GC perfmon counter please refer to my previous blog entry.

The full GC ratio

Often we tell people a healthy gen2:gen1 ratio is 1:10. If you are seeing a 1:1 ratio that’s certainly something to look into. When you have a really large heap and doing a gen2 could take some time it’s of course ideal to make the ratio of gen2 as low as possible. As with all perf advices there are always exceptions – for example if you have a heap where most objects are on the LOH and they don’t really move – we’ve seen this in certain server apps where they use some really arrays that live on the LOH to hold references to small objects and these arrays stay pretty much for the process lifetime, it’s not necessarily a bad situation. We do need to trace through the references that these large arrays contain but we don’t need to move memory around for them which would be an expensive operation. On the other hand, if you have an app that creates lots of temporary large objects you are likely to be in a bad situation if you also have a fairly big gen2 heap because the gen2’s that the large object allocations trigger will also need to collect your gen2 heap.

The common causes for doing a relatively high number of gen2’s include a managed cache implementation where the whole cache is in gen2 and it constantly gets churned; or if you have lots of finalizable objects whose finalizers get to run which increases the chance these objects make into gen2.

We’ve seen people who induce GCs, ie, calling GC.Collect, periodically. First of all, if it’s to “help not get OOMs” because you believe that GC is not kicking in early enough (if you are getting OOM because you have too many live objects and you exhausted memory it won’t be helped by inducing GCs anyway), you should tell us about it. Secondly, inducing GCs screws up GC’s tuning which is almost always a bad thing.

Fragmentation

Fragmentation is the free space you have on your managed heap which you can obtain by using the sos command

!dumpheap –type Free –stat

Of course the smaller the fragmentation the better but in the real world scenarios when you use IO and etc you will get pinned objects from the .net framework. Fragmenatation in different generations has different indications to how healthy your app is:

Fragmentation in gen0 is good because gen0 is where you allocate your objects and we will use the free space in gen0 to satisfy your allocation requests. As a hypothetical example, If we are looking a heap where all the Free objects are in gen0 that’s a very very ideal heap picture. You can specify a begin and an end address for !dumpheap to see how much space Free objects occupy in gen0. Take the example from the dump mentioned at the beginning:

Heap 0 (0015ad08)
generation 0 starts at 0x49521f8c
generation 1 starts at 0x494d7f64
generation 2 starts at 0x007f0038
ephemeral segment allocation context: none
segment begin allocated size
00178250 7a80d84c 7a82f1cc 0x00021980(137600)
00161918 78c50e40 78c7056c 0x0001f72c(128812)
007f0000 007f0038 047eed28 0x03ffecf0(67103984)
3a120000 3a120038 3a3e84f8 0x002c84c0(2917568)
46120000 46120038 49e05d04 0x03ce5ccc(63855820) [the last one is always the ephemeral segment]

0:119> ? 49e05d04-0x49521f8c
Evaluate expression: 9321848 = 008e3d78 [gen0 is about 9MB]

0:119> !dumpheap -type Free -stat 0x49521f8c 49e05d04
------------------------------
Heap 0
total 409 objects
------------------------------
Heap 1
total 0 objects
------------------------------
Heap 2
total 0 objects
------------------------------
Heap 3
total 0 objects
------------------------------
total 409 objects
Statistics:
MT Count TotalSize Class Name
0015a498 409 7296540 Free
Total 409 objects

So gen0 is 9MB and about 7MB is free space, ie, fragmentation.

Fragmentation in LOH is by design because we don’t compact LOH, which does NOT mean allocating on LOH is the same as malloc using the NT heap manager! Because of the nature of the way GC works, Free objects that are adjacent to each other are naturally collasped into one big free space which is available to satisfy your large object allocation requests.

The bad fragmentation is fragmenatation in gen2 and gen1. If after a GC you are still seeing lots of free space in gen2 and gen1 (especially gen2 since gen1 cannot exceed the size of a segment) that’s definitely something to look into. As I mentioned before we’ve been doing a lot of work on fragmenation and have improvement the situation by a lot but you can do your part by looking at how your app behaves and limit the fragmentation as much as possible from your code. Using GC Efficiently – Part 3 talks about good techniques to use when you do have to pin.

For gen2 if the fragmentation is lower than 20% it’s considered very good. Because gen2 can get really big it’s important to consider the ratio of fragmentation and not the absolute value.

Well, that’s all for today. See you next time!

[Editted on 05/16/2005 - I discovered that I actually misspelled "efficiently" (you'd think I can spell it but nooo...)...so I also took the opportunity to fix up the formatting that got messed up when I pasted the text from word to the editor for blogging the 1st time around.]