A first hand look from the .NET engineering teams
Garbage collection is one of premiere features of the .NET managed coding platform. As the platform has become more capable, we’re seeing developers allocate more and more large objects. Since large objects are managed differently than small objects, we’ve heard a lot of feedback requesting improvement. Today’s post is by Surupa Biswas and Maoni Stephens from the garbage collection feature team. -- Brandon
The CLR manages two different heaps for allocation, the small object heap (SOH) and the large object heap (LOH). Any allocation greater than or equal to 85,000 bytes goes on the LOH. Copying large objects has a performance penalty, so the LOH is not compacted unlike the SOH. Another defining characteristic is that the LOH is only collected during a generation 2 collection. Together, these have the built-in assumption that large object allocations are infrequent.
Because the LOH is not compacted, memory management is more like a traditional allocator. The CLR keeps a free list of available blocks of memory. When allocating a large object, the runtime first looks at the free list to see if it will satisfy the allocation request. When the GC discovers adjacent objects that died, it combines the space they used into one free block which can be used for allocation. Because a lot of interaction with the free list takes place at the time of allocation, there are tradeoffs between speed and optimal placement of memory blocks.
A condition known as fragmentation can occur when nothing on the free list can be used. This can result in an out-of-memory exception despite the fact that collectively there is enough free memory. For developers who work with a lot of large objects, this error condition may be familiar. We’ve received a lot of feedback requesting for a solution to LOH fragmentation.
In .NET 4.5, we made two improvements to the large object heap. First, we significantly improved the way the runtime manages the free list, thereby making more effective use of fragments. Now the memory allocator will revisit the memory fragments that earlier allocation couldn’t use. Second, when in server GC mode, the runtime balances LOH allocations between each heap. Prior to .NET 4.5, we only balanced the SOH. We’ve observed substantial improvements in some of our LOH allocation benchmarks as a result of both changes.
We’re also starting to collect telemetry about how the LOH is used. We’re tracking how often out-of-memory conditions in managed applications are due to LOH fragmentation. We’ll use this data to measure and improve memory management of real-world applications.
We still recommend some traditional techniques are for getting the best performance from the LOH. Many large objects are quite similar in nature, which creates the opportunity for object pooling. Frequently, types allocated on the LOH are byte-buffers that are filled by third-party libraries or devices. Rather than allocating and freeing the buffer, an object pool would let you reuse a previously-allocated buffer. Since fewer allocations and collections take place on the LOH, fragmentation is less likely to occur and the program’s performance is likely to improve.
Very usefull post. I wonder if you could point to a definitive object pooling sample for byte buffers, is there one done by anyone from MSDN?
Good to see there are some improvements on LOH handling.
What is the reason not to include LOH compacting as an option? I understand it might impact performance but it would still be nice to have this possibility in applications which needs to use lots of large objects.
object pooling can be hard in a garbage-collected language, because object pooling need a detemined acquire/release or create/dispose pattern, which maybe unintuitive, suddenly C# become C++.
for example, if I pass a byte to some other code, how could I know when the reference to this object is gone without change the pattern of the 'other code' ?
Wouldn't a seperate heap for objects that would live for ever during the app lifecycle be useful. Application developer can always mark such object during creation and GC does not have to compact such heap. ofcourse programing errors may cause a leak but having such an option would be very useful.
@Marshall just found some info on MSDN here msdn.microsoft.com/.../cc163856.aspx
wow,,,Ilike this post and your working in this site because very help ful for me and
other peoples all world peoples and
related this my site <a href="http://www.textiel4all.blogspot.com">fashion desinging</a>
so thank you for the beautiful shating at this time all world peoples .
Any changes on how double array are handled on the LOH? In old versions there were special rules on when a double array went onto the LOH, I believe the threshold was much lower than for other allocations. I never understood the rational for that. So it would be great if you could write a bit about that (I am sure there is a good reason for that) and let us know whether anything changed about it.
"when in server GC mode, the runtime balances LOH allocations between each heap. Prior to .NET 4.5, we only balanced the SOH."
What does "balancing" mean in this context? How do you "balance" allocations in only one heap?
Will you provide any API, which can LOH allocation behaviour? I mean, sth more than memory pressure settings available now.
In 32-bit architrctures CLR’s execution engine attempts to place these arrays > 1000 doubles on the LOH because of the performance benefit of accessing aligned doubles. However there are no benefits of applying the same heuristics on 64-bit architectures because doubles are already aligned on an 8-byte boundary. As such we have disabled this heuristics for 64-bit architectures in .,NET 4.5
Common Language Runtime
First, thank you for these improvements and for responding to the outcry of developers regarding LOH issues. That said, I'm still disappointed with the current state of things. As "WalkingCat" said, in many scenarios object pooling is not a viable option. You end up having to do your own reference counting and "garbage collection" and it doesn't work for many non-trivial scenarios. Even the suggestion of that as a workaround indicates a large unsolved problem in .NET memory management.
Second, I really don't understand why you wouldn't give the developer at least the option to do LOH compacting if they wanted. This could be disabled by default, but for those of use who would gladly pay any performance penalty in order to keep running without OutOfMemoryExceptions it would be a godsend. Why not allow the developer to make an informed tradeoff? A simple boolean configuration/runtime setting would have solved most of this years ago. This is incredibly frustrating.
I've been on many projects where LOH fragmentation is a showstopper. I've had managers point to LOH fragmentation as justification for choosing some other technology, even at the start of projects where that issue will probably never come into play. They've been burned once and that's it. The credibility that .NET loses because of the LOH fragmentation issue is significant. I know that in many situations LOH fragmentation gets the blame even if it's not the culprit. Regardless, it has developed quite the stigma. I think the developer community has been puzzled by the lack of a real solution or at minimum the option of controlling the behavior while understanding the tradeoffs.
As Hetzi mentions above, I too do not understand these sentences: "Second, when in server GC mode, the runtime balances LOH allocations between each heap. Prior to .NET 4.5, we only balanced the SOH". Could you please expand on what that means and its implications?
@Abhishek: Cool! Very happy to hear that. I have a large project with lots of double arrays that have 1050 elements. I had to go through such an amazing number of silly hacks to get things to run it was not funny anymore.
While I would expect that my specific issue is solve with 4.5, I still want to add another voice for more flexibility for power users. Take the previous situation: My arrays HAD to have 1050 elements, that came from the problem structure. If there had been an option to set the limit to 1051, it would have saved my days of complicated, error prone hacking around that limitation.
Providing developers a feature to trigger LOH compaction is in the list of things we wish to do in future. As you would understand, this list has other feature requests as well, and we keep evaluating them continuously to ensure that we provide the best value to our developers with the resources available on our end.
A lot of my Out Of Memory Exceptions that were related to the LOH where because of repeated binary serialization of a large ammount of data over an extended period of time. When this happened to our long lived process it eventually started to cause OOM Exceptions even though memory was available. I wonder if these changes will fix this problem?
Any way you can bump LOH compaction to the top of the backlog? It is causing us such significant problems we'd be tempted to fund your development costs! We've been complaining about this for a while; I even talked to Maoni in person about it at PDC 2008.