Unmanaged Memory Fragmentation -- an old story

SteveX Compiled » Blog Archive » Heap Fragmentation

SteveX Compiled » Blog Archive » Heap Fragmentation — Fri, 03 Feb 2006 06:02:25 GMT

PingBack from http://blog.stevex.net/index.php/2006/02/02/heap-fragmentation/

re: Unmanaged Memory Fragmentation -- an old story

Norman Diamond — Fri, 03 Feb 2006 06:33:38 GMT

> It took me over a week to diagnose just this
> one bug, although it took me only about 10
> minutes to fix it once I knew what was going
> on. Aren't you glad you don't have to deal
> with that sort of mess?

Um, I thought everyone had to deal with that kind of mess, not usually but often enough that it would happen to everyone.

On the other hand, the solutions that you describe here sound like more than 10 minutes of work.

re: Unmanaged Memory Fragmentation -- an old story

Davy Landman — Fri, 03 Feb 2006 11:31:11 GMT

why would you think it would take that much more work? maybe figuring out, but with a good design, such a change would only have to be made in one place..

I've never heard of this idea, but I must say, I like it! Did you get the idea by yourself? Or is it form a article/patern?

re: Unmanaged Memory Fragmentation -- an old story

Will Dean — Fri, 03 Feb 2006 13:45:20 GMT

"Aren't you glad you don't have to deal with that sort of mess?"

But I too have to deal with this sort of mess all the time, and I generally have to do it without access to the source code, developers or secret knowledge about a vast propertion of the stuff I'm dealing with. Incidentally, that's a fact which I would beat into Raymond Chen with a baseball bat if I ever had a quiet five minutes with him...

But I must say, I do like the simple ping-ponging strategy for heap tidying.

To the person who thought it would be hard, Rico's mod probably looked like a bit like this:

if((++nAllocs % 100000) == 0)
{
heapIndexOffset ^= 0x10;
}

Actually, Rico's so old, it probably looked like this:

if((++nAllocs % 100000) == 0) {
heapIndexOffset ^= 0x10;
}

;-)

There. There's only Larry Osterman to go and I shall have upset all the old folks...

re: Unmanaged Memory Fragmentation -- an old story

barrkel — Fri, 03 Feb 2006 13:49:48 GMT

"It took me over a week to diagnose just this one bug, although it took me only about 10 minutes to fix it once I knew what was going on. Aren't you glad you don't have to deal with that sort of mess?"

I think we all have these problems, Rico. Don't take this personally, because I don't mean this in any way disparagingly, it's just an observation: MS folks IMHO seem to think that what they do is more "heavyweight" than what anyone else does anywhere. In my opinion the biggest difference is the number of customers the rest of us have, not the depth or difficulty of the problems solved.

re: Unmanaged Memory Fragmentation -- an old story

ricom — Fri, 03 Feb 2006 21:01:54 GMT

>> Aren't you glad you don't have to deal with that sort of mess?

>> In my opinion the biggest difference is the number of customers the rest of us have, not the depth or difficulty of the problems solved.

Sorry about that, that was actually an attempt at humor (because it's old technology). I was thinking most of my readers write managed code or else are using more modern operating systems that don't have that problem anymore. I didn't mean to make any implication about the depth of my customers' problems...

That was very clumsy of me.

re: Unmanaged Memory Fragmentation -- an old story

ricom — Fri, 03 Feb 2006 21:58:39 GMT

I changed the comment to be closer to what I intended. "Aren't you glad our latest offerings do this better?"

I apologize that my attempt at a self-slam could be taken the other way. That was not intended.

re: Unmanaged Memory Fragmentation -- an old story

snaveen — Fri, 03 Feb 2006 23:02:25 GMT

What about managed heap fragmentation? Will you be able to write about this?

Interesting Finds

Jason Haley — Sat, 04 Feb 2006 04:14:20 GMT

re: Unmanaged Memory Fragmentation -- an old story

barrkel — Mon, 06 Feb 2006 14:25:15 GMT

> What about managed heap fragmentation? Will you be able to write about this?

The managed heap only gets fragmented when a pinned pointer refers to a block of memory in the middle of it and a garbage collection subsequently occurs. This fragmentation used to be pretty bad in .NET 1.0 and 1.1, but it's much better now. This one of the reasons that one should never pin an managed object for very long.

Heap fragmentation is almost always not a problem because the .NET GC is a compacting collecter. In moving objects into appropriate generations, the free space in the gaps is eliminated. Having said that, to my knowledge it is possible, but unlikely, for the large object heap (objects > 86000 bytes) to suffer from minor fragmentation problems for certain odd patterns of allocation (but the allocations would have to keep on going, i.e. a lot of memory would still have to be explicitly allocated). There was a presentation on the GC heap at the PDC; the videos and presentations are available somewhere on microsoft.com.

Rico has written on this:

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dndotnet/html/dotnetgcbasics.asp

Dev Diligence: Don’t Invest in the Wrong Code | Spiteful.com

Dev Diligence: Don’t Invest in the Wrong Code | Spiteful.com — Fri, 25 Apr 2008 23:32:59 GMT

PingBack from http://www.spiteful.com/2008/04/25/dev-diligence-dont-invest-in-the-wrong-code/

Back To Basics: Mark and Sweep Garbage Collection

I know the answer (it's 42) — Sun, 01 Feb 2009 23:42:07 GMT

This post is Part 4 in the series of posts on Garbage Collection (GC). Please see the index here. Recap