If broken it is, fix it you should

Using the powers of the debugger to solve the problems of the world - and a bag of chips    by Tess Ferrandez, ASP.NET Escalation Engineer (Microsoft)

.NET Memory usage - A restaurant analogy

.NET Memory usage - A restaurant analogy

Rate This
  • Comments 57

My favourite author Simon Singh is a wiz at analogies. In his book The big bang he explains concepts like the doppler effect and the theory of relativity using analogies with frogs and trains that makes it not only easy to understand but you will remember them forever because of the picture they paint in your head.

The other day at work I heard one of my colleagues explaining memory usage and why you get out of memory exceptions to one of his customers using a restaurant analogy.  I've talked about OOMs and memory management in an earlier post but I found the analogy so amusing that I thought I'd share it (and yes, before you say it, i do admit i might have stretched the analogy a little too far:), and that it doesn't hold a candle to Simon Singhs analogies, but then again he sells books and i just rant in a blog...).

Disclaimer:  In order to not get too longwinded I will simplify a lot of things, and say that for example the GC allocates 64 MB segments even though this differs between different framework versions and the size of the objects you allocate (read large object heap). Some other details are also dependent on configuration settings (i.e. using the /3GB switch etc.) but I will exclude such details from the analogy.

Analogy Part 1 - General memory usage

If you have read my earlier posts you will know that a process on a 32-bit system can typically address 2 GB of virtual address space. This is the memory that you have to work with, independently of how much RAM you have.  More RAM is good for performance since you page less with more RAM, but it doesn't do anything to expand the 2 GB address space.

Picture this 2 GB address space as being the floor space of a restaurant.

When you allocate an object (whether it is .net or non-.net) you typically follow a two step process.  You reserve the memory and then you commit space inside your reservation.

The reservation is equal to reserving a table at the restaurant.  And just like in a restaurant, depending on the memory manager you use (we will get to that later) you will reserve the memory in chunks.  Let's say for example you are a party of 3. It is not likely that there will be a table for 3 in the restaurant, but rather you would get a table for 4 out of which you use 3 seats and waste one seat.  

In memory terms the space for the table you have reserved is called reserved memory (virtual bytes), and the actual space you use (for the 3 seats) is comitted memory (private bytes).   The floor space that is not yet reserved is free memory.

On a pretty good restaurant night your restaurant/memory might look something like this where the blue areas are reserved space, red means committed space and white is free space.

      

Now, if someone calls in to make a reservation for 3 they will get the answer that the restaurant is full, since the only way to seat 3 people together is to seat them on a 4 seat table.  Even though you could fit in two 2-seat tables that wouldn't be good since they all want to sit together.

Similarily when you make memory allocations you won't split a memory reservation out into different locations, it has to be allocated in one chunk or not at all. So the memory result in this case would be "out of memory", even though there is plenty of space left.

An observant person might also note that if we put the tables closer together so that they are completely side-by-side you could easily fit in a new table of 4, but reserved memory areas much like tables at some restaurants can't be moved.

When we talk about memory fragmentation we either talk about the free but unusable (because it is not large enough to fit a new table) we have, or how much of our reserved memory we are not using (difference between virtual bytes and private bytes).

Analogy Part 2 - The .NET GC

Most of the time when you create objects in an application, whether it is .NET or not you use some kind of memory manager (NTHeap, C++ Heap, GC etc.), and in the restaurant case you can think of the memory manager as an hostess that reserves seats for you and ushers you to the location where you are to be seated.  For example if you call malloc you don't have to provide an address where you want your allocation to lie, instead you say that you want memory of a certain size and malloc returns, ok, you will be seated at table 1 in the "C++ heap" area.

The .NET GC takes this one step further and pre-reserves a large table for anyone who might want to use .NET objects in the process (let's say a 64 seat table).  And when anyone creates a .NET object, the GC ushers them to the next available seat on that table.  Once in a while the usher will walk around the table to check if someone is done eating and ask them to leave, and then scoots the rest of the people down the table.   Some people might be waiting on other people to finish up before they can leave (references), so they get to stay too. And some people may be really annoying and say, dude, i got a window seat, i am sooo not moving (pinned objects) which means that the rest of the people can't be scooted down towards the end of the table either.

Any empty seats between people are referred to as .NET memory fragmentation.

Once the 64 seat table is filled up the GC needs to reserve a new 64 seat table if it needs to accomodate newcommers, and if it can't you will get an out of memory exception.

But, how does it really look

Ok, enough with the analogy, here is what memory looks like in a real ASP.NET application

       

Again, the red parts are committed memory, the blue parts are reserved memory that is not committed and the white space is free space.

The dots you see towards the end of the memory space are probably dlls, and although just like in the restaurant scenario there is a lot of white space, it is likely that none of the gaps between the small red dots are large enough to house a 64 MB segment and thus the next time we fill up a GC segment and need a new one to accommodate a new object, we will get an out of memory exception.

The reason these small red dots (dlls) are spaced out like this is because they are loading at the prefered base addresses for those particular dlls.  You can't really do much about that type of fragmentation since it is hard to know in advance what a "good" prefered base address would be, but what you can do something about is finding out where the memory you are actually using is going.

A comment on performance counters and how not to use taskmanager

Throughout the analogy I talked about private bytes and virtual bytes and these are the two most important performance counters to look at when defining memory usage or memory leaks. 

There is another counter called working set which simplified consists of how much memory is in the memory pages that are currently or was recently touched by threads in the process or approximately, how much of the memory that is used by the process is currently in RAM.  The working set counter might be interesting if you have issues with too much paging and many processes on the same box competing about the RAM, but in order to determine how much memory you are using (reserved or committed) it offers little or no help.

If you want to see this in action, you can create a winforms application and allocate a bunch of objects and see the workingset go up, and then if you minimize the app, the working set drops.  This doesn't by any means mean that you have just released all this memory. It just means that you are looking at a counter that is totally irrelevant for determining how much stuff you store in memory :) Yet... this is the counter that people most often look at to determine memory usage...   

I know that by now you are probably thinking "yeah right", you haven't even heard of this counter before, why would I say that this is the counter most people look at???  The answer is, because most people use task manager to look at memory usage of a process, and specifically look at the Memory Usage column. Surprise surprise:) what this actually shows you is the working set of the process...

If you want to see private bytes which is a far more interesting counter, you sould look at the column in task manager that is labeled Virtual Memory Size (yeah, that's really intuitive:)), or better yet, look in performance monitor at process\private bytes and process\virtual bytes, there is no reason not to if your intent is to investigate high memory usage or a memory leak.

 

So tonight, go out, grab a bite to eat and see memory management in action:)  I bet you will probably find a lot more similarities than the ones me and my pal came up with...

Laters,





  • Where do you get those  memory map pictures?
  • I knew that if I would get one comment on this post, that would be it:)  

    One of my colleagues wrote a sample a long time ago that parses the output from !vadump and displays it in this way.

    Unfortunately I can't share the tool for different reasons, but the output from !vadump (in windbg) looks something like this if you want to write your own parser...

    0:000> !vadump
    BaseAddress:       00000000
    RegionSize:        00010000
    State:             00010000  MEM_FREE
    Protect:           00000001  PAGE_NOACCESS

    BaseAddress:       00010000
    RegionSize:        00001000
    State:             00001000  MEM_COMMIT
    Protect:           00000004  PAGE_READWRITE
    Type:              00020000  MEM_PRIVATE
    .........

  • Thanks Tess!  That was great.  But now I'm hungry . . .

    But seriously:  Why does Task Manager show private bytes (and why call it Virtual Memory?) and not give any option to see virtual bytes?  From your article I'd think virtual bytes would be a more useful statistic.   If there are 10 tables in my restaurant and they are all taken up by 10 different parties of 2 then I've maxed out my available memory - but this VM Size column would show I've only used up 20 seats.  If a process has reserved a ton of space I'd like to know about it - even if no one is sitting there at the moment.

    I expect part of your answer will be to use perfmon.  But I *like* taskmanager because I can get to it quick in a crises, see my processes immediately, and even kill one if I have to.  So it would be nice if it made a little more sense.

    OT:  All your blog entries are pleasure to read.  How about a book?
  • Best description of memory management ever! :)
  • You'll be happy to know that on Vista private pages is being shown by default instead of working set.
  • More on Atlas [Via: James Avery ] Smart Client Deployment with ClickOnce - Final Manuscript Complete!...
  • Awesome stuff that vista shows private pages by default, that is really great.

    George:  I hear ya, but it is really not as strange as you might think to call it virtual memory, after all it is committed virtual memory (and to not get endlessly long column names it is listed as just virtual memory).  

    I do agree with you that virtual memory is a ver important counter to look at (yes, unfortunately it is only available in perfmon), but in most memory cases the virtual/private bytes difference is not the problem, so looking at private bytes will give you a very good indication if you have a problem or not. (For example, in most asp.net apps around 1 GB private bytes, the virtual bytes will be ~1,5 GB and it is hard to get a ratio much smaller than that).    So for quick stuff, look in task manager for virtual mem, and for real investigations, look in perfmon at both virtual bytes and private bytes.

    The other thing is that if you look at a memory dump for example, the only thing that will help you recognize who is using the memory is by looking at the committed mem.  after all, the mem that is reserved but not committed contains no data, its just uninitialized space or garbage.

    Having said that, there is ways to track who owns virtual bytes that are not committed. The way to do this is to use a profiler or inject a dll into the process that hooks up to calls to virtualalloc and logs the stack anytime this is called, and then removes them when the space is deallocated.  

    One tool that does this is a tool that we use in support called debugdiag http://www.microsoft.com/windowsserver2003/iis/diagnostictools/default.mspx

    HTH
  • Hi Tess,

    Very good article. A question on your example of table:

    From the pic (table), it seems that all tables contains at least one red (committed).  How can the following statment be true since you can't split the reserved into two different area:

    "Even though you could give fit in two 2-seat tables that wouldn't be good since they all want to sit together. "

    Am I correct or I am missing something ?


    Thanks
  • I am not sure I follow...

    In the picture above we have had the following parties along with the types of tables they were assigned to

    party of 1 - 2 seat table
    party of 4 - 6 seat table
    party of 3 - 4 seat table
    party of 2 - 4 seat table
    party of 4 - 4 seat table
    party of 1 - 4 seat table

    It would have been more economical when it comes to space for the last party of 1 to sit at a 2 seat table instead of a 4 seater, but picture it like this...

    you call in for reservations, hostess: how many are you? you: not sure, i think we'll be 7...   so you get a table of 8, and then some of your pals never show up, but now you are already seated at a table of 8.   Similarily in the managed world you sometimes over reserve.

    HTH
  • I am really glad that there are not any real restaurants with GC-hostess. Why? I am a slow eater and I would not like to change seats during my meal.
  • Great blog.

    I have some questions on the memory counters -

    If the difference between the Private bytes and Virtual Bytes is high , does that indicate a problem? In our managed process we see that the private bytes is 250 MB whereas the virtual bytes is 500 MB. Does this mean there is some sort of fragementation?

    One more question, some of the code pages of the base dlls are shared across the process right, will those also show up in the Committed memory of a process i.e. private bytes?

    If working set is the pages recently used by the process , why do we sometimes see that Working Set Size (memory usage column in the taskmgr) is greater than the private bytes?

    thanks , Padma
  • Thank you Padma,

    It is fairly common that there is a difference between virtual and private bytes (up to around 500 MB or so if memory usage is high).  For example at the start of the process we will have one small object segment (around 64 MB) and one large object segment (around 16 MB) per processor (for server GC).  If these are not filled with data yet, virtual bytes will be a lot higher than private bytes.  Similarily dlls dont fit perfectly into the chunk sizes that can be reserved etc.   so I would start looking at VM fragmentation if the virtual bytes start increasing independently of private bytes or if for example you start getting out of memory exceptions at 300-400 MB.  Then you have a problem of someone reserving a lot more than they need.   If your app is purely managed, this is generally not something you have to worrry about since you don't have any custom components that would make unneccessary reservations.  One thing that does waste virtual bytes though is having debug=true in web apps.

    Private bytes are only private bytes so no shared bytes show up.

    Finally, working set contains shared bytes.

    Thanks,

  • I must be missing something. As you stated, blue areas are reserved space, red means committed space and white is free space. If I want to make a reservation, GC would need to use the white space to setup a table for me, right ? GC can't use the blue areas since they are reserved and can't fill in the request in some of the empty slot of the blue area ? I guess I misunderstood that you were talking about the blue area.

    Thanks
Page 1 of 4 (57 items) 1234
Leave a Comment
  • Please add 8 and 8 and type the answer here:
  • Post