“Out Of Memory” Does Not Refer to Physical Memory

re: “Out Of Memory” Does Not Refer to Physical Memory

Nick — Mon, 08 Jun 2009 23:40:28 GMT

I'm sure this is a stupid question related to my not having a clear understanding of the differences between address space, virtual memory, etc, but then where does the 2GB memory allocation limit come into play?

There is no 2GB memory allocation limit. You can allocate as much memory as you want; you have 2GB of address space to map it into. If you want to have 5GB allocated, great, you do that, but you only can have up to 2GB of it mapped at any one time. -- Eric

For example, when many applications I use attempt to cross the ~2GB limit, they error or crash.

If your app author is unwilling to take on the pain of writing its own memory mapper/unmapper, then clearly it is stuck with not allocating more memory than it can map. -- Eric

There's also the Windows limit of 2GB for user-mode applications (the /3GB switch stuff). What you've said here makes sense, so should I assume that the only way to get access to more than 2GB of address space is to use alternative memory allocation techniques (LARGEADDRESSAWARE for example)?

Indeed. All the 3GB switch does is give you an extra GB of user address space by stealing it from the operating system's address space. That's the only way to get more address space in 32 bit windows. Again, you can allocate as much memory as you want, but address space is strictly limited to 2GB or 3GB in 32 bit windows. -- Eric

It seems like your article ends right before a giant "BUT this is how it usually ends up working in real world applications" caveat.

In most apps, you can only allocate as much memory as you have room to map. -- Eric

re: “Out Of Memory” Does Not Refer to Physical Memory

Erik — Tue, 09 Jun 2009 00:08:24 GMT

Great article, thanks! :)

I would like to see a 'back-in-the-day' type article regarding Extended & Expanded memory, just for a history lesson - I never could get the hang of it back then...

re: “Out Of Memory” Does Not Refer to Physical Memory

Delmania — Tue, 09 Jun 2009 00:36:04 GMT

Excellent article, and I think your view of memory as large file on disk with the RAM being an optimization makes a lot more sense!

re: “Out Of Memory” Does Not Refer to Physical Memory

commongenius — Tue, 09 Jun 2009 01:35:28 GMT

I agree that thinking of RAM as a performance optimization makes the whole concept more straightforward. The CPU cache(s), the physical memory, and the virtual memory swap file are really just different levels of a multi-level cache, each level begin larger but slower than the level before.

re: “Out Of Memory” Does Not Refer to Physical Memory

Harry S. — Tue, 09 Jun 2009 01:42:37 GMT

We don't think about the system cache as being a device that swaps out to RAM when it gets too full, so it does make a lot of sense to think about memory as being a disk file and RAM just an optimization detail. Great article.

re: “Out Of Memory” Does Not Refer to Physical Memory

Jonathan — Tue, 09 Jun 2009 01:49:32 GMT

Physics nitpick:

The electrons in a circuit only travel a few miles per hour, it is their displacement wave that travels near the speed of light.

re: “Out Of Memory” Does Not Refer to Physical Memory

Blake Coverett — Tue, 09 Jun 2009 02:08:23 GMT

To follow up on Erik's wish (because I doubt it really has enough meat for a whole article from Eric) on Extended vs Expanded memory:

Extended Memory is what we think of as memory now, i.e. memory that is addressed further and further up from physical address 0 by using more and more of the address bus lines from the CPU. In the DOS days accessing this memory required magic, because DOS, and hence DOS apps, ran in "real mode" where the segment registers and offset registers were each 16 bits wide, and combined with a simple 4 bit shift and add to give a 20 bit physical address. Ignoring the HMA for a moment, to access beyond 2^20, i.e. 1MB, the CPU had to be switched to one of the protected modes, where the segment register treated as an index into a table of segment base addresses. This was the core of Win16's memory model.

Expanded Memory was, from the CPU's perspective, a device that happened to have RAM on it, not 'memory' at all. A portion of the reserved address space between 640K and 1024K, where other memory mapped devices like your VGA card, was used to map in pages of memory from that device. Once a page was mapped in, a real mode app could read or write it just like they'd read or write the VGA screen buffer's memory. An interrupt based API was used to change what page of expanded memory was mapped into that buffer. The closest we have to this more recently would be PAE, which like Expanded Memory is an nasty stopgap that's only used by a small fraction of apps.

For bonus points, that High Memory Area I mentioned was the first 64K past 1024K, that could be accessed from real mode. On a real 8086 (which real mode mimics) if the sum of the segment register shifted over 4 and the offset was greater than 2^20 it just wrapped, after all there were only 20 address lines. On a 286 or later you could choose whether it should wrap, or whether it should overflow into the 21's address line, giving you access to that extra little bit of memory. DOS quickly learned to tuck as much of itself as possible into that HMA, reducing the footprint it required in the bottom 640K where most apps were limited.

Okay, so it was a bit longer than I realized when I started rambling, even leaving out many details.

re: “Out Of Memory” Does Not Refer to Physical Memory

Néstor Sánchez — Tue, 09 Jun 2009 02:09:01 GMT

And about .NET development...

If I need to access many GBs of data... the CLR is intelligent enough to do it for me?

or... I must to write my own memory mapper to deal with the issue? and how?

On 64 bit CLR, you're all set. On 32 bit CLR, if you need to do custom mapping and unmapping of huge blocks of memory, then I personally would not want to handle that in managed code. There are ways to do it, of course, but it would not be pretty. -- Eric

re: “Out Of Memory” Does Not Refer to Physical Memory

configurator — Tue, 09 Jun 2009 02:43:09 GMT

This comment is by no means a tutorial on tutorials.

I find that posts that begin with such wording are usually the best tutorials available. This blog post is a great example of that. Previously, while I knew about these differences, I had to think to keep them in mind. Now, Eric has transformed that thought process into a state of mind. Yeah, memory is a file; it's cached in the RAM for performance. It truly makes perfect sense.

re: “Out Of Memory” Does Not Refer to Physical Memory

ShuggyCoUk — Tue, 09 Jun 2009 03:27:10 GMT

note on CLR, the CLR's memory constructs under the hood prevent any object (say an array or string) being more than 2GB in size so whilst you *could* write code to deal with greater than 2GB address space (which would be incredibly painful) you would still get an OOM exception i you tried to allocate new long[int.MaxValue] no matter what.

the nicer the box feels to be in the harder it is to jump out of it.

I like soft boxes iwht padding :)

re: “Out Of Memory” Does Not Refer to Physical Memory

Simon Buchan — Tue, 09 Jun 2009 03:38:54 GMT

A nitpick: considering RAM as a cache for the disk is not *quite* correct, since the manager can choose to never write a page in RAM to disk (although I believe it ensures there is enough storage in the page file just in case).

I just think of the combination of RAM and disk as the "physical memory" (or just "backing" to reduce confusion with RAM), where the manager tries to keep the often used stuff in the fast part of the memory.

re: “Out Of Memory” Does Not Refer to Physical Memory

Erik — Tue, 09 Jun 2009 04:38:07 GMT

@Blake Coverett - Thanks a bunch for the intro =) You're probably right about there not being enough meat for a full-blown article - but still, very interesting information nonetheless.

re: “Out Of Memory” Does Not Refer to Physical Memory

Mark — Tue, 09 Jun 2009 08:26:17 GMT

Simon Buchan: but that's just an optimisation applied to many caches. If the cached range never needs to touch the slow medium during its lifetime, there's not point doing so. Just as when you "allocate" a variable for a calcuation, it might get optimised away.

re: “Out Of Memory” Does Not Refer to Physical Memory

Simon Buchan — Tue, 09 Jun 2009 08:43:39 GMT

@Mark: Perhaps we have different definitions of "cache" then... I've always thought of it as being a copy of the authoritative value made for performance reasons. It's just the word though, this is getting super-nitpicky. (That is too a word, Safari!)

re: “Out Of Memory” Does Not Refer to Physical Memory

Jorge — Tue, 09 Jun 2009 10:46:07 GMT

"There is no 2GB memory allocation limit. You can allocate as much memory as you want; you have 2GB of address space to map it into."

What's the use? I can't have more than 2GB allocated AT THE SAME TIME.

The use case of memory is typically that you want to remember something. Perhaps you have four billion bytes of video that you are processing, and you want to keep the whole video in memory. You could map the bits you're currently editing into virtual memory while leaving the rest unmapped until you're done with what you've got. -- Eric

Out of memory can be raised because there's not enough virtual space free. The limit is the memory you can address not the memory you can map. If I allocate three 1 GB byte arrays it will probably fail with an out of memory error because I simple do not have enough memory to address the virtual address of the last one.

The strange case of Visual Studio getting "out of memory"

Carlos Quintero (Microsoft MVP) blog — Tue, 09 Jun 2009 11:53:38 GMT

I started programming in the old times of 16-bit MS-DOS, where you had only 640 KB of memory, and you

re: “Out Of Memory” Does Not Refer to Physical Memory

Anthony Jones — Tue, 09 Jun 2009 12:03:12 GMT

I agree with your conceptualisation. Its the way I've prefered to look at it ever since coding on VAX/VMS systems from which NT inherited these concepts via David Cutler.

re: “Out Of Memory” Does Not Refer to Physical Memory

Pop Catalin — Tue, 09 Jun 2009 13:37:02 GMT

Actually why is the hard disk size the limit of how much you could allocate? Why doesn't windows go to the cloud for more space ? And then the limit is the total amount of free space available on the internet ..... which is allot.

But of course if you go to the cloud with the allocation scheme, you need some URI scheme to reference your allocated space: pointer -> Uri translation.

And then you could think your PC is just a Cache for data allocated on the cloud/internet ;-).

P.S. This was meant as a joke :), obviously

re: “Out Of Memory” Does Not Refer to Physical Memory

Bob — Tue, 09 Jun 2009 15:54:10 GMT

Erik,

So your cross to bear was expanded/extended memory. Mine was overlay description files on the PDP-11.

As Anthony Jones said, your description brought back memories of VAX/VMS, which in the early days suffered from the high cost of RAM, making thrashing a not-so-theoretical problem. It was not uncommon to encounter VAX systems with 512KB of RAM.

re: “Out Of Memory” Does Not Refer to Physical Memory

TheCPUWizard — Tue, 09 Jun 2009 16:36:45 GMT

Bob,

I remember RSX-11/M ODL files well (along with RT-11 means of dealing with the same memory management issues). In fact I still have a couple of working PDP-11's (primarily Q-BUS), and a working PDP-8.

Now that was some real fun memory management. MAX 4KW of Code and 4KW of Data. And the mapping unit was also 4KW (which meant that when you changed mapping you immediately lost your current context <eek>).

[People interested in the "classics" can contact me via: david dot corbin at dynconcepts dot com

Interesting Finds: June 9, 2009

Jason Haley — Tue, 09 Jun 2009 17:21:47 GMT

Interesting Finds: June 9, 2009

re: “Out Of Memory” Does Not Refer to Physical Memory

halpierson — Tue, 09 Jun 2009 17:48:20 GMT

Mapping data files into your address space is another way of using your process space. It's more efficient than buffered access, that has to copy data into you process space buffers.

Can you map files in .NET?

re: “Out Of Memory” Does Not Refer to Physical Memory

Joren — Tue, 09 Jun 2009 20:19:12 GMT

@halpierson

There is a new namespace in .NET 4.0 that allows for memory mapped files:

http://msdn.microsoft.com/en-us/library/system.io.memorymappedfiles(VS.100).aspx

re: “Out Of Memory” Does Not Refer to Physical Memory

James — Tue, 09 Jun 2009 21:56:52 GMT

Lovely post, thanks! Of course, electrons flowing through conductors in the form of electricity actually move at significantly slower speeds than even your miata. The conductor just happens to always be full of electrons, making the signal propagate at that speed. Think of a hose full of water. Turn on the faucet and water comes out immediately. That doesn't mean the water is moving at the speed of light, just that the hose was full of water.

re: “Out Of Memory” Does Not Refer to Physical Memory

TheCPUWizard — Tue, 09 Jun 2009 23:23:42 GMT

The actual flow rate of electrons can be VERY VERY slow....Even the wave propogation is significantly slower in a non-vacuum environment.

I remember well laying out circuit boards and measuring trace lengths to determine propogation delay (and the fact that a signal would arrive at two different points in the circuit at different times).

IIRC: the normal number we used for copper PCB's was 1.5nS per FOOT.....

Indeed, Grace Hopper used to hand out a foot of phone wire when she'd give talks. She'd use the foot of wire to illustrate the concept of a nanosecond. -- Eric

Community Convergence XLIX (IL)

Charlie Calvert's Community Blog — Tue, 09 Jun 2009 23:45:02 GMT

Welcome to the 49th edition of Community Convergence. The big excitment of late has been the recent release

re: “Out Of Memory” Does Not Refer to Physical Memory

Samin — Wed, 10 Jun 2009 10:26:04 GMT

"Accessing data in RAM, where the information is stored in tiny, lightweight electrons that move at close to the speed of light is much faster than accessing data on disk, where information is stored in enormous, heavy iron oxide molecules that move at close to the speed of my Miata."

-- made my day :)

great article, I have to admit that I was one of those “I got an ‘out of memory’ error but I checked and the machine has plenty of RAM, what’s up with that?”. Shame on me. But now I know better.

re: “Out Of Memory” Does Not Refer to Physical Memory

hyperspaced — Wed, 10 Jun 2009 11:08:17 GMT

I can't seem to fully understand your view of the memory management regarding storage file as being the primary level of memory and RAM as an optimization

Let's assume a process is started and requires memory. What's the primary memory source? The storage file and *then* the OS copies the memory to RAM?

Please carefully re-read the sixth paragraph. -- Eric

Let's assume I have 6GB of RAM and only 2GB of storage file (in contradiction to MS recommendations) What happens there?

Probably nothing pleasant. -- Eric

“Недостаточно Памяти” не относится к физической памяти

Невероятные приключения в коде (перевод блога Эрика Липперта) — Wed, 10 Jun 2009 14:32:26 GMT

Я начал программировать на x86 машинах во время периода значительных и быстрых изменений в стратегиях

re: “Out Of Memory” Does Not Refer to Physical Memory

Daniel Earwicker — Wed, 10 Jun 2009 16:56:58 GMT

Simon Buchan said: A nitpick: considering RAM as a cache for the disk is not *quite* correct, since the manager can choose to never write a page in RAM to disk (although I believe it ensures there is enough storage in the page file just in case).

The file system's in-memory cache can make the same decision, especially if you tell CreateFile that a file is "temporary" and then you delete it quickly. It may never get written to the disk at all, but only ever exist in memory. And yet we still call it the "disk cache"! The reality (in both VM and disk caching) is that we have *some* stuff in memory, and *some* stuff on disk, and there's *some* overlap between the two, and that's about all we can say in general.

re: “Out Of Memory” Does Not Refer to Physical Memory

Eddie — Thu, 11 Jun 2009 08:11:57 GMT

>>For example, when many applications I use attempt to cross the ~2GB limit, they error or crash.

>If your app author is unwilling to take on the pain of writing its own memory mapper/unmapper, then clearly it is stuck with not allocating more memory than it can map. -- Eric

Which is it? Your main article appeared to say 'It Just Works' but this response seems to contradict that earlier statement.

It's always been my experience that if you're going to work with very very large amounts of data, you have to be able to process them without needing it all in memory at the same time. When you hit that stage, you're working with files on disk from the start.

It's the same idea as iterating over an enumerator. You don't always want always assume you can Count() or ToList() it because what happens if it's streaming data?

re: “Out Of Memory” Does Not Refer to Physical Memory

Bruce Pierson — Thu, 11 Jun 2009 09:03:35 GMT

Ha, this was great. But speaking as an old hard-disk guy, iron oxide is almost as outdated as expanded/extended memory. Your new shiny computer probably uses cobalt thin film media rather than rust on its disk platters. So maybe more like your Ferrari (come on, we know you have one)?

re: “Out Of Memory” Does Not Refer to Physical Memory

Gokhank KORALTURK — Thu, 11 Jun 2009 09:53:20 GMT

Well, in my case, i sometimes prefer to disable paging file, thus removing

1. the cost of unnecessary disk io

2. the chance of allocating memory more than what i have as RAM.

All in all, if i don't need too much memory on my laptop (i have 4GB of RAM and it's almost always enough) this method reduces the I/O and also helps to keep myalways moving laptop's harddrive physicaly safer (less IO less chance of bad sectors)

In this case, there is no use for page faults, and it's a happy system as long as you don't need big chunks of memory. And trust me, it boosts the performance since Windows DOES paging even if everything fits in RAM, but unable to do so with this method.

“Out Of Memory” Does Not Refer to Physical Memory - Eric Lippert

DotNetShoutout — Thu, 11 Jun 2009 15:12:56 GMT

Thank you for submitting this cool story - Trackback from DotNetShoutout

re: “Out Of Memory” Does Not Refer to Physical Memory

Jim — Thu, 11 Jun 2009 17:42:42 GMT

"The amount of data storage reserved for a process is only limited by the amount of space that the operating system can get on the disk."

"RAM can be seen as merely a performance optimization."

This is not always correct because windows can operate without a paging file. And if you have a lot of ram (which btw is very cheap these days) you should kill the paging file.

Indeed. Re-read the sixth paragraph. -- Eric

I have 3 computers at home (3GB, 3GB and 8GB) and 2 at work (4GB and 16GB) and none of them uses a paging file. Having a paging file makes sense if you don't have a lot of memory but nowadays it less and less useful. 2GB of DDR2 can be bought for $30 or less. Buy RAM and kill the paging file and do not think of ram as data in a file on disk.

re: “Out Of Memory” Does Not Refer to Physical Memory

Vrv — Thu, 11 Jun 2009 18:26:52 GMT

Cool. Reminded me of my operating systems classes in college. A brief explanation of the dining philosophers problem would have been excellent.

re: “Out Of Memory” Does Not Refer to Physical Memory

Gab EL-D — Fri, 12 Jun 2009 11:27:27 GMT

Started programming in the 32-bit systems (x86). I think it makes a lot of sense, although i work mostly with managed code. It explains why an app running on a system with almost 4GB of ram and free disk space of 20GB, throws an out of memory exception when it ram usage bearly 1.5GB (after running for long hours).

And it is the only major app running on the system.

What do you suggest will solve the problem? (Leaving mapping/unmapping memory aside, as it is managed code).

My suggestion: use the CLR memory profiler to figure out why you are using so much memory. If you have a memory leak, fix it. If you don't, figure out what the largest consumer of memory is, and optimize it so that it doesn't consume so much memory. -- Eric

re: “Out Of Memory” Does Not Refer to Physical Memory

Loddie — Fri, 19 Jun 2009 15:39:43 GMT

in all of this, I'm curious. What does this set of memory guru's recommend as a swap file setting? Is it Swap File = RAM or Swap File = RAM * 1.5 (or some other setting)? Or, do you really recommend letting Windows manage the swap file size? I'd heard (long ago) that this just eats up system resources letting Windows manage it and I've always opted to just set it to 1.5 times the RAM.

Thanks!

re: “Out Of Memory” Does Not Refer to Physical Memory

Bob Beasley — Fri, 19 Jun 2009 17:00:26 GMT

To dive deeper into this article I need to know if you race your Miata? If yes, autocross or road race?

Seriously, thank you for the article!

Bob

re: “Out Of Memory” Does Not Refer to Physical Memory

Mike — Wed, 24 Jun 2009 16:56:12 GMT

Excellent arcticle. One question. I have a website that relies heavily on caching to reduce database access. About once a day I start to receive OutOfMemory exceptions and the worker process recycles itself. Would you recommend upgrading RAM ( Currently 4GB ) or the OS to 64 bit?

Better than either of those strategies would be to fix the actual bug, which is that the caching logic leaks memory. If the cache is growing without bound and eventually filling up all of memory, giving it more room to grow is just making the problem worse, not better. -- Eric

re: “Out Of Memory” Does Not Refer to Physical Memory

LegacyOfHerot — Fri, 03 Jul 2009 18:53:05 GMT

I'm sure this won't be a popular question: I have a number of boxes running XP, and have been for years. I used to be able to run dozens of applications at the same time. Now, after the past few service packs, I've noticed that GIVEN THE SAME INSTALLATIONS, AND THE SAME HARDWARE I cannot run nearly as many applications at the same time. A number of colleagues have noticed similar issues. Its almost as though something has been pushed out in a service pack to peg the number of concurrent apps, or limit memory in some way. Of course this might encourage some to "upgrade" to x64, but given the number of everyday apps that dont install / run on x64 that is not an option. Just to re-iterate - this is not the gradual slowing down of performance we have all come to expect, this has been a distinct change over the past few packs, on several machines. Any thoughts?

re: “Out Of Memory” Does Not Refer to Physical Memory

seo — Thu, 23 Jul 2009 02:16:18 GMT

Reminded me of my operating systems classes in Uni

re: “Out Of Memory” Does Not Refer to Physical Memory

Tom — Sat, 15 Aug 2009 16:18:57 GMT

I do like this article quite a bit, and the concept of treating RAM as a disk cache is refreshing. Of course, you did neglect to acknowledge memory allocations that neither (a) mirror data on disk or (b) are ever intended to reach the disk. I'm thinking mainly of the application stack, but also of temporary allocations used for decompressing data, and other such transitive operations. Perhaps it's best to ignore these exceptions for sanity's sake!

re: “Out Of Memory” Does Not Refer to Physical Memory

Mike Dimmick — Wed, 26 Aug 2009 16:20:53 GMT

Loddie, the OS already uses the 1.5 x RAM rule for the minimum when there is less than 2GB fitted or 1 x RAM for more than 2GB. The maximum is 3 x RAM in both cases. At least, that's my recollection, I don't have my copy of Windows Internals 5th Edition to hand right now.

There really is no right setting for the swap file. In Task Manager, before Windows Vista, Commit Charge Peak shows the maximum amount of the swap file that has been used (actually physical + swap file) since the system was booted. Limit is the current maximum possible commit. ('Commit Charge' - you 'commit' virtual memory to make it usable, 'reserve' just stops anything else from using that address range, but you get access violation exceptions if you try to reference reserved, but not committed, memory. The process is 'charged' for memory it commits, so 'commit charge' is the total memory committed by all processes.)

Windows Vista rather more sensibly shows the actual page file usage. Commit Charge Peak isn't shown.

re: “Out Of Memory” Does Not Refer to Physical Memory

Alex K — Tue, 10 Nov 2009 18:06:21 GMT

While horribly late to the party (way beyond fashionable), I'd still like to ask a question.

What do I do if this "out of memory" error happens in the managed app running under CLR? I don't get direct access to any memory/pages. So while the problem is clear to me after reading this article, I am in complete darkness as to what should I do about it.

Rewrite the program to not use so much memory per process. Or, less good, tell the user to stop throwing problems at the program that require so much memory. -- Eric

re: “Out Of Memory” Does Not Refer to Physical Memory

Mohit Jain — Wed, 16 Dec 2009 09:15:15 GMT

Very nice article, explains the concept of a win32's process memory/address space in a very simple way.

Thanks