Summary of the recent spate of /3GB articles

The /3Gb Switch

Exchange - The Blog — Sun, 22 Aug 2004 18:39:00 GMT

re: Summary of the recent spate of /3GB articles

winden — Sun, 22 Aug 2004 17:03:00 GMT

The confusion on some comments just highlights what we all know: memory management is hard to grasp and is easier to use when handled by the OS or the language.

Not mocking.

Bryce — Sun, 22 Aug 2004 20:30:00 GMT

I think that the wording now reflects what I was thinking. I didn't think that I was confused but now I'm not so sure. Help?

re: Summary of the recent spate of /3GB articles

Ian Griffiths — Sun, 22 Aug 2004 21:08:00 GMT

For what it's worth, I really liked this series - I always enjoy your blog, and it was particularly good to have such thorough and deep coverage of a topic.

If some people have misunderstood certain fundamental points like the distinction between virtual and physical address space, then I don't think that's necessarily indicative of a problem with the series. I would say simply that it indicates that those readers weren't really ready to read the articles.

I suppose you could have assumed a different level of knowledge from your readers, but that would simply have made the articles useful to a different set of readers. It wouldn't necessarily have made them better as such.

re: Summary of the recent spate of /3GB articles

mpz — Sun, 22 Aug 2004 23:12:00 GMT

I have to say I've enjoyed these articles too. To top the series off, do you have any tidbits about the x86-64 Windows that you can share, especially memory related?

Like somebody else commented in one of the articles - it's a revolution waiting to happen. Once x86-64 Windows is out, there's no going back.

re: Summary of the recent spate of /3GB articles

Carlos — Mon, 23 Aug 2004 01:15:00 GMT

Myth:Virtual memory is system memory that is simulated by the hard drive.

This definition comes from googling "what is virtual memory":
http://www.google.com/search?hl=en&ie=UTF-8&oe=UTF-8&q=what+is+virtual+memory

Surely developers aren't this clueless?

When interviewing I used to ask some questions about virtual memory. I never got good answers - the interviewees would have no idea or they would give me something similar to the definition above. Virtual address space was a mystery.

I'm not even slightly surprised that people have trouble with /3GB, PAE, etc.

The articles are excellent in themselves, but I think that the basic misunderstandings fuel the myths, so an introductory article would have helped.

For those folks that don't

E-Bitz - SBS MVP the Official Blog of the SBS — Mon, 23 Aug 2004 04:40:00 GMT

re: Summary of the recent spate of /3GB articles

Norman Diamond — Mon, 23 Aug 2004 01:53:00 GMT

Well, what the hardware does with virtual memory isn't the entirety of the meaning of virtual memory. Sure all the rest of the baggage associated with virtual memory has part of the CPU's hardware as its foundation, but the baggage is what users usally have to deal with. Plus in Windows the settings for page file sizes often say virtual memory instead of page file. When the abstraction doesn't leak, no one has to remember that it's founded on address translation units and stuff like that.

Meanwhile, virtual memory isn't alone in confusing people. Recently I had a run-in with someone who didn't understand the possible benefit of increasing the physical memory in his notebook from 64MB to 128MB. He designs electronic circuits, and he uses computers (using both hardware and software), but I guess the design and meaning of computer hardware just never entered into his experience.

re: Summary of the recent spate of /3GB articles

Raymond Chen — Mon, 23 Aug 2004 02:00:00 GMT

I'm not surprise that your average computer user has trouble with the virtual memory / physical memory / address space distinction. But your average computer user also doesn't go spouting off about the /3GB switch.

re: Summary of the recent spate of /3GB articles

Frodo — Mon, 23 Aug 2004 02:17:00 GMT

This is a very difficult subject for most people to understand, even programmers. I mean, this:

>>Virtual memory is system memory that is simulated by the hard drive.

This isn't a bad explanation for a non-programmer. I assume Carlos is taking the position that this is wrong, but -- conceptually -- it works for laymen.

re: Summary of the recent spate of /3GB articles

Steve Dispensa — Mon, 23 Aug 2004 04:43:00 GMT

If virtual memory is simulated by the hard drive, what are virtual disks simulated by?

Hmmm.... :)

/3GB switch myths

Praveen Kumar's Blog — Mon, 23 Aug 2004 11:04:00 GMT

/3GB switch myths

Praveen Kumar's Blog — Mon, 23 Aug 2004 11:45:00 GMT

re: Summary of the recent spate of /3GB articles

Tito — Mon, 23 Aug 2004 13:51:00 GMT

I sincerely thank Raymond for covering this topic. I don't think I was alone in the confusion. My main incorrect notion was about the distiction between Virtual Memory and Virtual Address Space. (I had no idea there even was a distiction.)

However, it is really unfortunate that on msdn, the functions that were mentioned here (CreateFileMapping,(Un)MapViewOfFile) aren't mentioned anywhere in the "memory management" section:
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/memory/base/virtualalloc.asp

That the "virutal memory" functions listed there actually handle both the allocation AND the mapping is surely the cause of much of this confusion (it was for me).

re: Summary of the recent spate of /3GB articles

David Candy — Mon, 23 Aug 2004 15:14:00 GMT

It seems to me the confusion is per machine vs per process. Raymond is talking per process and people are thinking per machine and interpreting it from that perspective.

Larry O was talking about VM from the machine perspective (page file size + physical memory + (I presume) code segments in exe and dlls) which is called commit charge.
http://weblogs.asp.net/larryosterman/archive/2004/03/18/92010.aspx (and look at surrounding days).

With raymond, he is talking about things that are normally done for system configuration (editing boot.ini) but is talking about the effect on a process not the box.

The person you linked too is also thinking you are talking about system configuration (eg setting the page file size).

re: Summary of the recent spate of /3GB articles

David Candy — Mon, 23 Aug 2004 15:17:00 GMT

Although in 3.1 enhanced mode it was said that a swap file made the system run quicker, even if it wasn't used as it wqas designed to swap and was slower for ????????????? reason if it couldn't.

Still changing LRU sweep frequency et al never made any detectable difference.

re: Summary of the recent spate of /3GB articles

Ben Cooke — Mon, 23 Aug 2004 19:29:00 GMT

I've been quite interested in this stuff because I rarely encounter these kinds of problems in my day-to-day programming and so I've never encounted this 3GB switch nor had to worry about distinctions between memory types.

Of course, as a programmer I can't help but know what virtual address space is. However, I don't recall anyone ever saying specifically what virtual *memory* is, as a concrete definition. I have some hazy ideas in my head about it, but I don't feel confident that I could actually give an accurate definition in English. Can anyone offer one?

re: Summary of the recent spate of /3GB articles

Andrew Feldstein — Mon, 23 Aug 2004 20:05:00 GMT

Thank you for the jump table.

Summary of the recent spate of /3GB articles

TrackBack — Tue, 24 Aug 2004 21:56:00 GMT

re: Summary of the recent spate of /3GB articles

Larry Osterman — Tue, 24 Aug 2004 21:20:00 GMT

David,
In general, having a paging file makes systems run faster - that's a corrolary to my post (btw, I wish you'd also included a reference to the correction: http://weblogs.asp.net/larryosterman/archive/2004/05/05/126532.aspx - while my post was roughly correct, there were certain critical details I missed).

re: Summary of the recent spate of /3GB articles

Peter Evans — Thu, 26 Aug 2004 08:26:00 GMT

Raymond,

Is this series a personal rant? (An early entry mentioned something about having to live with it for six years.)

I am puzzled as to how these posts relates to your other programming posts in your blog. Well there were those few discussion about lock free approaches.

Hmm. Maybe you are just leading up to details of 64-bit pointers issues and other memory management issues and design issues with regard to 64-bit hardware and drivers.

Haven't read the each entry in the series yet. Been catching up. Good to see a little coverage of these poorly understood things.

Will the series continue to cover other aspects of memory management such as the secured memory concepts of .Net and Non-executable memory regions.

re: Importance of alignment even on x86 machines

The Old New Thing — Fri, 27 Aug 2004 18:48:00 GMT

Backlog

C#deSamurai — Sun, 29 Aug 2004 03:36:00 GMT

re: Summary of the recent spate of /3GB articles

Norman Diamond — Wed, 01 Sep 2004 02:28:00 GMT

See:
http://www.sonic.net/~ezzell/Error%20Messages/Its_Virtual_Memory.htm

Interesting summary of how Windows memory arguments wrok

Software is too expensive to build cheaply... — Tue, 19 Oct 2004 15:06:00 GMT

At work, there's been some debate on how best to structure our J2EE servers to maximise utilisation of a scarce resource (per-CPU licenses!). The crux of the debate centered, for some reason, on how we can allocate more JVM instances...

/3GB and /PAE - Original Posted Mar 15, 2005

Cluster Help — Fri, 05 Aug 2005 22:56:17 GMT

WOW64 on HTTP.sys

Windows Network Development — Tue, 11 Jul 2006 02:02:45 GMT

Back when Windows Server 2003 SP1 shipped, HTTP.sys hit an important milestone, WOW64 support.What is...

Windows Not Recognizing total RAM Upgrade - MacNN Forums

Windows Not Recognizing total RAM Upgrade - MacNN Forums — Wed, 10 Jan 2007 09:25:00 GMT

PingBack from http://forums.macnn.com/104/alternative-operating-systems/322800/windows-not-recognizing-total-ram-upgrade/#post3264015

poundbang - Harish Mallipeddi's Blog » Interesting links from delicious

poundbang - Harish Mallipeddi's Blog » Interesting links from delicious — Wed, 28 Mar 2007 22:50:34 GMT

PingBack from http://poundbang.in/2007/03/29/interesting-links-from-delicious-9/

Memory Management - Demystifying /3GB » D’ Technology Weblog — Technology, Blogging, Gadgets, Fashion, Life Style.

Sun, 15 Apr 2007 00:28:36 GMT

PingBack from http://www.ditii.com/blog/2007/04/14/memory-management-demystifying-3gb/

32bit Windows is Not Limited to 2GB/4GB

Nir's Software Company — Tue, 24 Jul 2007 10:13:37 GMT

32bit Windows is Not Limited to 2GB/4GB

Citrix mit 8 GB und 32 Bit betreiben - MCSEboard.de MCSE Forum

Citrix mit 8 GB und 32 Bit betreiben - MCSEboard.de MCSE Forum — Tue, 18 Sep 2007 22:34:24 GMT

PingBack from http://www.mcseboard.de/windows-forum-ms-backoffice-31/citrix-8-gb-32-bit-betreiben-121295.html#post748073

Unable to install OpenType .otf & PostScript Type 1 fonts « Phinney on Fonts

Fri, 26 Dec 2008 15:02:23 GMT

PingBack from http://tphinney.wordpress.com/2008/12/19/no-font-install/

Phinney on Fonts » Unable to install OpenType .otf & PostScript Type 1 fonts

Wed, 21 Jan 2009 02:15:54 GMT

PingBack from http://www.thomasphinney.com/2008/12/19/no-font-instal/

The strange case of Visual Studio getting "out of memory"

Carlos Quintero (Microsoft MVP) blog — Tue, 09 Jun 2009 11:53:35 GMT

I started programming in the old times of 16-bit MS-DOS, where you had only 640 KB of memory, and you

“Out Of Memory” Does Not Refer to Physical Memory

Fabulous Adventures In Coding — Thu, 11 Jun 2009 10:05:11 GMT

I started programming on x86 machines during a period of large and rapid change in the memory management