• The Old New Thing

    Myth: In order to use AWE, you must enable PAE


    Address Windowing Extensions (AWE) does not require PAE. I don't know why some people claim that it does, since it is so easy to demonstrate otherwise.

    Take a program that uses AWE. If you don't have one handy, you can use the one that comes in MSDN as a sample program that demonstrates how to use AWE. Grant yourself "Lock Pages in Memory" privileges and run the program. Observe that it works.

    Now remove the /PAE switch from your boot.ini, reboot, and run the program again. Observe that it still works.

    Myth disproved by direct experimentation and observation.

  • The Old New Thing

    Myth: PAE increases the virtual address space beyond 4GB


    This is another non sequitur. PAE increases the amount of physical memory that can be addressed by the processor, but that is unrelated to virtual address space. (Remember that PAE stands for Physical Address Extensions.)

    PAE increases the physical address space (the address space that the CPU can use to access the memory chips on your computer) from 32 bits to 36 on a Pentium 2, for a theoretical maximum physical memory capacity of 64GB. However, the size of a pointer variable hasn't changed. It's still 32 bits (for a 32-bit processor), which means that the virtual address space is still 4GB.

    With PAE enabled, the page table and page directory entries double in size (to accomodate the additional bits in the page frame), which significantly increases the amount of memory required for page tables and page directories (since each page table describes only half as much memory as it used to).

    Notice that this has as a consequence that PAE and /3GB conflict with each other to some degree. If you turn on both PAE and /3GB, then the kernel will limit itself to 16GB of physical memory. That's because there isn't enough address space in the kernel to fit all the necessary memory bookkeeping into the 1GB of memory you told the kernel to squeeze itself into.

    (On an AMD processor, the physical address space expands to 40 bits, for a theoretical maximum of 1TB. However, the memory manager uses only 37 of those bits, for an actual maximum of 128GB. Why? For the same reason that the kernel limits itself to 16GB in /3GB mode: Not enough address space. It's time to move to 64-bit processors...)

  • The Old New Thing

    Why all these articles about PAE and /3GB?


    Apparently there is some unrest in comment-land with people who are sick of this whole /3GB series.

    Why have I been spending over two weeks exploring the consequences of the /3GB switch and exploding various common myths about it?

    Because too many people don't understand what /3GB means but talk as if they do. As you saw yesterday, there are still lots of people out there that don't understand the differences between physical memory, virtual memory, and virtual address space, and end up misconfiguring their computers or treating the switch as magic fairy dust. I've gotten tired of explaining it to misguided person after misguided person over the years, so I figured if I wrote up the explanation and debunkings once and for all, I won't have to visit the topic ever again.

    You think you're sick of /3GB? You've only had to deal with it for two weeks. Imagine having to explain the /3GB switch for six years!

    At any rate, the 3GB series will draw to a close at the end of the week, assuming everything goes according to schedule. Then there will be other topics for you to be sick of.

  • The Old New Thing

    Why is the virtual address space 4GB anyway?


    The size of the address space is capped by the number of unique pointer values. For a 32-bit processor, a 32-bit value can represent 232 distinct values. If you allow each such value to address a different byte of memory, you get 232 bytes, which equals four gigabytes.

    If you were willing to forego the flat memory model and deal with selectors, then you can combine a 16-bit selectors value with a 32-bit offset for a combined 48-bit pointer value. This creates a theoretical maximum of 248 distinct pointer values, which if you allowed each such to address a different byte of memory, yields 256TB of memory.

    This theoretical maximum cannot be achieved on the Pentium class of processors, however. On reason is that the lower bits of the segment value encode information about the type of selector. As a result, of the 65536 possible selector values, only 8191 of them are usable to access user-mode data. This drops you to 32TB.

    The real limitation on the address space using the selector:offset model is that each selector merely describes a subset of a flat 32-bit address space. So even if you could get to use all 8191 selectors, they would all just be views on the same underlying 32-bit address space.

    (Besides, I seriously doubt people would be willing to return the the exciting days of segmented programming.)

    In 64-bit Windows, the 2GB limit is gone; the user-mode virtual address space is now a stunning 8 terabytes. Even if you allocated a megabyte of address space per second, it would take you three months to run out. (Notice however that you can set /LARGEADDRESSAWARE:NO on your 64-bit program to tell the operating system to force the program to live below the 2GB boundary. It's unclear why you would ever want to do this, though, since you're missing out on the 64-bit address space while still paying for it in pointer size. It's like paying extra for cable television and then not watching.)

    Armed with what you have learned so far, maybe you can respond to this request that came in from a customer:

    Oen of our boot.ini files has a /7GB switch. Our consultant told us that we should set it to 1GB less than the system memory. Since we have 8GB, 8GB - 1GB = 7GB. The consultant said that setting this value allows an application to allocate more than 2GB of memory. We would like Microsoft to comment on this analysis.
  • The Old New Thing

    Myth: The /3GB switch lets me map one giant 3GB block of memory


    Just because the virtual address space is 3GB doesn't mean that you can map one giant 3GB block of memory. The standard holes in the virtual address space are still there: 64K at the bottom, and 64K near the 2GB boundary.

    Moreover, the system DLLs continue to load at their preferred virtual addresses which lie just below the 2GB boundary. The process heap and other typical process bookkeeping also take their bites out of your virtual address space.

    As a result, even though the user-mode virtual address space is nearly 3GB, it is not the case that all of the available space is contiguous. The holes near the 2GB boundary prevent you from getting even 2GB of contiguous address space.

    Some people may try to relocate the system DLLs to alternate addresses in order to create more room, but that won't work for multiple reasons. First, of course, is that it doesn't get rid of the 64K gap near the 2GB boundary. Second, the system allocates other items such as thread information blocks and the process environment variables before your program gets a chance to start running, so by the time your program gets around to allocating memory, the space it wanted may already have been claimed.

    Third, the system really needs certain key DLLs to be loaded at the same address in all processes. For example, the syscall trap must reside at a fixed location so that the kernel-mode trap handler will recognize it as a valid syscall trap and not as an illegal instruction. The debugger requires this as well, so that it can use CreateRemoteThread to inject a breakpoint into the process when you tell it to break into the process you are debugging.

  • The Old New Thing

    Why does Exchange recommend /3GB if you have more than 1GB of physical memory?


    If you look through the Knowledge Base, you'll see an article that say that Exchange 2000 requires the /3GB switch with more than 1 gigabyte of physical RAM. Yet I've been writing all this time that /3GB has nothing to do with physical RAM. What's the deal?

    The title of the article could be a bit clearer. It really should be something more like "Exchange 2000 requires the /3GB switch to take advantage of more than 1 gigabyte of physical RAM".

    It appears that Exchange 2000 doesn't use the bank-switching technique I described in an earlier entry. (I don't blame them. It's extraordinarily cumbersome.) Consequently, for Exchange 2000, virtual address space equals virtual memory.

    The capacity of a program is typically a combination of multiple factors, the lowest of which sets the limit. By analogy, suppose you need two piece of bread, two pieces of bologna, and a slice of cheese to make a bologna and cheese sandwich. Whichever ingredient you run out of first determines how many sandwiches you can make. If you run out of cheese first, adding more bologna won't help any.

    Okay, so what do bologna and cheese sandwiches have to do with Exchange 2000?

    From the description in the article, it appears that the store.exe program is RAM-constrained most of the time (you run out of cheese first). But once you get the memory on the machine up to one gigabyte, you have excess RAM and address space becomes the new limiting factor. (You added plenty of cheese and now you've run out of bologna.) That's where the /3GB switch comes in. It increases the user-mode address space, thereby relieving pressure on the address space constraint.

  • The Old New Thing

    Myth: The /3GB switch expands the user-mode address space of all programs


    Only programs marked as /LARGEADDRESSAWARE are affected.

    For compatibility reasons, only programs that explicitly indicate that they are prepared to handle a virtual address space larger than 2GB will get the larger virtual address space. Unmarked programs get the normal 2GB virtual address space, and the address space between 2GB and 3GB goes unused.


    Because far too many programs assume that the high bit of user-mode virtual addresses is always clear, often unwittingly. MSDN has a page listing some of the ways programs make this assumption. One such assumption you may be making is taking the midpoint between two pointers by using the formula (a+b)/2. As I noted in a previous exercise, this is subject to integer overflow and consequently can result in an erroneous pointer computation. Consequently, you can't just take an existing program that you didn't write, mark it /LARGEADDRESSAWARE, and declare your job done. You have to check with the authors of that program that they verified that their code does not make any 2GB assumptions. (And the fact that the authors didn't mark their program as 3GB-compatible strongly suggests that no such verification has occurred. If it had, they would have marked the program /LARGEADDRESSAWARE!)

    Marking your program /LARGEADDRESSAWARE indicates to the operating system, "Go ahead and give this program access to that extra gigabyte of user-mode address space," and as a result, addresses in the third gigabyte become possible return values from memory allocation functions. If you set the "Top down" flag in the memory manager allocation preferences mask (search for "top down"), you can instruct the memory manager to allocate high-address memory first, thereby forcing your program to deal with those addresses sooner than it normally would. This is very handy when testing your program in a /3GB configuration since it forces the troublesome memory addresses to be used sooner than normal.

    Exercise: Find the bug in the following function. Hint: What's today's topic?

    #define BUFFER_SIZE 32768
    BOOL  IsPointerInsideBuffer(const BYTE *p, const BYTE *buffer)
      return p >= buffer && p - buffer < BUFFER_SIZE;
  • The Old New Thing

    Myth: You need /3GB if you have more than 2GB of physical memory


    Physical memory is not virtual address space.

    In my opinion, this is another non sequitur. I'm not sure what logical process led to this myth. It can't be a misapprehension of a 1-1 mapping between physical memory and virtual memory, because that mapping is blatantly not one-to-one. You typically have far more virtual memory than physical memory. Free physical memory doesn't have any manifestation in any virtual address space. And shared memory has manifestations in multiple virtual address spaces yet correspond to the same physical page.

    Though this brings up a historical note.

    In Windows/386, the kernel just so happened to map all physical memory into the kernel-mode virtual address space. There was a function _MapPhysToLinear. You gave it a physical memory range and it returned the base of a range of linear addresses that could be used to access that physical memory. Some driver developers discovered that the kernel mapped all of physical memory and just handed out pointers into that single mapping. As a result, they called _MapPhysToLinear(0, 0x1000) and whenever they wanted to access physical memory in the future, they just added the address to the return value from that single call. In other words, they assumed that

     _MapPhysToLinear(p, x) = _MapPhysToLinear(0, x) + p 

    In Windows 95, the memory manager was completely rewritten and the above coincidence was no longer true. To conserve kernel-mode virtual address space, physical memory was now mapped linearly only as necessary.

    Of course, the drivers that relied on the old behavior were now broken because the undocumented behavior they relied upon was no longer present.

    As a result, when it starts up, Windows 95 looks around to see if any drivers known to rely on this undocumented behavior are loaded. (Windows 3.1 didn't support dynamically-loaded kernel drivers so looking at boot time was sufficient.) If so, then it went ahead and mapped all of physical memory into the kernel-mode virtual address space to keep those driver happy. This wasted virtual address space but kept your machine running.

    I can already hear people saying, "Microsoft shouldn't have made those buggy drivers work. They should have just let the computer crash in order to put pressure on the authors of those drivers to fix their bugs." This assumes, of course, that the cause of the crash could be traced back to the buggy driver in the first place. A very common manifestation of a stray pointer in kernel mode is memory corruption, which means that the component that crashes is rarely the one that caused the problem in the first place.

    For example, nearly all Windows 95 bluescreen crashes in VMM(01) are caused by memory corruption. VMM(01) is the non-swappable part of the Windows 95 kernel which is where the memory manager lives. If a driver corrupts the kernel-mode heap, a bluescreen in the memory manager is typically how the corruption manifests itself.

  • The Old New Thing

    Myth: Without /3GB a single program can't allocate more than 2GB of virtual memory


    Virtual memory is not virtual address space (part 2).

    This myth is being perpetuated even as I write this series of articles.

    The user-mode virtual address space is normally 2GB, but that doesn't limit you to 2GB of virtual memory. You can allocate memory without it being mapped into your virtual address space. (Those who grew up with Expanded Memory or other forms of bank-switched memory are well-familiar with this technique.)

    HANDLE h = CreateFileMapping(INVALID_HANDLE_VALUE, 0,
                                 PAGE_READWRITE, 1, 0, NULL);

    Provided you have enough physical memory and/or swap file space, that 4GB memory allocation will succeed.

    Of course, you can't map it all into memory at once on a 32-bit machine, but you can do it in pieces. Let's read a byte from this memory.

    BYTE ReadByte(HANDLE h, DWORD offset)
     SYSTEM_INFO si;
     DWORD chunkOffset = offset % si.dwAllocationGranularity;
     DWORD chunkStart = offset - chunkOffset;
     LPBYTE pb = (LPBYTE*)MapViewOfFile(h, FILE_MAP_READ, 0,
          chunkStart, chunkOffset + sizeof(BYTE));
     BYTE b = pb[chunkOffset];
     return b;

    Of course, in a real program, you would have error checking and probably a caching layer in order to avoid spending all your time mapping and unmapping instead of actually doing work.

    The point is that virtual address space is not virtual memory. As we have seen earlier, you can map the same memory to multiple addresses, so the one-to-one mapping between virtual memory and virtual address space has already been violated. Here we showed that just because you allocated memory doesn't mean that it has to occupy any space in your virtual address space at all.

    [Updated: 10:37am, fix minor typos reported in comments.]

  • The Old New Thing

    Myth: Without /3GB the total amount of memory that can be allocated across all programs is 2GB


    Virtual memory is not virtual address space (part 1).

    I don't know where this myth comes from; it's a non sequitur.

    Virtual address space describes how addresses are resolved, but since each process has its own virtual address space, the amount consumed by one program has no effect on that consumed by another program.

    Say you have a program that allocates 1GB of memory. Run three copies of it. Now you have a total of 3GB of allocated memory. And none of the programs came even close to exhausting its 2GB virtual address space allotment.

    Tomorrow, the debunking of a variation on this myth.

Page 399 of 448 (4,472 items) «397398399400401»