• The Old New Thing

    Why does Exchange recommend /3GB if you have more than 1GB of physical memory?


    If you look through the Knowledge Base, you'll see an article that say that Exchange 2000 requires the /3GB switch with more than 1 gigabyte of physical RAM. Yet I've been writing all this time that /3GB has nothing to do with physical RAM. What's the deal?

    The title of the article could be a bit clearer. It really should be something more like "Exchange 2000 requires the /3GB switch to take advantage of more than 1 gigabyte of physical RAM".

    It appears that Exchange 2000 doesn't use the bank-switching technique I described in an earlier entry. (I don't blame them. It's extraordinarily cumbersome.) Consequently, for Exchange 2000, virtual address space equals virtual memory.

    The capacity of a program is typically a combination of multiple factors, the lowest of which sets the limit. By analogy, suppose you need two piece of bread, two pieces of bologna, and a slice of cheese to make a bologna and cheese sandwich. Whichever ingredient you run out of first determines how many sandwiches you can make. If you run out of cheese first, adding more bologna won't help any.

    Okay, so what do bologna and cheese sandwiches have to do with Exchange 2000?

    From the description in the article, it appears that the store.exe program is RAM-constrained most of the time (you run out of cheese first). But once you get the memory on the machine up to one gigabyte, you have excess RAM and address space becomes the new limiting factor. (You added plenty of cheese and now you've run out of bologna.) That's where the /3GB switch comes in. It increases the user-mode address space, thereby relieving pressure on the address space constraint.

  • The Old New Thing

    Myth: The /3GB switch expands the user-mode address space of all programs


    Only programs marked as /LARGEADDRESSAWARE are affected.

    For compatibility reasons, only programs that explicitly indicate that they are prepared to handle a virtual address space larger than 2GB will get the larger virtual address space. Unmarked programs get the normal 2GB virtual address space, and the address space between 2GB and 3GB goes unused.


    Because far too many programs assume that the high bit of user-mode virtual addresses is always clear, often unwittingly. MSDN has a page listing some of the ways programs make this assumption. One such assumption you may be making is taking the midpoint between two pointers by using the formula (a+b)/2. As I noted in a previous exercise, this is subject to integer overflow and consequently can result in an erroneous pointer computation. Consequently, you can't just take an existing program that you didn't write, mark it /LARGEADDRESSAWARE, and declare your job done. You have to check with the authors of that program that they verified that their code does not make any 2GB assumptions. (And the fact that the authors didn't mark their program as 3GB-compatible strongly suggests that no such verification has occurred. If it had, they would have marked the program /LARGEADDRESSAWARE!)

    Marking your program /LARGEADDRESSAWARE indicates to the operating system, "Go ahead and give this program access to that extra gigabyte of user-mode address space," and as a result, addresses in the third gigabyte become possible return values from memory allocation functions. If you set the "Top down" flag in the memory manager allocation preferences mask (search for "top down"), you can instruct the memory manager to allocate high-address memory first, thereby forcing your program to deal with those addresses sooner than it normally would. This is very handy when testing your program in a /3GB configuration since it forces the troublesome memory addresses to be used sooner than normal.

    Exercise: Find the bug in the following function. Hint: What's today's topic?

    #define BUFFER_SIZE 32768
    BOOL  IsPointerInsideBuffer(const BYTE *p, const BYTE *buffer)
      return p >= buffer && p - buffer < BUFFER_SIZE;
  • The Old New Thing

    Myth: You need /3GB if you have more than 2GB of physical memory


    Physical memory is not virtual address space.

    In my opinion, this is another non sequitur. I'm not sure what logical process led to this myth. It can't be a misapprehension of a 1-1 mapping between physical memory and virtual memory, because that mapping is blatantly not one-to-one. You typically have far more virtual memory than physical memory. Free physical memory doesn't have any manifestation in any virtual address space. And shared memory has manifestations in multiple virtual address spaces yet correspond to the same physical page.

    Though this brings up a historical note.

    In Windows/386, the kernel just so happened to map all physical memory into the kernel-mode virtual address space. There was a function _MapPhysToLinear. You gave it a physical memory range and it returned the base of a range of linear addresses that could be used to access that physical memory. Some driver developers discovered that the kernel mapped all of physical memory and just handed out pointers into that single mapping. As a result, they called _MapPhysToLinear(0, 0x1000) and whenever they wanted to access physical memory in the future, they just added the address to the return value from that single call. In other words, they assumed that

     _MapPhysToLinear(p, x) = _MapPhysToLinear(0, x) + p 

    In Windows 95, the memory manager was completely rewritten and the above coincidence was no longer true. To conserve kernel-mode virtual address space, physical memory was now mapped linearly only as necessary.

    Of course, the drivers that relied on the old behavior were now broken because the undocumented behavior they relied upon was no longer present.

    As a result, when it starts up, Windows 95 looks around to see if any drivers known to rely on this undocumented behavior are loaded. (Windows 3.1 didn't support dynamically-loaded kernel drivers so looking at boot time was sufficient.) If so, then it went ahead and mapped all of physical memory into the kernel-mode virtual address space to keep those driver happy. This wasted virtual address space but kept your machine running.

    I can already hear people saying, "Microsoft shouldn't have made those buggy drivers work. They should have just let the computer crash in order to put pressure on the authors of those drivers to fix their bugs." This assumes, of course, that the cause of the crash could be traced back to the buggy driver in the first place. A very common manifestation of a stray pointer in kernel mode is memory corruption, which means that the component that crashes is rarely the one that caused the problem in the first place.

    For example, nearly all Windows 95 bluescreen crashes in VMM(01) are caused by memory corruption. VMM(01) is the non-swappable part of the Windows 95 kernel which is where the memory manager lives. If a driver corrupts the kernel-mode heap, a bluescreen in the memory manager is typically how the corruption manifests itself.

  • The Old New Thing

    Myth: Without /3GB a single program can't allocate more than 2GB of virtual memory


    Virtual memory is not virtual address space (part 2).

    This myth is being perpetuated even as I write this series of articles.

    The user-mode virtual address space is normally 2GB, but that doesn't limit you to 2GB of virtual memory. You can allocate memory without it being mapped into your virtual address space. (Those who grew up with Expanded Memory or other forms of bank-switched memory are well-familiar with this technique.)

    HANDLE h = CreateFileMapping(INVALID_HANDLE_VALUE, 0,
                                 PAGE_READWRITE, 1, 0, NULL);

    Provided you have enough physical memory and/or swap file space, that 4GB memory allocation will succeed.

    Of course, you can't map it all into memory at once on a 32-bit machine, but you can do it in pieces. Let's read a byte from this memory.

    BYTE ReadByte(HANDLE h, DWORD offset)
     SYSTEM_INFO si;
     DWORD chunkOffset = offset % si.dwAllocationGranularity;
     DWORD chunkStart = offset - chunkOffset;
     LPBYTE pb = (LPBYTE*)MapViewOfFile(h, FILE_MAP_READ, 0,
          chunkStart, chunkOffset + sizeof(BYTE));
     BYTE b = pb[chunkOffset];
     return b;

    Of course, in a real program, you would have error checking and probably a caching layer in order to avoid spending all your time mapping and unmapping instead of actually doing work.

    The point is that virtual address space is not virtual memory. As we have seen earlier, you can map the same memory to multiple addresses, so the one-to-one mapping between virtual memory and virtual address space has already been violated. Here we showed that just because you allocated memory doesn't mean that it has to occupy any space in your virtual address space at all.

    [Updated: 10:37am, fix minor typos reported in comments.]

  • The Old New Thing

    Myth: Without /3GB the total amount of memory that can be allocated across all programs is 2GB


    Virtual memory is not virtual address space (part 1).

    I don't know where this myth comes from; it's a non sequitur.

    Virtual address space describes how addresses are resolved, but since each process has its own virtual address space, the amount consumed by one program has no effect on that consumed by another program.

    Say you have a program that allocates 1GB of memory. Run three copies of it. Now you have a total of 3GB of allocated memory. And none of the programs came even close to exhausting its 2GB virtual address space allotment.

    Tomorrow, the debunking of a variation on this myth.

  • The Old New Thing

    Kernel address space consequences of the /3GB switch


    One of the adverse consequences of the /3GB switch is that it forces the kernel to operate inside a much smaller space.

    One of the biggest casualties of the limited address space is the video driver. To manage the memory on the video card, the driver needs to be able to address it, and the apertures required are typically quite large. When the video driver requests a 256MB aperture, the call is likely to fail since there simply isn't that much address space available to spare.

    All of kernel's bookkeeping needs to fit inside that one gigabyte. Page tables, page directories, bitmaps, video driver apertures. It's a very tight squeeze, but if you're willing to cut back (for example by not requiring such a large video aperture), you can barely squeak it through. (A later entry will discuss another casualty of the reduced address space.)

    It's like trying to change your clothes inside a small closet. You can do it, but it's a real struggle, you're going to have to make sacrifices, and the results aren't always very pretty.

  • The Old New Thing

    The oft-misunderstood /3GB switch


    It's simple to explain what it does, but people often misunderstand.

    The /3GB switch changes the way the 4GB virtual address space is split up. Instead of splitting it as 2GB of user mode virtual address space and 2GB of kernel mode virtual address space, the split is 3GB of user mode virtual address space and 1GB of kernel mode virtual address space.

    That's all.

    And yet people think it does more than that.

    I think the problem is that people think that "virtual address space" means something other than just "virtual address space".

    The term "address space" refers to how a numerical value (known as an "address") is interpreted when it is used to access some type of resource. There is a physical address space; each address in the physical address space refers to a byte in a memory chip somewhere. (Note for pedants: Yes, it's actually spread out over several memory chips, but that's not important here.) There is an I/O address space; each address in the I/O address space allows the CPU to communicate with a hardware device.

    And then there is the virtual address space. When people say "address space", they usually mean "virtual address space".

    The virtual address space is the set of possible pointer values (addresses) that can be used at a single moment by the processor. In other words, if you have an address like 0x12345678, the virtual address space determines what you get if you try to access that memory. The contents of the virtual address space changes over time, for example, as you allocate and free memory. It also varies based on context: each process has its own virtual address space.

    Saying that 2GB (or 3GB) of virtual address space is available to user mode means that at any given moment in time, out of the 4 billion virtual addresses available in a 32-bit value, 2 billion (or 3 billion) of them are potentially usable by user-mode code.

    Over the next few entries, I'll talk about the various consequences and misinterpretations of the /3GB switch.

  • The Old New Thing

    Why .shared sections are a security hole


    Many people will recommend using shared data sections as a way to share data between multiple instances of an application. This sounds like a great idea, but in fact it's a security hole.

    Proper shared memory objects created by the CreateFileMapping function can be secured. They have security descriptors that let you specify which users are allowed to have what level of access. By contrast, anybody who loads your EXE or DLL gets access to your shared memory section.

    Allow me to demonstrate with an intentionally insecure program.

    Take the scratch program and make the following changes:

    #pragma comment(linker, "/SECTION:.shared,RWS")
    #pragma data_seg(".shared")
    int g_iShared = 0;
    #pragma data_seg()
    void CALLBACK TimerProc(HWND hwnd, UINT, UINT_PTR, DWORD)
      int iNew = g_iShared + 1;
      if (iNew == 10) iNew = 0;
      g_iShared = iNew;
      InvalidateRect(hwnd, NULL, TRUE);
    OnCreate(HWND hwnd, LPCREATESTRUCT lpcs)
        SetTimer(hwnd, 1, 1000, TimerProc);
        return TRUE;
    PaintContent(HWND hwnd, PAINTSTRUCT *pps)
      TCHAR sz[2];
      wsprintf(sz, TEXT("%d"), g_iShared);
      TextOut(pps->hdc, 0, 0, sz, 1);

    Go ahead and run this program. It counts from 0 to 9 over and over again. Since the TimerProc function never lets g_iShared go above 9, the wsprintf is safe from buffer overflow.

    Or is it?

    Run this program. Then use the runas utility to run a second copy of this program under a different user. For extra fun, make one of the users an administrator and another a non-administrator.

    Notice that the counter counts up at double speed. That's to be expected since the counter is shared.

    Okay, now close one of the copies and relaunch it under a debugger. (It's more fun if you let the administrator's copy run free and run the non-administrator's copy run under a debugger.) Let both programs run, then break into the debugger and change the value of the variable g_iShared to something really big, say, 1000000.

    Now, depending on how intrusive your debugger is, you might or might not see the crash. Some debuggers are "helpful" and "unshare" shared memory sections when you change their values from the debugger. Helpful for debugging (maybe), bad for my demonstration (definitely).

    Here's how I did it with the built-in ntsd debugger. I opened a command prompt, which runs as myself (and I am not an administrator). I then used the runas utility to run the scratch program as administrator. It is the administrator's copy of the scratch program that I'm going to cause to crash even though I am just a boring normal non-administrative user.

    From the normal command prompt, I typed "ntsd scratch" to run the scratch program under the debugger. From the debugger prompt, I typed "u TimerProc" to disassemble the TimerProc function, looking for

    01001143 a300300001       mov     [scratch!g_iShared (01003000)],eax
    (note: your numbers may differ). I then typed "g 1001143" to instruct the debugger to execute normally until that instruction is reached. When the debugger broke, I typed "r eax=12341234;t" to change the value of the eax register to 0x12341324 and then trace one instruction. That one-instruction trace wrote the out-of-range value into shared memory, and one second later, the administrator version of the program crashed with a buffer overflow.

    What happened?

    Since the memory is shared, all running copies of the scratch program have access to it. ALl I did was use the debugger to run a copy of the scratch program and change the value of the shared memory variable. Since the variable is shared, the value also changes in the administrator's copy of the program, which then causes the wsprintf buffer to overflow, thereby crashing the administrator's copy of the program.

    A denial of service is bad enough, but you can really do fun things if a program keeps anything of value in shared memory. If there is a pointer, you can corrupt the pointer. If there is a string, you can remove the null terminator and cause it to become "impossibly" long, resulting in a potential buffer overflow if somebody copies it without checking the length.

    And if there is a C++ object with a vtable, then you have just hit the mother lode! What you do is redirect the vtable to a bogus vtable (which you construct in the shared memory section), and put a function pointer entry in that vtable that points into some code that you generated (also into the shared memory section) that takes over the machine. (If NX is enabled, then the attack is much harder but still possible in principle.)

    Even if you can't trigger a buffer overflow by messing with variables in shared memory, you can still cause the program to behave erratically. Just scribbling random numbers all over the shared memory section will certainly induce "interesting" behavior in the program under attack.

    Moral of the story: Avoid shared memory sections. Since you can't attach an ACL to the section, anybody who can load your EXE or DLL can modify your variables and cause havoc in another instance of the program that is running at a higher security level.

  • The Old New Thing

    Spammers look stupid when they don't read the blog they spam on


    Yesterday, I got a 419 scam via the contact form. This is a new low for spam stupidity. You'd think people who blog are more likely to be aware of Internet scams because they're reading the news, are clearly more comfortable with technology, and often make fun of news stories about 419 scammers.

    But if you're for real, I have this to say to you: Be careful! I'm sure you are being completely truthful when you say you are the only child of Deposed Corrupt Dictator XYZ, but be aware that there are many unscrupulous people claiming the same thing! So watch out for Vivian, Pierre, Lama, Austine, Martin, Kabilla, Jean Paul, Farek, David, Curimbada, Cynthia, and of course, Dictator Junior. It's very disappointing that there are so many impostors trying to get their hands on the fortune that you stole and that by rights belongs to you.

    Also watch out for XYZ's former Finance Minister, accountant, widow, brother, and hair stylist. They're after the same thing.

    Good luck.

    (I also find it impressive that you were able to embezzle $100 billion U.S. dollars, since your country raises less than $2 billion per year in tax revenues. How you managed to amass that much money so quickly is truly an amazing accomplishment of economics. If you returned even just half of that money back to the country in exchange for amnesty, that would fund your government for a generation. Think about it. Do it for the children.)

    P.S. I was joking about the hair stylist. Please don't hurt him.

  • The Old New Thing

    Never leave focus on a disabled control


    One of the big no-no's in dialog box management is disabling the control that has focus without first moving focus somewhere else. When you do this, the keyboard becomes dead to the dialog box, since disabled windows do not receive input. For users who don't have a mouse (say, because they have physical limitations that confine them to the keyboard), this kills your dialog box.

    (I've seen this happen even in Microsoft software. It's very frustrating.)

    Before you disable a control, check whether it has focus. If so, then move focus somewhere else before you disable it, so that the user isn't left stranded.

    If you don't know which control focus should go to, you can always let the dialog manager decide. The WM_NEXTDLGCTL message once again comes to the rescue.

    void DialogDisableWindow(HWND hdlg, HWND hwndControl)
      if (hwndControl == GetFocus()) {
        SendMessage(hdlg, WM_NEXTDLGCTL, 0, FALSE);
      EnableWindow(hwndControl, FALSE);

    (And of course you should never disable the last control on a dialog. That would leave the user completely stranded with no hope of escape!)

    [This was supposed to go out yesterday but the autoblog tool had a bad day and forgot to post this. Sorry.]

Page 399 of 447 (4,467 items) «397398399400401»