January, 2006

  • The Old New Thing

    Why does the Recycle Bin have different file system names on FAT and NTFS?


    On FAT drives, the directory that stores files in the Recycle Bin is called C:\RECYCLED, but on NTFS drives, its name is C:\RECYCLER. Why the name change?

    The FAT and NTFS Recycle Bins have different internal structure because NTFS has this thing called "security" and FAT doesn't. All recycled files on FAT drives are dumped into a single C:\RECYCLED directory, whereas recycled files on NTFS drives are separated based on the user's SID into directories named C:\RECYCLER\S-.... (It has nothing to do with whether you are running English or Swedish Windows.)

    Suppose the same directory name were used for both file systems, say, C:\RECYCLED. Since it is possible to upgrade a FAT drive to an NTFS drive with the CONVERT utility, this means that a FAT drive converted to NTFS would have a FAT-style Recycle Bin after the conversion. But since the names are the same, the Recycle Bin says, "Hey, look, here's a C:\RECYCLED directory. That must be my NTFS Recycle Bin!" except that it isn't. It's a FAT Recycle Bin left over from the conversion.

    Giving the NTFS Recycle Bin a different name means that the Recycle Bin shell folder won't get confused by the "wrong" type of recycle bin directory structure on an NTFS volume.

    Yes, the problem could have been solved some other way. For example, there could have been code to inspect the Recycle Bin directory to determine what format it is and ignore it if it didn't match the actual file system. (Or, if you're feeling really ambitious, somehow convert from one format to the other.) But that would be over-engineering. You have to write and test the detection (and possibly conversion) code, there's the risk of a false-positive, the code runs at every boot, and it needs to be maintained whenever either the FAT or NTFS recycle bin format changes. All for a scenario that happens at most once per drive.

    Or you could change one text string and be done with it. (I could make some really awful "Gordian knot"/"string" remark here but will refrain.)

  • The Old New Thing

    "I attack the kobold wearing the headdress made of human ears"


    If you dare, spend eleven minutes of your life watching the most painfully compelling mockumentary on the lives of two basement fantasy role playing gamers. (via Chris Williams.)

  • The Old New Thing

    Why does a corrupted binary sometimes result in "Program too big to fit in memory"?


    If you take a program and corrupt the header, or just take a large-ish file that isn't a program at all and give it a ".exe" extension, then try to run it (Warning: Save your work first!), you will typically get the error "Program too big to fit in memory". Why such a confusing error message? Why doesn't it say "Corrupted program"?

    Because the program isn't actually corrupted. Sort of.

    A Win32 executable file begins with a so-called "MZ" header, followed by a so-called "PE" header. If the "PE" header cannot be found, then the loader attempts to load the program as a Win16 executable file, which consists of an "MZ" header followed by an "NE" header.

    If neither a "PE" nor an "NE" header can be found after the "MZ" header, then the loader attempts to load the program as an MS-DOS relocatable executable. If not even an "MZ" header can be found, then the loader attempt to load the program as an MS-DOS non-relocatable executable (aka "COM format" since this is the format of CP/M .COM files).

    In pictures:

    elseMS-DOS relocatable
    else MS-DOS non-relocatable

    Observe that no matter what path you take through the chart, you will always end up at something. There is no exit path that says "Corrupted program".

    But where does "Program too big to fit in memory" come from?

    If the program header is corrupted, then various fields in the header such as those which specify the amount of memory required by the program will typically be nonsensical values. The loader sees an MS-DOS relocatable program that requires 800KB of conventional memory, and that's where "Out of memory" comes from.

    An MS-DOS non-relocatable program contains no such information about memory requirements. The rule for loading non-relocatable programs is simply to load the program into a single 64KB chunk of memory and set it on its way. Therefore, a program with no "MZ" header but which is larger than 64KB in size won't fit in the single 64KB chunk and consequently results in an "Out of memory" error.

    And since people are certain to ask:

    • "MZ" = the legendary Mark Zbikowski.
    • "NE" = "New Executable", back when Windows was "new".
    • "PE" = "Portable Executable", because one of Windows NT's claims to fame was its portability to architectures other than the x86.
    • "LE" = "Linear Executable", used by OS/2 and by Windows 95 device drivers.
  • The Old New Thing

    Liquefied NBA points apparently cannot be sold on eBay


    Of course we don't find this out until somebody tries. (Google cache for a limited time only.)

  • The Old New Thing

    Waiting for all handles with MsgWaitForMultipleObjects is a bug waiting to happen


    The MsgWaitForMultipleObjects and MsgWaitForMultipleObjectsEx functions allow you to specify whether you want to want for any or all of the handles (either by passing bWaitAll = TRUE or by passing dwFlags = MWMO_WAITALL, accordingly). But you never want to wait for all handles.

    Waiting for all handles means that the call does not return unless all the handles are signalled and a window message meeting your wake criteria has arrived. Since you are typically waiting for messages out of a sense of obligation (keeping the UI thread responsive) rather than one of expectation, even if all the handles you pass become signalled, your program will still stall until a window message arrives. (And if you thought you were being a good UI citizen by using MsgWaitForMultipleObjectsEx, you aren't actually helping any because waiting for all objects means that the call will not return even if a message is ready, since it's also waiting for those handles you passed.) Functions which are built on top of the MsgWaitForMultipleObjectsEx function such as MsgWaitForMultipleObjects and CoWaitForMultipleHandles suffer from the same problem.

    The reason for this can be gleaned from the MsgWaitForMultipleObjectsEx documentation; you just have to put on your thinking cap. Notice that if a message arrives when you are waiting for any handle, the return value is WAIT_OBJECT_0 + cHandles. Notice also that the maximum number of objects you can wait on is MAXIMUM_WAIT_OBJECTS - 1. Obviously, what's happening under the covers is that the MsgWaitForMultipleObjectsEx function creates a handle that will be signalled when the message queue reaches one of the states you requested in the wake mask, adds that handle to the end of the array you passed in, and then passes the whole thing to the WaitForMultipleObjectsEx function. (Note that the getting access to that internal handle won't be of any use to you, the application, since you don't know how to tell the window manager what wait states should result in the event being set.)

    (Larry Osterman reminded me that he covered the same topic a while back. So now you get to see it twice.)

  • The Old New Thing

    Stephen Tolouse's reminiscences of Windows 95 RTM day


    Stephen Tolouse (known around Microsoft as "stepto", pronounced "step-toe") from the Microsoft Security Response Center reminisces about Windows 95 RTM.

    Stephen mentions that "the build numbers were artificially inflated to reach 950". There's actually a technical reason for this inflation, which I intend to write about when I have the time to give the topic the treatment it deserves.

  • The Old New Thing

    Pumping messages while waiting for a period of time


    We can use the MsgWaitForMultipleObjects function (or its superset MsgWaitForMultipleObjectsEx) to carry out a non-polling "sleep while processing messages".

    #define MSGF_SLEEPMSG 0x5300
    BOOL SleepMsg(DWORD dwTimeout)
     DWORD dwStart = GetTickCount();
     DWORD dwElapsed;
     while ((dwElapsed = GetTickCount() - dwStart) < dwTimeout) {
      DWORD dwStatus = MsgWaitForMultipleObjectsEx(0, NULL,
                        dwTimeout - dwElapsed, QS_ALLINPUT,
                        MWFMO_WAITANY | MWMO_INPUTAVAILABLE);
      if (dwStatus == WAIT_OBJECT_0) {
       MSG msg;
       while (PeekMessage(&msg, NULL, 0, 0, PM_REMOVE)) {
        if (msg.message == WM_QUIT) {
         return FALSE; // abandoned due to WM_QUIT
        if (!CallMsgFilter(&msg, MSGF_SLEEPMSG)) {
     return TRUE; // timed out

    This function pumps messages for up to dwTimeout milliseconds. The kernel of the idea is merely to use the MsgWaitForMultipleObjects/Ex function as a surrogate for WaitMessageTimeout, pumping messages until the cumulative timeout has been reached. There are a lot of small details to pay heed to, however. I've linked them to earlier postings that discuss the specific issues, if you need a refresher. The CallMsgFilter you might find gratuitous, but you'll change your mind when you realize that users might press a keyboard accelerator while you're sleeping, and you presumably want it to go through somebody's TranslateAccelerator. The message filter lets you hook into the modal loop and do your accelerator translation.

    Extending this function to "wait on a set of handles up to a specified amount of time, while pumping messages" is left as an exercise. (You can do it without changing very many lines of code.)

    [Call the right function. -2pm]

  • The Old New Thing

    Welcome to the United States, unless you're a Canadian technologist who is an invited guest at a Microsoft conference, in which case, keep out


    Vancouver technologist Darren Barefoot was invited to Redmond by the MSN Search team but was stopped by Immigration and denied entry.

    Ultimately, the customs agents concluded that because Microsoft was covering my flight and accommodation, I was being compensated for consulting activities. In order to enter the country, I'd need a work permit.

    He says Customs, but I'm pretty sure it was really Immigration that stopped him.

    Sorry, Darren. We miss you already. We even asked the rain to take a few days off and invited the sun out to play just for you.

  • The Old New Thing

    You can call MsgWaitForMultipleObjects with zero handles


    There is no WaitMessageTimeout function, but you can create your own with the assistance of the MsgWaitForMultipleObjects function.

    BOOL WaitMessageTimeout(DWORD dwTimeout)
     return MsgWaitForMultipleObjects(
        0, NULL, FALSE, dwTimeout, QS_ALLINPUT)
          == WAIT_TIMEOUT;

    To wait for a message with timeout, we use the MsgWaitForMultipleObjects in a vacuous sense: You pass it a list of objects you want to wait for, as well as a timeout and a set of queue states, asking that the function return when any of the objects is signalled or when a message is ready. By passing no objects, the only thing left to wait for is an incoming message.

    Next time, we'll see how this basic idea can be used to build a slightly more complex function.

    [1/26: Fix call to MsgWaitForMultipleObjects; had it confused with MsgWaitForMultipleObjectsEx. That's what happens when you write entries on an airplane.]

  • The Old New Thing

    Creepy messages in your baggage


    Heather Leigh had a creepy experience with her baggage last week. The story continued yesterday. I'm watching to see how things turn out.

Page 1 of 4 (36 items) 1234