October, 2003

  • The Old New Thing

    What's the deal with those reserved filenames like NUL and CON?

    • 22 Comments

    Set the wayback machine to DOS 1.0.

    DOS 1.0 didn't support subdirectories, lowercase, or filenames longer than 8.3.

    When you ran the assembler (or compiler if you were really fancy) the conversation went something like this:

    A>asm foo    the ".asm" extension on "foo" is implied
    Assembler version blah blah blah
    Source file: FOO.ASM
    Listing file [FOO.LST]:
        just hit Enter to accept the default
    Object file [FOO.OBJ]:     just hit Enter to accept the default
    Assembler cranks away

    You only had to type the base name of the file; the ".LST" and ".OBJ" extensions were appended automatically. In fact, I don't think you could disable the extensions; they were always added.

    But what if you didn't want a listing file? The assembler demanded a filename, and if you didn't type any filename at all, it created one with the same basename as your source file.

    That's where the magic filenames come in. Suppose you wanted the listing file to go straight to the printer. You didn't want to create a file on your floppy drive because there might not be enough space to hold it, or just because you didn't want to waste the time creating a file just to delete it anyway. So you typed "PRN" as the filename.

    Now, the assembler doesn't know about these magic filenames. So the assembler will try to create the file "PRN.LST" and then start writing to it. Little does the assembler realize that the output is actually going to the printer.

    If you wanted to discard the output entirely, you would type "NUL", of course. And if you wanted it to go to the screen, you would type "CON".

    Now, if you followed closely, you can see that the above story explains two things already:

    • Why are the magic filenames magical even if I add an extension?

      Answer: If an extension removed the magic, then when the assembler added ".LST" to the filename, it would no longer be recognized as magical, thereby defeating the purpose of the magic.

    • Why do these magic files exist in every directory?

      Answer: Because DOS 1.0 didn't have subdirectories. There was only one directory, which today we would call the root directory, but back then, since there was no such thing as a subdirectory, there was no need to talk about directories in the first place, much less give the only one you have a name. It was just called "the files on your disk". If magic files didn't work in subdirectories, then when you tried to, for example, chdir into a subdirectory and then run the assembler, you wouldn't be able to type "NUL" as the filename and get the magic.

    But why do we carry these magic filenames forward even today?

    Because everybody still relies on them. Just look at all the batch files that do things like redirect to >NUL or test if a directory exists by asking "if exist directoryname\nul", or all the documentation that says to create a file with "copy CON ...".

  • The Old New Thing

    What about BOZOSLIVEHERE and TABTHETEXTOUTFORWIMPS?

    • 58 Comments

    In a previous entry I discussed the story behind the functions with the funny names BEAR, BUNNY and PIGLET. But what about the ones with goofier names like BOZOSLIVEHERE and TABTHETEXTOUTFORWIMPS?

    For this, you need a deeper history lesson.

    Back in the old days of real-mode Windows, all callback functions had to be exported. This was necessary for complicated technical reasons I may bother to explain if anybody really cared but I doubt anybody does any more. So the window procedures for all of the standard window classes (edit controls, list boxes, check boxes, etc.) were exported from USER. So too were various other callback functions like timer procedures. This was in addition to the usual collection of internal functions so USER, KERNEL and GDI could coordinate their efforts.

    Some people reverse-engineered all these internal functions and printed books on how they worked, and as a result there were a lot of programs that actually used them. Which was quite a surprise to us because they were internal functions. And then when we wanted to redesign these internal functions (for example, to add a parameter, or if we decided that we didn't need it any more and tried to delete it), we found found that the progams stopped working.

    So we had to put the functions back, with their old behavior. The new features were were contemplating had to be redesigned, redirected, or possibly even abandoned entirely. (If we wanted to delete a function, then the work could continue, but the old function had to stay around with its old behavior. It was basically dead code from the OS's point of view, hanging around just because some random app or other decided to cheat and bypass the documented way of doing things.) But to teach people a lesson, they often got given goofy names.

    For example, BOZOSLIVEHERE was originally the window procedure for the edit control, with the rather nondescript name of EditWndProc. Then some people who wanted to use the edit control window procedure decide that GetWindowLong(GWL_WNDPROC) was too much typing, so they linked to EditWndProc directly. Then when Windows 2.0 (I think) removed the need to export window procedures, we removed them all, only to find that programs stopped working. So we had to put them back, but they got goofy names as a way of scolding the programs that were doing these invalid things.

    Things got even worse in Windows 95, when all our window procedures were converted to 32-bit versions. The problem is that the old window procedures were only 16-bit. So we couldn't even simply export the 32-bit window procedure under the name BOZOSLIVEHERE. We had to write a conversion function that took an illegal 16-bit function call and converted it to the corresponding illegal 32-bit function call.

    This is just the tip of the iceberg with respect to application compatibility. I could probably write for months solely about bad things apps do and what we had to do to get them to work again (often in spite of themselves). Which is why I get particularly furious when people accuse Microsoft of maliciously breaking applications during OS upgrades. If any application failed to run on Windows 95, I took it as a personal failure. I spent many sleepless nights fixing bugs in third-party programs just so they could keep running on Windows 95. (Games were the worst. Often the game vendor didn't even care that their program didn't run on Windows 95!)
  • The Old New Thing

    Why Daylight Savings Time is nonintuitive

    • 32 Comments
    Welcome Knowledge Base article 932955 readers! Remember, the information on this Web site is not official Microsoft documentation.

    Daylight Savings Time ends this weekend in most of North America and Europe, so it seems a good time to discuss the whole problem of Daylight Savings Time and timestamps.

    A common complaint is that all the time zone conversion functions like FileTimeToLocalFileTime apply the current Daylight Savings Time (DST) bias rather than the bias that was in effect at the time in question.

    For example, suppose you have a FILETIME structure that represents "1 January 2000 12:00AM". If you are in Redmond during the summertime, this converts to "31 December 1999 5:00PM", seven hours difference, even though the time difference between Redmond and UTC was eight hours at that time. (I.e., when people in London were celebrating the new year, it was 4pm in Redmond, not 5pm.)

    The reason is that the time got converted from "1 January 2000 12:00AM UTC" to "31 December 1999 5:00PM PDT". So, technically, the conversion is correct. Of course, nobody was using PDT on December 31, 1999 in Redmond; everybody was on PST.

    Why don't the time zone conversion functions use the time zone appropriate for the time of year?

    One reason is that it means that FileTimeToLocalFileTime and LocalFileTimeToFileTime would no longer be inverses of each other. If you had a local time during the "limbo hour" during the cutover from standard time to daylight time, it would have no corresponding UTC time because there was no such thing as 2:30am local time. (The clock jumped from 2am to 3am.) Similarly, a local time of 2:30am during the cutover from daylight time back to standard time would have two corresponding UTC times.

    Another reason is that the laws regarding daylight savings time are in constant flux. For example, if the year in the example above was 1977 instead of 2000, the conversion would have been correct because the United States was running on year-round Daylight Savings Time due to the energy crisis. Of course, this information isn't encoded anywhere in the TIME_ZONE_INFORMATION structure. Similarly, during World War 2, the United States went on DST all year round. And between 1945 and 1966, the DST rules varied from region to region.

    DST rules are in flux even today. The DST cutover dates in Israel are decided on a year-by-year basis by the Knesset. As a result, there is no deterministic formula for the day, and therefore no way to know it ahead of time.

    (Warning: .NET content ahead; two days in a row, what's gotten into me!?)

    Compare the output of FileInfo.LastWriteTime.ToString("f") with what you see in the property sheet for a file that was last written to on the other side of the DST transition. For example, suppose the file was last modified on October 17, during DST but DST is not currently in effect. Explorer's file properties reports Thursday, October 17, 2003, 8:45:38 AM, but .NETs FileInfo reports Thursday, October 17, 2003, 9:45 AM.

    En gang til for prins Knud: Win32 does not attempt to guess which time zone rules were in effect at that other time. So Win32 says, "Thursday, October 17, 2002 8:45:38 AM PST". Note: Pacific Standard Time. Even though October 17 was during Pacific Daylight Time, Win32 displays the time as standard time because that's what time it is now.

    .NET says, "Well, if the rules in effect now were also in effect on October 17, 2003, then that would be daylight time" so it displays "Thursday, October 17, 2003, 9:45 AM PDT" - daylight time.

    So .NET gives a value which is more intuitively correct, but is also potentially incorrect, and which is not invertible. Win32 gives a value which is intuitively incorrect, but is strictly correct.

    I suspect that the .NET behavior was for compatibility with Visual Basic, but I don't know for sure.

  • The Old New Thing

    Why doesn't the clock in the taskbar display seconds?

    • 27 Comments

    Early beta versions of the taskbar clock did display seconds, and it even blinked the colon like some clocks do. But we had to remove it.

    Why?

    Because that blinking colon and the constantly-updating time were killing our benchmark numbers.

    On machines with only 4MB of memory (which was the minimum memory requirement for Windows 95), saving even 4K of memory had a perceptible impact on benchmarks. By blinking the clock every second, this prevented not only the codepaths related to text rendering from ever being paged out, it also prevented the taskbar's window procedure from being paged out, plus the memory for stacks and data, plus all the context structures related to the Explorer process. Add up all the memory that was being forced continuously present, and you had significantly more than 4K.

    So out it went, and our benchmark numbers improved. The fastest code is code that doesn't run.
  • The Old New Thing

    Why is address space allocation granularity 64K?

    • 8 Comments

    You may have wondered why VirtualAlloc allocates memory at 64K boundaries even though page granularity is 4K.

    You have the Alpha AXP processor to thank for that.

    On the Alpha AXP, there is no "load 32-bit integer" instruction. To load a 32-bit integer, you actually load two 16-bit integers and combine them.

    So if allocation granularity were finer than 64K, a DLL that got relocated in memory would require two fixups per relocatable address: one to the upper 16 bits and one to the lower 16 bits. And things get worse if this changes a carry or borrow between the two halves. (For example, moving an address 4K from 0x1234F000 to 0x12350000, this forces both the low and high parts of the address to change. Even though the amount of motion was far less than 64K, it still had an impact on the high part due to the carry.)

    But wait, there's more.

    The Alpha AXP actually combines two signed 16-bit integers to form a 32-bit integer. For example, to load the value 0x1234ABCD, you would first use the LDAH instruction to load the value 0x1235 into the high word of the destination register. Then you would use the LDA instruction to add the signed value -0x5433. (Since 0x5433 = 0x10000 - 0xABCD.) The result is then the desired value of 0x1234ABCD.

    LDAH t1, 0x1235(zero) // t1 = 0x12350000
    LDA  t1, -0x5433(t1)  // t1 = t1 - 0x5433 = 0x1234ABCD
    

    So if a relocation caused an address to move between the "lower half" of a 64K block and the "upper half", additional fixing-up would have to be done to ensure that the arithmetic for the top half of the address was adjusted properly. Since compilers like to reorder instructions, that LDAH instruction could be far, far away, so the relocation record for the bottom half would have to have some way of finding the matching top half.

    What's more, the compiler is clever and if it needs to compute addresses for two variables that are in the same 64K region, it shares the LDAH instruction between them. If it were possible to relocate by a value that wasn't a multiple of 64K, then the compiler would no longer be able to do this optimization since it's possible that after the relocation, the two variables no longer belonged to the same 64K block.

    Forcing memory allocations at 64K granularity solves all these problems.

    If you have been paying really close attention, you'd have seen that this also explains why there is a 64K "no man's land" near the 2GB boundary. Consider the method for computing the value 0x7FFFABCD: Since the lower 16 bits are in the upper half of the 64K range, the value needs to be computed by subtraction rather than addition. The naïve solution would be to use

    LDAH t1, 0x8000(zero) // t1 = 0x80000000, right?
    LDA  t1, -0x5433(t1)  // t1 = t1 - 0x5433 = 0x7FFFABCD, right?
    

    Except that this doesn't work. The Alpha AXP is a 64-bit processor, and 0x8000 does not fit in a 16-bit signed integer, so you have to use -0x8000, a negative number. What actually happens is

    LDAH t1, -0x8000(zero) // t1 = 0xFFFFFFFF`80000000
    LDA  t1, -0x5433(t1)   // t1 = t1 - 0x5433 = 0xFFFFFFFF`7FFFABCD
    

    You need to add a third instruction to clear the high 32 bits. The clever trick for this is to add zero and tell the processor to treat the result as a 32-bit integer and sign-extend it to 64 bits.

    ADDL t1, zero, t1    // t1 = t1 + 0, with L suffix
    // L suffix means sign extend result from 32 bits to 64
                         // t1 = 0x00000000`7FFFABCD
    

    If addresses within 64K of the 2GB boundary were permitted, then every memory address computation would have to insert that third ADDL instruction just in case the address got relocated to the "danger zone" near the 2GB boundary.

    This was an awfully high price to pay to get access to that last 64K of address space (a 50% performance penalty for all address computations to protect against a case that in practice would never happen), so roping off that area as permanently invalid was a more prudent choice.
  • The Old New Thing

    Why do I get spurious WM_MOUSEMOVE messages?

    • 8 Comments

    In order to understand this properly, it helps to know where WM_MOUSEMOVE messages come from.

    When the hardware mouse reports an interrupt, indicating that the physical mouse has moved, Windows determines which thread should receive the mouse move message and sets a flag on that thread's input queue that says, "The mouse moved, in case anybody cares." (Other stuff happens, too, which we will ignore here for now. In particular, if a mouse button event arrives, a lot of bookkeeping happens to preserve the virtual input state.)

    When that thread calls a message retrieval function like GetMessage, and the "The mouse moved" flag is set, Windows inspects the mouse position and does the work that is commonly considered to be part of mouse movement: Determining the window that should receive the message, changing the cursor, and determining what type of message to generate (usually WM_MOUSEMOVE or perhaps WM_NCMOUSEMOVE).

    If you understand this, then you already see the answer to the question, "Why does my program not receive all mouse messages if the mouse is moving too fast?"

    If your program is slow to call GetMessage, then multiple mouse interrupts may arrive before your program calls GetMessage to pick them up. Since all that happens when the mouse interrupt occurs is that a flag is set, if two interrupts happen in succession without a message retrieval function being called, then the second interrupt will merely set a flag that is already set, which has no effect. The net effect is that the first interrupt acts as if it has been "lost" since nobody bothered to pick it up.

    You should also see the answer to the question, "How fast does Windows deliver mouse movement messages?"

    The answer is, "As fast as you want." If you call GetMessage frequently, then you get mouse messages frequently; if you call GetMessage rarely, then you get mouse messages rarely.

    Okay, so back to the original question, "Why do I get spurious WM_MOUSEMOVE messages?"

    Notice that the delivery of a mouse message includes lots of work that is typically thought of as being part of mouse movement. Often, Windows wants to do that follow-on work even though the mouse hasn't actually moved. The most obvious example is when a window is shown, hidden or moved. When that happens, the mouse cursor may be over a window different from the window it was over previously (or in the case of a move, it may be over a different part of the same window). Windows needs to recalculate the mouse cursor (for example, the old window may have wanted an arrow but the new window wants a pointy finger), so it artificially sets the "The mouse moved, in case anybody cares" flag. This causes all the follow-on work to happen, a side-effect of which is the generation of a spurious WM_MOUSEMOVE message.

    So if your program wants to detect whether the mouse has moved, you need to add a check in your WM_MOUSEMOVE that the mouse position is different from the position reported by the previous WM_MOUSEMOVE message.
  • The Old New Thing

    What is the Alt+Tab order?

    • 23 Comments

    What determines the order in which icons appear in the Alt+Tab list?

    The icons appear in the same order as the window Z-order. When you switch to a window, then it comes to the top of the Z-order. If you minimize a window, it goes to the bottom of the Z-order. The Alt+Esc hotkey (gosh, does anybody still use Alt+Esc?) takes the current top window and sends it to the bottom of the Z-order (and the window next in line comes to the top). The Alt+Shift+Esc hotkey (I bet you didn't know that hotkey even existed) takes the bottom-most window and brings it to the top, but does not open the window if it is minimized.

    The presence of "always on top" windows makes this a little more complicated. The basic rule is that an "always on top" window always appears on top of a "not always on top" window. So if the above rules indicate that a "not always on top" window comes to the top, it really just goes as high as it can without getting on top of any "always on top" windows.

    You may have run across the term "fast task switching". This was the term used to describe the precursor to the current Alt+Tab switching interface. The old way of switching via Alt+Tab (Windows 3.0 and earlier) was just like Alt+Esc, except that the window you switched to was automatically opened if it had been minimized. When the new Alt+Tab was added to Windows 3.1, we were concerned that people might prefer the old way, so there was a switch in the control panel to set it back to the slow way. (There is also a setting SPI_SETFASTTASKSWITCH that lets you change it programmatically.) It turns out nobody complained, so the old slow way of task switching was removed entirely and the setting now has no effect.

    This does highlight the effort we take to try to allow people who don't like the new way of doing something to go back to the old way. It turns out that corporations with 10,000 employees don't like it when the user interface changes, because it forces them to spend millions of dollars retraining all their employees. If you open up the Group Policy Editor, you can see the zillions of deployment settings that IT administrators can use to disable a variety of new Windows UI features.

  • The Old New Thing

    Why is there no WM_MOUSEENTER message?

    • 6 Comments

    There is a WM_MOUSELEAVE message. Why isn't there a WM_MOUSEENTER message?

    Because you can easily figure that out for yourself.

    When you receive a WM_MOUSELEAVE message, set a flag that says, "The mouse is outside the window." When you receive a WM_MOUSEMOVE message and the flag is set, then the mouse has entered the window. (And clear the flag while you're there.)

    Let's illustrate this with our sample program, just to make sure you get the idea:

    BOOL g_fMouseInClient;
    void OnMouseMove(HWND hwnd, int x, int y, UINT keyFlags)
    {
        if (!g_fMouseInClient) {
            g_fMouseInClient = TRUE;
            MessageBeep(0);
            TRACKMOUSEEVENT tme = { sizeof(tme) };
            tme.dwFlags = TME_LEAVE;
            tme.hwndTrack = hwnd;
            TrackMouseEvent(&tme);
        }
    }
        case WM_MOUSELEAVE: g_fMouseInClient = FALSE; return 0;
        HANDLE_MSG(hwnd, WM_MOUSEMOVE, OnMouseMove);
    

    This program beeps whenever the mouse enters the client area.

    Exercise: If the program starts with the mouse already in the client area without moving, why do you get a beep?
  • The Old New Thing

    I'm doing this instead of writing a book

    • 24 Comments

    Some commenters mentioned that I should write a book. It turns out that writing a book is hard.

    A few years ago, MS Press actually approached me about writing a book for them. But I declined because the fashion for technical books is to take maybe fifty pages of information and pad it to a 700-page book, and I can't write that way. None of my topics would ever make it to a 100-page chapter. They're just little page-and-a-half vignettes. And it's not like the world needs yet another book on Win32 programming.

    So I'll just continue to babble here. It's easier.

  • The Old New Thing

    Stupid memory-mapping tricks

    • 9 Comments

    Shared memory is not just for sharing memory with other processes. It also lets you share memory with yourself in sneaky ways.

    For example, this sample program (all error checking and cleanup deleted for expository purposes) shows how you can map the same shared memory into two locations simultaneously. Since they are the same memory, modifications to one address are reflected at the other.

    #include <windows.h>
    #include <stdio.h>
    
    void __cdecl main(int argc, char **argv)
    {
        HANDLE hfm = CreateFileMapping(INVALID_HANDLE_VALUE, NULL,
                        PAGE_READWRITE, 0, sizeof(DWORD), NULL);
    
        LPDWORD pdw1 = (LPDWORD)MapViewOfFile(hfm, FILE_MAP_WRITE,
                                              0, 0, sizeof(DWORD));
    
        LPDWORD pdw2 = (LPDWORD)MapViewOfFile(hfm, FILE_MAP_WRITE,
                                              0, 0, sizeof(DWORD));
    
        printf("Mapped to %x and %x\n", pdw1, pdw2);
    
        printf("*pdw1 = %d, *pdw2 = %d\n", *pdw1, *pdw2);
    
        /* Now watch this */
        *pdw1 = 42;
        printf("*pdw1 = %d, *pdw2 = %d\n", *pdw1, *pdw2);
    }
    

    This program prints

    
        
    Mapped to 280000 and 290000
    *pdw1 = 0, *pdw2 = 0
    *pdw1 = 42, *pdw2 = 42
    

    (Missing asterisks added, 8am - thanks to commenter Tom for pointing this out.)

    The addresses may vary from run to run, but observe that the memory did get mapped to two different addresses, and changing one value to 42 magically changed the other.

    This is a nifty consequence of the way shared memory mapping works. I stumbled across it while investigating how I could copy large amounts of memory without actually copying it. The solution: Create a shared memory block, map it at one location, write to it, then unmap it from the old location and map it into the new location. Presto: The memory instantly "moved" to the new location. This a major win if the memory block is large, since you didn't have to allocate a second block, copy it, then free the old block - the memory block doesn't even get paged in.

    It turns out I never actually got around to using this trick, but it was a fun thing to discover anyway.
Page 1 of 4 (37 items) 1234