July, 2012

  • The Old New Thing

    Why doesn't the Low Disk Space warning balloon show up as soon as I run low on disk space

    • 66 Comments

    A customer reported an issue with the title "The notification balloon for Low Disk Space does not appear even if the free disk is very low." They provided the following steps:

    • Install Windows 7 64-bit on a SATA drive.
    • Copy files to the system drive until disk space becomes low.
    • Observe that the notification balloon for Low Disk Space does not immediately appear.
    • The balloon appears approximately ten minutes later.

    You read through the steps nodding, "uh huh, uh huh", and then you get to the last step and you say, "Wait a second, the subject of your report was that the balloon doesn't appear at all, and now you're saying that it appears after ten minutes. So it does appear after all. What is the problem?"

    The customer explained that on earlier versions of Windows, the Low Disk Space warning balloon appeared within one minute, whereas in Windows 7 it can take up to ten minutes for the balloon to appear.

    Yup, that's right.

    In previous versions of Windows, Explorer checked for low disk space once a minute. The Windows performance folks requested that the shell reduce the frequency of checks to improve overall system performance, and the shell team agreed to reduce the frequency to once every ten minutes. (The performance team made other suggestions to reduce the impact of that code that runs every ten minutes.)

    So yes, in Windows 7, it may take up to ten minutes for Explorer to report that you are low on disk space. But Explorer never promised that those reports would be timely. Or that they would even appear in the first place. The behavior is not contractual; it's just a courtesy notification.

    Related: How full does a hard drive have to get before Explorer will start getting concerned? and How do I disable the low disk space notifications?

  • The Old New Thing

    One way to make sure you pass an array of the correct size

    • 61 Comments

    Another entry in the very sporadic series of "very strange code I've seen." The code has been changed to protect the guilty, but the essence has been preserved.

    class Store
    {
    public:
        // Retrieve "count" colors from item "itemId" into "values"
        bool GetItemColors(int itemId, int count, COLORREF *values);
    
        // Set "count" colors from "values" into item "itemId"
        bool SetItemColors(int itemId, int count, const COLORREF *values);
    };
    
    bool CopyUpToFourColors(Store *store1, Store *store2, int itemId, int count)
    {
        COLORREF size1[1];
        COLORREF size2[2];
        COLORREF size3[3];
        COLORREF size4[4];
    
        int *buffer = ((count == 1) ? size1 :
                      ((count == 2) ? size2 :
                      ((count == 3) ? size3 :
                      ((count == 4) ? size4 :
                                      nullptr))));
    
        if (buffer == nullptr)
            return false;
    
        if (!store1->GetItemColors(itemId, count, buffer))
            return false;
    
        if (!store2->SetItemColors(itemId, count, buffer))
            return false;
    
        return true;
    }
    
  • The Old New Thing

    Raymond's subjective, unfair, and completely wrong impressions of the opening ceremonies of a major athletic event which took place recently

    • 41 Comments

    Like many other people, I watched the opening ceremonies of a major athletic event which took place a few days ago. (The organization responsible for the event has taken the step of blocking the mention of the name of the city hosting the event and the year the event takes place, or the name of the event itself except in editorial news pieces or journalistic statements of fact, of which this is neither, so I will endeavour to steer clear of the protected marks.)

    I wish somebody had let me know in advance that the opening ceremonies came with a reading list. I hope that at least the British history majors enjoyed it.

    NBC, the media organization which obtained the rights to broadcast the event in the United States, explained that they were not streaming the opening or closing ceremonies live because they "do not translate well online because they require context, which our award-winning production team will provide." And now we learned what sort of contextualization their award-winning production team provided: For Tim Berners-Lee, their valuable context was, "I don't know who that guy is." (The Guardian provides a snarky backgrounder.)

    During the entry of the various national teams, the standard activity is to make fun of their outfits.

    Dear Czech Republic: Spandex shorts and blue rain galoshes? It's as if you're trying to look hideous.

    Dear Germany: Wha??? I'm speechless.

    Dear United States of America: I hope you enjoy your shore leave. (Somebody seriously has a navy fetish going on.)

    Dear Sweden: I know it's late, but you're not supposed to wear your jammy-jams to the opening ceremony. Jag säger bara...

    Dear gracious hosts: Oh, now I get it. It's the 100th anniversary of the sinking of the Titanic. But still, you could've chosen a better tribute than wearing dresses from 1912.

  • The Old New Thing

    Why doesn't RealGetWindowClass return the real window class for my superclass?

    • 41 Comments

    A customer was reporting that the Real­Get­Window­Class function was not reporting the base window class of their superclass. (Error checking has been elided for expository purposes.)

    // Get the static window class window procedure
    WNDCLASS wc;
    GetClassInfo(NULL, TEXT("static"), &wc);
    WNDPROC StaticWndProc = wc.lpfnWndProc;
    
    // Build our derived class
    wc.lpfnWndProc = AwesomeWndProc;
    wc.hInstance = g_hinst;
    wc.lpszClassName = TEXT("AwesomeWindow");
    RegisterClass(&wc);
    
    LRESULT CALLBACK AwesomeWndProc(HWND hwnd, UINT uMsg,
                                    WPARAM wParam, LPARAM lParam)
    {
        TCHAR szClass[128];
        RealGetWindowClass(hwnd, szClass, 128);
        ASSERT(strcmp(szClass, TEXT("static")) == 0);
    
        switch (uMsg) { ... }
    
        return CallWindowProc(StaticWndProc, hwnd, uMsg, wParam, lParam);
    }
    

    The customer found that the assertion fails, returning a window class name of "AwesomeWindow" instead of "static". "I thought the point of RealGetWindowClass was to dig through the superclassing to find the base class. But it's not returning the base class."

    That's right, because you haven't told it what the base class is yet!

    "What do you mean I haven't told it? It's right there at the end of my function: CallWindowProc(StaticWndProc)."

    Yeah, but that line of code hasn't executed yet. The external behavior of your program is like this:

    WNDCLASS wc;
    wc.style = (something);
    wc.lpfnWndProc = AwesomeWndProc;
    wc.cbClsExtra = (something);
    wc.cbWndExtra = (something);
    wc.hInstance = g_hinst;
    wc.hIcon = (something);
    wc.hCursor = (something);
    wc.hbrBackground = (something);
    wc.lpszMenuName = (something);
    wc.lpszClassName = TEXT("AwesomeWindow");
    RegisterClass(&wc);
    
    
    LRESULT CALLBACK AwesomeWndProc(HWND hwnd, UINT uMsg,
                                    WPARAM wParam, LPARAM lParam)
    {
        TCHAR szClass[128];
        RealGetWindowClass(hwnd, szClass, 128);
        ASSERT(strcmp(szClass, TEXT("static")) == 0);
    
        // ... it doesn't matter what goes here
        // because it hasn't executed yet ...
    

    The window manager isn't clairvoyant. It doesn't know that AwesomeWndProc is going to do a CallWindowProc(StaticWndProc) in the future. All it knows is that somebody registered a class, and then in response to its very first message, that class asked, "Hey, you're so smart, tell me what my base class is."

    The window manager says, "Dude, you haven't shown me any base class yet. So I'm just going to say that you are your own base class."

    Since anything can go into the "... it doesn't matter what goes here ...", we can demonstrate that the window manager cannot possibly know what you're going to pass to CallWindowProc by rewriting it like this:

    // Get the static window class window procedure
    WNDCLASS wc;
    GetClassInfo(NULL, TEXT("static"), &wc);
    WNDPROC StaticWndProc = wc.lpfnWndProc;
    
    // Build our class
    wc.lpfnWndProc = AwesomeWndProc;
    wc.hInstance = g_hinst;
    wc.lpszClassName = TEXT("AwesomeWindow");
    RegisterClass(&wc);
    
    LRESULT CALLBACK AwesomeWndProc(HWND hwnd, UINT uMsg,
                                    WPARAM wParam, LPARAM lParam)
    {
        TCHAR szClass[128];
        RealGetWindowClass(hwnd, szClass, 128);
        ASSERT(strcmp(szClass, TEXT("static")) == 0);
    
        switch (uMsg) { ... }
    
        // Psych! You thought that when I asked for StaticWndProc
        // I was going to be a superclass of "static", but in fact
        // I'm just a regular boring window class.
        return DefWindowProc(hwnd, uMsg, wParam, lParam);
    }
    

    If you felt really crazy, you could do this:

    // Get the button window class procedure
    WNDCLASS wcButton;
    GetClassInfo(NULL, TEXT("button"), &wcButton);
    WNDPROC ButtonWndProc = wcButton.lpfnWndProc;
    
    // Get the static window class window procedure
    WNDCLASS wc;
    GetClassInfo(NULL, TEXT("static"), &wc);
    WNDPROC StaticWndProc = wc.lpfnWndProc;
    
    // Build our class
    wc.lpfnWndProc = AwesomeWndProc;
    wc.hInstance = g_hinst;
    wc.lpszClassName = TEXT("AwesomeWindow");
    wc.cbWndExtra = max(wc.cbWndExtra, wcButton.cbWndExtra);
    wc.cbClsExtra = max(wc.cbClsExtra, wcButton.cbClsExtra);
    RegisterClass(&wc);
    
    LRESULT CALLBACK AwesomeWndProc(HWND hwnd, UINT uMsg,
                                    WPARAM wParam, LPARAM lParam)
    {
        TCHAR szClass[128];
        RealGetWindowClass(hwnd, szClass, 128);
        ASSERT(strcmp(szClass, TEXT("static")) == 0);
    
        switch (uMsg) { ... }
    
        // Decide at the last moment what we are.
        static WNDPROC BaseClass = nullptr;
        if (BaseClass == nullptr)
        {
            BaseClass = rand() % 2 ? StaticWndProc : ButtonWndProc;
            // Or if you are particularly perverse,
            // BaseClass = radioactive_decay_has_occurred() ?
            //                StaticWndProc : ButtonWndProc;
        }
        return CallWindowProc(BaseClass, hwnd, uMsg, wParam, lParam);
    }
    

    Since the code to decide the base class hasn't run yet, the window manager will have to use that time machine that the research division has been working on.

  • The Old New Thing

    The continuing battle between people who offer a service and others who want to hack into the service

    • 32 Comments

    In the history of the Internet, there have been many cases of one company providing a service, and others trying to piggyback off the service through a nonstandard client. The result is usually a back-and-forth where the provider changes the interface, the piggybacker reverse-engineers the interface, back and forth, until one side finally gives up.

    Once upon a time, there was one company with a well-known service, and another company that was piggybacking off it. (I first heard this story from somebody who worked at the piggybacking company.) The back-and-forth continued for several rounds, until the provider made a change to the interface that ended the game: They exploited a buffer overflow bug in their own client. The server sent an intentional buffer overflow to the client, resulting in the client being pwned by the server. I'm not sure what happened next, but presumably the server sent some exploit code to the client and waited for the client to respond in a manner that confirmed that the exploit had executed.

    With that discovery, the people from the piggybacking company gave up. They weren't going to introduce an intentional security flaw into their application. The service provider could send not only the exploit but also some code to detect and disable the rogue client.

    By an amazing stroke of good fortune, I happened to also hear the story of this battle from somebody who worked at the provider. He said that they had a lot of fun fighting this particular battle and particularly enjoyed timing the releases so they caused maximum inconvenience for their adversaries, like, for example, 2am on Saturday.

    Reminder: The ground rules prohibit "trying to guess the identity of a program whose name I did not reveal."

  • The Old New Thing

    A brief and also incomplete history of Windows localization

    • 32 Comments
    The process by which Windows has been localized has changed over the years.

    Back in the days of 16-bit Windows, Windows was developed with a single target language: English.

    Just English.

    After Windows was complete and masters were sent off to the factory for duplication, the development team handed the source code over to the localization teams. "Hey, by the way, we shipped a new version of Windows. Go localize it, will ya?"

    While the code that was written for the English version was careful to put localizable content in resources, there were often English-specific assumptions hard-coded into the source code. For example, it may have assumed that the text reading direction was left-to-right or assumed that a single character fit in a single byte. (Unicode hadn't been invented yet.)

    The first assumption is not true for languages such as Hebrew and Arabic (which read right-to-left), and to a lesser degree Chinese and Japanese (which read top-to-bottom in certain contexts). The second assumption is not true for languages like Chinese, Japanese, and Korean, which use DBCS (double-byte character sets).

    The localization teams made the necessary code changes to make Windows work in these other locales and merged them back into the master code base. The result was that there were three different versions of the code for Windows, commonly known as Western, Middle-East, and Far-East. If you wanted Windows to support Chinese, you had to buy the Far-East version of Windows. And since the code was different for the three versions, they had different sets of bugs, and workarounds for one version didn't always work on the others. (Patches didn't exist back then, there being no mechanism for distributing them.)

    If you ran into a problem with a Western language, like say, German, then you were out of luck, since there was no German Windows code base; it used the same Western code base. Windows 95 tried out a crazy idea: Translate Windows into German during the development cycle, to help catch these only-on-German problems while there was still time to do something about it. This, of course, created significant additional expense, since you had to have translators available throughout the product cycle instead of hiring them just once at the end. I remember catching a few translation errors during Windows 95: A menu item Sort was translated as Art (as in "What sort of person would do this?") rather than Sortieren ("put in a prearranged order"). And a command line tool asked the user a yes/no question, promting "J/N" (Ja/Nein), but if you wanted to answer in the affirmative, you had to type "Y".

    The short version of the answer to the question "Why can't the localizers change the code if they have to?" is "Because the code already shipped. What are you going to do, recall every copy of Windows?"

    At least in Windows 95, the prohibition on changing code was violated if circumstances truly demanded them, but doing so was very expensive. The only one I can think of is the change to remove the time zone highlighting from the world map. And the change was done in the least intrusive possible way: Patching four bytes in the binary to make the highlight and not-highlight colors the same. You dare not do something like introduce a new variable; who knows what kinds of havoc could result!

    Having all these different versions of Windows made servicing very difficult, because you had to develop and test a different patch for each code base. Over the years, the Windows team has developed techniques for identifying these potential localization problems earlier in the development cycle. For a time, Windows was "early-localized" into German and Japanese, so as to cover the Western and Far-East scenarios. Arabic was added later, expanding coverage to the Mid-East cases, and Hindi was added in Windows 7 to cover languages which are Unicode-only.

    Translating each internal build of Windows has its pros and cons: The advantage is that it can find issues when there is still time to make code changes to address them. The disadvantage is that code can change while you are localizing, and those code changes can invalidate the work you've done so far, or render it pointless. For example, somebody might edit a dialog you already spent time translating, forcing you to go back and re-translate it, or at least verify that the old translation still works. Somebody might take a string that you translated and start using it in a new way. Unless they let you know about the new purpose, you won't know that the translation needs to be re-evaluated and possibly revised.

    The localization folks came up with a clever solution which gets most of the benefits while avoiding most of the drawbacks: They invented pseudo-localization, which simulates what Michael Kaplan calls "an eager and hardworking yet naïve intern localizer who is going to translate every single string." This was so successful that they hired a few more naïve intern localizers, one which performed "Mirrored pseudo-localization" (covering languages which read right-to-left) and "East Asian pseudo-localization" (covering Chinese, Japanese, and Korean).

    But the rule prohibiting code changes remains in effect. Changing any code resets escrow, which means that the ship countdown clock gets reset back to its original value and all the testing performed up until that point needs to be redone in order to verify that the change did not affect them.

  • The Old New Thing

    What happens when you mark a section as DISCARDABLE?

    • 31 Comments

    In the flags you pass to the linker, you can specify that a section be made discardable. What does that mean?

    If you are a kernel-mode driver, the discardable flag means that the contents will be removed from memory after initialization is complete. This is where you put your initialization code and data.

    But if you're writing user-mode code, the discardable flag has no effect.

    Not relevant to the topic but people are going to ask anyway: The discardable flag on resources also has no effect.

    The discardable attribute for user-mode code is a left-over from 16-bit Windows, which had to simulate a hardware memory manager in software. The rule in 16-bit code was that if you marked a segment or resource as discardable, then when memory ran out, the kernel was allowed to throw the segment away, safe in the knowledge that it could get the information back by reading it from the original image.

    In 32-bit Windows, this marking of discardable versus non-discardable memory is not necessary because the memory manager (with the assistance of hardware) can manage it all transparently. For example, if you never modified a code segment, the memory manager knows that it can simply discard the memory because it can recover the data from the original image. If you allocated some zero-initialized memory and never modified it, then the memory manager can just throw the data away because it is very easy to "recover" a page full of zeroes. On the other hand, if you modified some memory, then there is nowhere the memory manager can go to recover the data, so it has to put it in the page file.

    Bonus chatter: "If discardability is meaningless in user mode, should we just delete it?"

    Well, the PE file format is used for both user-mode and kernel-mode components, so you can't delete it from one and not the other since they are the same thing.

    "I have some code that uses the pragma to make a section discardable. Should I just delete it?"

    Maybe. Or maybe that flag is being used by some other part of your application. After all, the flag exists. Maybe some other part of your program uses it as a "free flag" that it usurps for some other purpose. For example, it might be used as a signal to some post-processing tool to mean "This section is exempt from the frob catalog."

    Hopefully there's a comment that tells you why the section is being marked as discardable, and that will help you decide whether it's safe to remove the marking. Windows doesn't care, but some other part of your program might.

  • The Old New Thing

    Why don't any commands work after I run my batch file? I'm told that they are not recognized as an internal or external command, operable program, or batch file.

    • 30 Comments

    I sort of forgot to celebrate CLR Week last year, so let's say that CLR week is "on hiatus" until next year. To fill the summertime time slot, I'm going to burn off a busted pilot: This week is Batch File Week 2012. Remember, nobody actually enjoys batch programming. It's just something you have to put up with in order to get something done. Batch programming is the COBOL of Windows. (Who knows, if people actually like Batch File Week [fat chance], maybe it'll come back as a regular series.)

    We'll open Batch File Week with a simple puzzle.

    A customer reported that after running their batch file, almost no commands worked any more!

    C:\> awesomebatchfile.bat
    ... awesome batch file does its work ...
    
    C:\> reg query "HKLM\Software\Clients\Mail" /ve
    'reg' is not recognized as an internal or external command,
    operable program or batch file.
    

    Wha? Maybe I can run regedit.

    C:\> regedit
    'regedit' is not recognized as an internal or external command,
    operable program or batch file.
    

    OMG OMG OMG OMG.

    C:\> notepad
    'notepad' is not recognized as an internal or external command,
    operable program or batch file.
    

    Okay, first, sit down and take a deep breath. Maybe take a Chill Pill.

    My first question was "Does awesomebatchfile.bat modify the PATH variable?" (This was, strictly speaking, a psychic debugging question, but a rather obvious one.)

    The customer replied, "Nope. Here, I'll send you the whole thing."

    And there it was, right there at the top of awesomebatchfile.bat:

    set path=C:\awesomedir
    if NOT "%1"=="" set path=%1
    cd /d %path%
    echo Awesomeness commencing in the %path% directory!
    ...
    

    The customer figured it would be convenient to have a variable called path, unaware that this variable has special meaning to the command interpreter. The customer didn't make the connection that their seemingly private variable called path was connected to the system variable of the same name (but by convention capitalized as PATH).

  • The Old New Thing

    You already got your answer, why are you re-asking the question?

    • 29 Comments

    Today's rant is a blend of two earlier rants: People didn't answer your first email for a reason and If you didn't like the answer, asking the same question again is unlikely to help.

    A customer submitted a list of questions (via their customer liaison) to the Widgets discussion list, and somebody wrote up a reply, which was sent back to the customer. So far so good.

    A few days later, the same list of questions was submitted to the Gizmo discussion list via a different customer liaison. Since the question was about Widgets, the question was forwarded to the Widgets discussion list, at which point the same answer was forwarded back. Okay, so now we have a fishing expedition.

    Three weeks later, the same list of question was submitted to the Gizmo discussion list via yet another customer liaison. The fishing expedition continues. The question was once again forwarded to the Widgets discussion list, where the same answer was forwarded back.

    When I asked why the same set of questions was being asked three times, the third customer liaison explained, "The customer is looking for more detail."

    Asking the same question over and over again is not a way to get more detail.

    "By what mechanism does SetWidgetColor inform the widget that its color state has changed?"

    The widget receives an OnColorChanged event.

    "By what mechanism does SetWidgetColor inform the widget that its color state has changed?"

    The widget receives an OnColorChanged event.

    "By what mechanism does SetWidgetColor inform the widget that its color state has changed?"

    The widget receives an OnColorChanged event. Why do you keep asking?

    "I want more details."

    If you want more details, you have to say that you're asking for more details, and you have to say what kind of details you're looking for.

    It turns out that this customer didn't even know what kind of details they wanted. They just wanted to know "everything" about widget color changes.

  • The Old New Thing

    Why hasn't the API hook mechanism for x64 been standardized like it was for x86?

    • 28 Comments
    Joshua posted to the Suggestion Box, "Around the time of WinXP SP2 x86, the API hook mechanism was standardized. Why wasn't the same thing done for x64?"

    Who said it was standardized for x86?

    Hooking APIs is not supported by Windows. There may be specific interfaces that expose hooks (like Co­Register­Initialize­Spy to let you monitor calls to CoInitialize and CoUninitialize, and Set­Windows­Hook­Ex to let you hook various window manager operations) but there is no supported general API hooking mechanism provided by the operating system.

    So I don't know where you got that idea from.

Page 1 of 3 (24 items) 123