• The Old New Thing

    What is the correct way of temporarily changing a thread's preferred UI language?

    • 10 Comments

    A customer ran into a crashing bug in their shell extension. The shell extension wants to change the thread's preferred UI language temporarily, so that it can load its resources from a specific language. You'd think this would be easy:

    // error checking elided for simplicity
    // There is a bug in this code - read on
    
    // Get the current thread preferred UI languages
    ULONG cLanguages;
    PZZWSTR pszzPrevLanguages;
    ULONG cchPrevLanguages = 0;
    GetThreadPreferredUILanguages(MUI_LANGUAGE_NAME,
                                  &cLanguages, NULL,
                                  &cchPrevLanguages);
    pszzPrevLanguages = new WCHAR[cchPrevLanguages];
    GetThreadPreferredUILanguages(MUI_LANGUAGE_NAME,
                                  &cLanguages,
                                  pszzPrevLanguages,
                                  &cchPrevLanguages);
    
    ... change the thread preferred UI languages ...
    ... load resources ...
    
    // Restore the original thread preferred UI languages
    SetThreadPreferredUILanguages(MUI_LANGUAGE_NAME,
                                  pszzPrevLanguages,
                                  &cLanguages);
    delete[] pszzPrevLanguages;
    

    Approximately ten seconds after this code runs, Explorer crashes with the exception STATUS_CALLBACK_RETURNED_LANG whose description is "A threadpool worker thread enter a callback, which left with preferred languages set. This is unexpected, indicating that the callback missed clearing them." (Just before Explorer crashes, the message "ThreadPool: callback 77180274(05B67430) returned with preferred languages set" appears on the debugger, which says basically the same thing as the status code.)

    Exercise: Why does it take ten seconds before the crash occurs?

    This crash is puzzling, because it's claiming that the callback didn't reset the thread preferred languages, but you can see us doing it right there in the code when we call Set­Thread­Preferred­UI­Languages! Somebody's on crack, but who?

    A closer reading of the error message indicates that the callback needs to "clear" the thread preferred languages, not merely reset them to their original values, and the documentation for Set­Thread­Preferred­UI­Languages says, "To clear the thread preferred UI languages list, the application can set this parameter to a null string or an empty double null-terminated string." Okay, so now the question is, "How can I tell, when I call Get­Thread­Preferred­UI­Languages, that the list of languages I receive back represents the clear state as opposed to indicating that some other code called Set­Thread­Preferred­UI­Languages before I did?"

    The magic is the flag MUI_THREAD_LANGUAGES. If you pass this flag when you call Get­Thread­Preferred­UI­Languages, it will return a null string if the thread has not customized its preferred UI languages, indicating that the way to restore the thread's preferred UI language state is to clear it rather than setting it. Fortunately, this lines up nicely with the way you're supposed to clear the state, so at the end of the day there is no special case.

    The fix to the above code, then, is to make the following simple change:

    // error checking elided for simplicity
    
    // Get the current thread preferred UI languages
    ULONG cLanguages;
    PZZWSTR pszzPrevLanguages;
    ULONG cchPrevLanguages = 0;
    GetThreadPreferredUILanguages(MUI_LANGUAGE_NAME |
                                  MUI_THREAD_LANGUAGES,
                                  &cLanguages, NULL,
                                  &cchPrevLanguages);
    pszzPrevLanguages = new WCHAR[cchPrevLanguages];
    GetThreadPreferredUILanguages(MUI_LANGUAGE_NAME |
                                  MUI_THREAD_LANGUAGES,
                                  &cLanguages,
                                  pszzPrevLanguages,
                                  &cchPrevLanguages);
    
    ... change the thread preferred UI languages ...
    ... load resources ...
    
    // Restore the original thread preferred UI languages
    SetThreadPreferredUILanguages(MUI_LANGUAGE_NAME,
                                  pszzPrevLanguages,
                                  &cLanguages);
    delete[] pszzPrevLanguages;
    

    As you might expect, Michael Kaplan has his own thoughts on the Set­Thread­Preferred­UI­Languages function. In fact, he has several such thoughts.

  • The Old New Thing

    The __fortran calling convention isn't the calling convention used by FORTRAN

    • 36 Comments

    Although the Microsoft C compiler supports a calling convention called __fortran, that's just what the calling convention is called; its relationship with the FORTRAN programming language is only coincidental. The __fortran keyword is now just an old-fashioned synonym for __stdcall.

    Various FORTRAN compilers use different calling conventions; the one I describe here applies to the now-defunct Microsoft Fortran PowerStation.

    Fortran Powerstation pushes parameters on the stack right-to-left, with callee-cleanup. (So far, this matches __fortran aka __stdcall.) Function names are converted to all-uppercase, with an underscore at the beginning and @n appended, where n is the number of bytes of parameters. (This still matches __stdcall aside from the uppercase conversion.)

    As for how the parameters are passed, well, that's where things get weird. FORTRAN natively passes all parameters by reference. This is the source of a famous classic FORTRAN bug known as constants aren't.

          PROGRAM MYSTERY
          CALL MAGIC(1)
          PRINT *, 'According to the computer, 3 + 1 is ', ADDUP(3, 1)
          END
    
          FUNCTION ADDUP(I, J)
          ADDUP = I + J
          END
    
    C     What does this subroutine actually do?
          SUBROUTINE MAGIC(I)
          I = 9
          RETURN
    
          END
    

    (It's been a long time since I've written a FORTRAN program, so I may have gotten some of the details wrong, but any errors shouldn't detract from the fundamental issue.)

    When you run this program, it says

    According to the computer, 3 + 1 is 12
    

    How did that happen? We called a function that adds two numbers together, and instead of getting 4, we get 12?

    The reason is the subroutine MAGIC: We passed it the constant 1, and since all FORTRAN parameters are passed by reference, the assignment I = 9 modifies the constant 1. In C:

    int One = 1;
    int Three = 3;
    int Nine = 9;
    
    void Magic(int *i) { *i = Nine; }
    int AddUp(int *i, int *j) { return *i + *j; }
    
    void main()
    {
     Magic(&One);
     printf("According to the computer, 3 + 1 is %d\n",
            AddUp(&Three, &One));
    }
    

    Since Magic modified the constant One, any further use of the constant 1 ends up using the value 9! (According to the FORTRAN standard, modifying a constant results in undefined behavior.)

    Okay, back to calling conventions. Other significant differences between C and FORTRAN: In FORTRAN, array indices begin at 1, not 0, and arrays are stored in column-major order rather than row-major as in C.

    COMPLEX variables in FORTRAN are stored as two floating point numbers (corresponding to the real and imaginary components).

    Functions which return COMPLEX or CHARACTER*(*) are internally rewritten as subroutines where the location to store the return value is passed as a hidden first parameter. (This is analogous to how C returns large structures.)

    The final commonly-encountered weirdness of FORTRAN is that CHARACTER*n data types (which are used to hold strings) are passed as two parameters: The address of the character buffer, followed by the size of the buffer (n). Note that FORTRAN CHARACTER*n variables are fixed-length; if you assign a string shorter than the buffer, it is padded with spaces. There is no null terminator.

    Anyway, I sort of got carried away with the FORTRAN calling convention. It's definitely more complicated than just sticking __fortran in front of your function. But at least the __fortran keyword takes care of the part that can't be expressed in C. The rest you can manage on your own.

  • The Old New Thing

    How do I simulate input without SendInput?

    • 15 Comments

    Michal Zygmunt wants to create a system where multiple applications can have focus, with different users generating input and directing them at their target applications. Attempting to simulate this by posting input messages didn't work. "Can you tell us maybe how SendInput is internally implemented so that we can use it to simulate only part of the actions (like without acquiring focus)?"

    SendInput operates at the bottom level of the input stack. It is just a backdoor into the same input mechanism that the keyboard and mouse drivers use to tell the window manager that the user has generated input. The SendInput function doesn't know what will happen to the input. That is handled by much higher levels of the window manager, like the components which hit-test mouse input to see which window the message should initially be delivered to. So if your goal is to change the way you call SendInput so it changes the focus management rules, you're barking up the wrong tree. It's like asking, "Please tell me how RAM chips work so I can use it to change the way Lotus 1-2-3 resolves circular references."

  • The Old New Thing

    Developing the method for taking advantage of the fact that the OVERLAPPED associated with asynchronous I/O is passed by address

    • 26 Comments

    You can take advantage of the fact that the OVERLAPPED associated with asynchronous I/O is passed by address, but there was some confusion about how this technique could "work" when kernel mode has no idea that you are playing this trick.

    Whether kernel mode is in on the trick is immaterial since it is not part of the trick.

    Let's start with a version of the code which does not take advantage of the OVERLAPPED structure address in the way described in the article. This is a technique I found in a book on advanced Windows programming:

    #define MAX_OVERLAPPED 10 // let's do 10 I/O's at a time
    
    // data to associate with each OVERLAPPED
    struct OTHERDATA { ... };
    
    OVERLAPPED MasterOverlapped[MAX_OVERLAPPED];
    OTHERDATA OtherData[MAX_OVERLAPPED];
    
    OTHERDATA* FindOtherDataFromOverlapped(OVERLAPPED *lpOverlapped)
    {
     ptrdiff_t index = lpOverlapped - MasterOverlapped;
     return &OtherData[index];
    }
    
    // I/O is issued via
    // ReadFileEx(hFile, lpBuffer, nNumberOfBytesToRead,
    //            &MasterOverlapped[i], CompletionRoutine);
    
    void CALLBACK CompletionRoutine(
        DWORD dwErrorCode,
        DWORD dwNumberOfBytesTransferred,
        LPOVERLAPPED lpOverlapped)
    {
     OTHERDATA *lpOtherData =
                           FindOtherDataFromOverlapped(lpOverlapped);
     ... do stuff with lpOverlapped and lpOtherData ...
    }
    

    This version of the code uses the address of the OVERLAPPED structure to determine the location in the MasterOverlapped table and uses the corresponding entry in the parallel array at OtherData to hold the other data.

    Let's make this code worse before we make it better:

    OTHERDATA* FindOtherDataFromOverlapped(OVERLAPPED *lpOverlapped)
    {
     for (int index = 0; index < MAX_OVERLAPPED; index++) {
      if (&MasterOverlapped[index] == lpOverlapped) {
       return &OtherData[index];
      }
     }
     FatalError(); // should never be reached
    }
    

    Instead of doing simple pointer arithmetic to recover the index, we walk the array testing the pointers. This is naturally worse than doing pointer arithmetic, but watch what this step allows us to do: First, we reorganize the data so that instead of two parallel arrays, we have a single array of a compound structure.

    struct OVERLAPPEDEX
    {
     OVERLAPPED Overlapped;
     OTHERDATA OtherData;
    };
    
    OVERLAPPEDEX Master[MAX_OVERLAPPED];
    
    OTHERDATA* FindOtherDataFromOverlapped(OVERLAPPED *lpOverlapped)
    {
     for (int index = 0; index < MAX_OVERLAPPED; index++) {
      if (&Master[index].Overlapped == lpOverlapped) {
       return &Master[index].OtherData;
      }
     }
     FatalError(); // should never be reached
    }
    
    // I/O is issued via
    // ReadFileEx(hFile, lpBuffer, nNumberOfBytesToRead,
    //            &Master[i].Overlapped, CompletionRoutine);
    

    All we did was consolidate the parallel arrays into a single array.

    Now that it's an array of compound structures, we don't need to carry two pointers around (one to the OVERLAPPED and one to the OTHERDATA). We can just use a single OVERLAPPEDEX pointer and dereference either the Overlapped or the OtherData part.

    OVERLAPPEDEX* FindOverlappedExFromOverlapped(
        OVERLAPPED *lpOverlapped)
    {
     for (int index = 0; index < MAX_OVERLAPPED; index++) {
      if (&Master[index].Overlapped == lpOverlapped) {
       return &Master[index];
      }
     }
     FatalError(); // should never be reached
    }
    
    void CALLBACK CompletionRoutine(
        DWORD dwErrorCode,
        DWORD dwNumberOfBytesTransferred,
        LPOVERLAPPED lpOverlapped)
    {
        OVELRAPPEDEX *lpOverlappedEx =
                        FindOverlappedExFromOverlapped(lpOverlapped);
        ... do stuff with lpOverlappedEx ...
    }
    

    Finally, we can optimize the FindOverlappedExFromOverlapped function that we de-optimized earlier. Observe that the de-optimized loop is an example of the "for/if" anti-pattern.

    The "for/if" anti-pattern goes like this:

    for (int i = 0; i < 100; i++) {
     if (i == 42) do_something(i);
    }
    

    This can naturally be simplified to

    do_something(42);
    

    Our FindOverlappedExFromOverlapped function is a special case of this anti-pattern. It becomes more evident if we do some rewriting. Start with

    &Master[index].Overlapped == lpOverlapped
    

    Apply CONTAINING_RECORD to both sides.

    CONTAINING_RECORD(&Master[index].Overlapped, OVERLAPPEDEX, Overlapped) ==
        CONTAINING_RECORD(lpOverlapped, OVERLAPPEDEX, Overlapped)
    

    The left-hand side of the comparison simplifies to

        &Master[index]
    

    resulting in

    &Master[index] ==
       CONTAINING_RECORD(lpOverlapped, OVERLAPPEDEX, Overlapped)
    

    Recall that a[b] is equivalent to *(a+b), and therefore &a[b] is equivalent to a+b.

    Master + index ==
       CONTAINING_RECORD(lpOverlapped, OVERLAPPEDEX, Overlapped)
    
    Now subtract Master from both sides:
    index == CONTAINING_RECORD(lpOverlapped, OVERLAPPEDEX, Overlapped) - Master
    

    We have transformed the test into a clear case of the for/if anti-pattern, and the function can be simplified to

    OVERLAPPEDEX* FindOverlappedExFromOverlapped(
        OVERLAPPED *lpOverlapped)
    {
     ptrdiff_t index =
       CONTAINING_RECORD(lpOverlapped, OVERLAPPEDEX, Overlapped) - Master;
     return &Master[index];
    }
    

    Again, rewrite &a[b] as a+b:

     return Master + index;
    

    Substitute the value of index computed on the previous line:

     return Master + 
       CONTAINING_RECORD(lpOverlapped, OVERLAPPEDEX, Overlapped) - Master;
    

    The two occurrences of Master cancel out, leaving

    OVERLAPPEDEX* FindOverlappedExFromOverlapped(
        OVERLAPPED *lpOverlapped)
    {
     return CONTAINING_RECORD(lpOverlapped, OVERLAPPEDEX, Overlapped);
    }
    

    And there you have it. By a series of purely mechanical transformations, we have rediscovered the technique of extending the OVERLAPPED structure.

  • The Old New Thing

    What happened to the return code from WinMain in 16-bit Windows?

    • 7 Comments

    Commenter S asks, "What happened to the return code from WinMain in a Windows 3.1 app?" After all, there was no GetExitCodeProcess function in 16-bit Windows.

    Basically, the exit code vanished into the ether.

    Unless you captured it.

    The Toolhelp library provided a low-level hook into various parts of the kernel, allowing you to monitor, among other things, the creation and destruction of tasks. That was how you captured the return code of a Windows program in 16-bit Windows.

    But if you didn't catch it as it happened, it was gone forever, lost in the ether.

  • The Old New Thing

    The OVERLAPPED associated with asynchronous I/O is passed by address, and you can take advantage of that

    • 26 Comments

    When you issue asynchronous I/O, the completion function or the I/O completion port receives, among other things, a pointer to the OVERLAPPED structure that the I/O was originally issued against. And that is your key to golden riches.

    If you need to associate information with the I/O operation, there's no obvious place to put it, so some people end up doing things like maintaining a master table which records all outstanding overlapped I/O as well as the additional information associated with that I/O. When each I/O completes, they look up the I/O in the master table to locate that additional information.

    But it's easier than that.

    Since the OVERLAPPED structure is passed by address, you can store your additional information alongside the OVERLAPPED structure:

    // in C
    struct OVERLAPPEDEX {
     OVERLAPPED o;
     CClient *AssociatedClient;
     CLIENTSTATE ClientState;
    };
    
    // or in C++
    struct OVERLAPPEDEX : OVERLAPPED {
     CClient *AssociatedClient;
     CLIENTSTATE ClientState;
    };
    

    When the I/O completes, you can use the CONTAINING_RECORD macro or just static_cast the LPOVERLAPPED to OVERLAPPEDEX* and bingo, there's your extra information right there. Of course, you have to know that the I/O that completed is one that was issued against an OVERLAPPEDEX structure instead of a plain OVERLAPPED structure, but there are ways of keeping track of that. If you're using a completion function, then only use an OVERLAPPEDEX-aware completion function when the OVERLAPPED structure is part of an OVERLAPPEDEX structure. If you're using an I/O completion port, then you can use the completion key or the OVERLAPPED.hEvent to distinguish OVERLAPPEDEX asynchronous I/O from boring OVERLAPPED I/O.

  • The Old New Thing

    Why does SHCOLUMNINFO have unusually tight packing?

    • 22 Comments

    Alternate title: News flash: Sometimes things happen by mistake

    rbirkby asks why the SHCOLUMNINFO structure has 1-byte packing. "Was the expectation that there would be so many columns in a details view that the saving would be worthwhile?"

    Hardly anything that clever or ingenious. It's just the consequence of a mistake.

    When the SHCOLUMNINFO structure was added to the header file in the Windows 2000 timeframe, it was added with no specific packing directive. But it turns out that there was a specific packing directive; it just wasn't obvious. Near the top of the shlobj.h header file was the following:

    #include <pshpack1.h>   /* Assume byte packing throughout */
    

    (There was of course a matching #include <poppack.h> at the bottom.) This set the default packing for the entire header file to byte packing instead of natural alignment.

    By the time this mistake was identified, it was too late. Windows 2000 had already shipped, byte packing and all. And once the code ships, it's done. You're stuck with it.

    Sorry.

  • The Old New Thing

    There is no interface for preventing your notification icon from being hidden

    • 38 Comments

    Yes, it's another installment of I bet somebody got a really nice bonus for that feature. A customer had this question for the Windows 7 team:

    Our program creates a notification icon, and we found that on Windows 7 it is hidden. It appears properly on all previous versions of Windows. What is the API to make our icon visible?

    First of all, I'd like to congratulate you on writing the most awesome program in the history of the universe.

    Unfortunately, Windows 7 was not prepared for your awesomeness, because there is no way to prevent your notification icon from being hidden. That's because if there were, then every other program (the ones that aren't as awesome as you) would use it, thereby causing your awesome icon to be lost among all the non-awesome ones.

    I'm sorry. You will just have to find some other medium for self-actualization.

  • The Old New Thing

    The subtleties of a Will Ferrell movie, and other observations from the in-flight entertainment on a Chinese airline

    • 20 Comments

    My flights to and from Beijing were on Hainan Airlines, a Chinese airline. One consequence of this is that Mandarin Chinese is the primary language of communication; English is a distant second. It also means that the in-flight movies are subtitled in Chinese, so if you can't read Chinese, you are restricted to movies in languages you understand.

    I wasn't interested in the English-language movies, although I did watch a little bit of "The Other Guys", a Will Ferrell vehicle. In one scene, Ferrell's character and his friend have dinner in a Chinese restaurant. Ferrell's character says to the waiter, "謝謝" which means "Thank you" in Mandarin. The waiter responds, "唔該" which means (in this context) "You're welcome" in Cantonese.¹

    Part of my brain wondered if this language mismatch was some sort of subtle commentary about the nature of Will Ferrell's character, that he's perhaps a bit of a poseur, or that he's out of place and doesn't realize it?

    And part of my brain couldn't believe that the other part of my brain used "subtle" and "Will Ferrell" in the same sentence.

    Anyway, the only other language I knew that was offered by the in-flight entertainment system was German. So I watched Willi und die Wunder dieser Welt, a movie-length version of the German children's television show Willi wills wissen. And watching the movie reminded me that Germans are obsessed with poop. During the course of the movie, you see a flying fox pooping, you see a polar bear pooping, and you investigate a Japanese toilet. I didn't stick around for the whole movie, but I wouldn't be surprised if you also saw a scorpion pooping in the final segment.

    (In the Canadian segment, somebody talks with Willi in heavily Canadian-accented German which was apparently learned phonetically. I could barely understand him. It reminded me of my high school German class and the students who couldn't shake their thick American accents.)

    Footnote

    ¹There are several phrases that roughly mean "Thank you" in Cantonese. The two primary ones are the aforementioned "唔該" and "多謝", and the rules governing proper use of each one are complicated.

  • The Old New Thing

    Microspeak: Informing a product

    • 11 Comments

    Microspeak is not always about changing a word from a verb to a noun or changing a noun to a verb or even changing a noun into a verb and then back into a noun. Sometimes it's about data compression.

    This testing won't inform RC, but we'll need it to inform an RTM release.

    First, you need to familiarize yourself with a less-used sense of the verb to inform, namely to guide or direct. A typical use would be something like "This data will inform the decision whether to continue with the original plan in Product Q." In other words, this data will be used to help decide whether to continue with the original plan.

    But of course, at Microsoft, it's all rush rush hurry hurry no time for slow sentences just get to the point. So we drop out a few words from the sentence and instead of informing a decision about something, we just inform the thing itself: "This data will inform Product Q."

    Therefore, the sentence at the start of this article is shorthand for "This testing won't inform [some sort of decision regarding] RC, but we'll need it to inform [that same decision in] an RTM release." In other words, the result of this testing won't have any effect on the release candidate, but it will be used to help make some sort of decision regarding the RTM release.

    Another citation, taken from an executive slide deck:

    • Other take-aways
      • What learnings can we get today to inform the release?

    I like how that brief snippet combines three pieces of Microspeak: take-away, learnings, and inform.

Page 119 of 429 (4,290 items) «117118119120121»