May, 2013

  • The Old New Thing

    Posted messages are processed ahead of input messages, even if they were posted later

    • 9 Comments

    Regardless of which interpretation you use, it remains the case that posted messages are processed ahead of input messages. Under the MSDN interpretation, posted messages and input messages all go into the message queue, but posted messages are pulled from the queue before input messages. Under the Raymond interpretation, posted messages and input messages are kept in separate queues, and the message retrieval functions will look first in the posted message queue before looking in the input queue.

    Let's run an experiment to see posted messages get processed ahead of input messages. Start with the new scratch program and make these changes:

    #include <strsafe.h>
    
    class RootWindow : public Window
    {
    public:
     virtual LPCTSTR ClassName() { return TEXT("Scratch"); }
     static RootWindow *Create();
    
     void AppendText(LPCTSTR psz)
     {
        ListBox_SetCurSel(m_hwndChild,
                          ListBox_AddString(m_hwndChild, psz));
     }
    
     void AppendFormat(LPCTSTR pszFormat, ...)
     {
      va_list ap;
      va_start(ap, pszFormat);
      TCHAR szMsg[256];
      StringCchVPrintf(szMsg, ARRAYSIZE(szMsg), pszFormat, ap);
      AppendText(szMsg);
      va_end(ap);
     }
    
     void LogMessage(const MSG *pmsg)
     {
       AppendFormat(TEXT("%d\t%04x\t%p\t%p"),
                    pmsg->time,
                    pmsg->message,
                    pmsg->wParam,
                    pmsg->lParam);
     }
    
     ...
    };
    
    LRESULT RootWindow::OnCreate()
    {
     m_hwndChild = CreateWindow(
          TEXT("listbox"), NULL,
          LBS_HASSTRINGS | LBS_USETABSTOPS |
          WS_CHILD | WS_VISIBLE | WS_TABSTOP | WS_VSCROLL,
          0, 0, 0,0, GetHWND(), (HMENU)1, g_hinst, 0);
     return 0;
    }
    
    int WINAPI WinMain(HINSTANCE hinst, HINSTANCE hinstPrev,
                       LPSTR lpCmdLine, int nShowCmd)
    {
       ...
       while (GetMessage(&msg, NULL, 0, 0)) {
    
        switch (msg.message) {
        case WM_KEYDOWN:
         prw->AppendText(TEXT("Sleeping"));
         UpdateWindow(prw->GetHWND());
         Sleep(1000);
         prw->AppendText(TEXT("Posting"));
         PostMessage(prw->GetHWND(), WM_USER, 0, 0);
         break;
        case WM_KEYUP:
        case WM_USER:
         prw->LogMessage(&msg);
         break;
        }
    
        TranslateMessage(&msg);
        DispatchMessage(&msg);
       ...
    }
    

    This program creates a list box so we can display some output. In the message loop, it sniffs at all the queued messages and does the following:

    • If the message is WM_KEY­UP or WM_USER, then it logs the message timestamp and some parameters.
    • If the message is WM_KEY­DOWN, then it sleeps without processing messages for one second, and then posts a WM_USER message to the main window (which ignores it).

    Run this program, and then tap the shift key.

    The window gets a WM_KEY­DOWN for the shift key. It sleeps for one second (plenty of time for you to release the shift key), and then posts a WM_USER message.

    The WM_USER and WM_KEY­UP messages arrive, and observe via the log window that they arrive out of order. WM_USER message arrived first!

    That's because of the rule that says that posted messages are processed ahead of input messages. (Depending on how you want to look at it, you might say that posted messages are "called out for preferential treatment" in the queue, or you might say that posted messages are placed in a different queue from input messages, and the posted message queue has higher priority.)

    Observe also that the timestamp on the WM_USER message is greater than the timestamp on the WM_KEY­UP message, because the key went up before the WM_USER message was posted. Time has gone backward.

    Make the following change to our program: Change the message we post from WM_USER to WM_KEY­UP:

          PostMessage(hwnd, WM_KEYUP, 0, 0);
    

    Run the program again, and again tap the shift key. Observe that the posted WM_KEY­UP message is processed ahead of the WM_KEY­UP input message. (You can see the difference because we posted the WM_KEY­UP message with wParam and lParam both zero, whereas the WM_KEY­UP input message has information in those parameters.)

    This little demonstration also reinforces some other things we already knew. For example, it once again shows that the input manager does not wiretap your posted messages. If you post a WM_KEY­UP message, it is treated like a posted message not an input message. We saw earlier that posting a keyboard message does not update internal input states. The keyboard shift states are not updated to match your prank call message. If somebody calls Get­Queue­Status, they will not be told that there is input waiting. It will not wake a Msg­Wait­For­Multiple­Objects function that is waiting for QS_INPUT. And as we saw here today, the message gets processed out of order.

  • The Old New Thing

    The posted message queue vs the input queue vs the message queue

    • 8 Comments

    There are multiple ways of viewing the interaction between posted messages and input messages. MSDN prefers to view posted messages and input messages as part of one giant pool of messages in a message queue, with rules about which ones get processed first. I, on the other hand, prefer to think of posted messages and input messages as residing in different queues that are processed in sequence.

    By analogy, consider a business with a policy that loyalty program members are served ahead of regular customers.

    One way of organizing this is to form a single queue, but sorting them so members go to the front. To call the next person in line, you just take whoever is at the head of the queue.

    AddToQueue(Customer c)
    {
     if (c is a member and the queue contains non-members) {
      let n = the first non-member in the queue;
      insert c in front of n;
     } else {
      insert c at the end of the queue;
     }
    }
    
    GetNextCustomer()
    {
     if (there is somebody in the queue) {
      return the first person in the queue;
     }
     // nobody is waiting
     return null;
    }
    

    This approach works fine from a programmatic standpoint, but people might not like it when they see others cutting in front of them. You might therefore choose to create two queues, one for members and one for non-members:

    AddToQueue(Customer c)
    {
     if (c is a member) {
      insert c at end of member queue;
     } else {
      insert c at end of non-member queue;
    }
    
    GetNextCustomerInLine()
    {
     if (there is somebody in the member queue) {
      return the first person in the member queue;
     }
     if (there is somebody in the non-member queue) {
      return the first person in the non-member queue;
     }
     // nobody is waiting
     return null;
    }
    

    Note that this second algorithm serves customers in exactly the same order; the only difference is psychological. Customers might resent the presence of a members line since it reminds them that other people are getting special treatment. Or this algorithm may not be practical because you don't have room in your lobby for two lines. You might choose yet another algorithm, where you select members at the time the customers are served rather than when they arrive. Everybody forms a single queue, but sometimes a customer in the middle of the queue gets pulled out for special treatment. (Maybe they wear red hats so they are easy to spot.)

    AddToQueue(Customer c)
    {
     insert c at end of queue;
    }
    
    GetNextCustomerInLine()
    {
     if (there is a member in the queue) {
      return the first member in the queue;
     }
     // else only non-members remaining
     if (there is somebody in the queue) {
      return the first person in the queue;
     }
     // nobody is waiting
     return null;
    }
    

    (Some banks use a variant of this algorithm to separate potential bank-robbers from the rest of the customers.)

    From the standpoint of determining who gets served in what order, all of these algorithms are equivalent. One may be more efficient, one may be easier to understand, one may be easier to maintain, one may be easier to debug, but they all accomplish the same thing.

    So go ahead and use whatever interpretation of the message queue you like. They are all equivalent.

    But the interpretation that I use is the one I presented several years ago at the PDC, where the posted message queue and input queues are considered separate queues. I prefer that interpretation because it makes input queue attachment easier to understand.

    In the analogy above, I prefer viewing things in terms of the second algorithm, with separate queues; MSDN prefers viewing things in terms of the third algorithm, with a single queue, but with some messages given preferential treatment over others.

    But as I've already noted, the outputs of the algorithms are the same given the same inputs, so it doesn't matter how they are implemented internally, since all you can observe as a program are the inputs and outputs. It's like the multiple interpretations of quantum mechanics: They all attempt to describe the world from a particular viewpoint, but at the end of the day, the world simply is, and a description is just a description.

    Next time, we'll explore some consequences of the interaction between posted messages and input messages.

  • The Old New Thing

    What's the point of SecureZeroMemory?

    • 37 Comments

    The Secure­Zero­Memory function zeroes out memory in a way that the compiler will not optimize out. But what's the point of doing that? Does it really make the application more secure? I mean, sure the data could go into the swap file or hibernation file, but you need to have Administrator access to access those files anyway, and you can't protect yourself against a rogue Administrator. And if the memory got swapped out before it got zeroed, then the values went into the swap file anyway. Others say that it's to prevent other applications from reading my process memory, but they could always have read the memory before I called Secure­Zero­Memory. So what's the point?

    The Secure­Zero­Memory function doesn't make things secure; it just makes them more secure. The issue is a matter of degree, not absolutes.

    if you had a rogue Administrator or another application that is probing your memory, then that rogue operator has to suck out the data during the window of opportunity between the time you generate the sensitive data and the time you zero it out. This is typically not a very long time, so it makes the attacker work harder to get the data. Similarly, the data has to be swapped out during the window between the sensitive data being generated and the data being zeroed. Whereas if you never called Secure­Zero­Memory, the attacker could take their sweet time looking for the sensitive information, because it'll just hang around until the memory gets re-used for something else.

    Furthermore, the disclosure may not be due to a rogue operative, but may be due to your own program! If your program crashes, and you're signed up your program for Windows Error Reporting, then a crash dump file is generated and uploaded to Microsoft so that you can download and investigate why your program is failing. In preparation for uploading, the crash dump is saved to a file on the user's hard drive, and an attacker may be able to mine that crash dump for sensitive information. Zeroing out memory which contained sensitive information reduces the likelihood that the information will end up captured in a crash dump.

    Another place your program may inadvertently reveal sensitive information is in the use of uninitialized buffers. If you have a bug where you do not fully-initialize your buffers, then sensitive information may end up leaking into them and then accidentally transmitted over the network or written to disk. Using the Secure­Zero­Memory function when finished with sensitive information is a defense-in-depth way of making it harder for sensitive information to go where it's not supposed to.

  • The Old New Thing

    Why don't hotkeys for items on my Start menu work when I am in a fullscreen application?

    • 38 Comments

    You can set a hotkey on the shortcuts in your Start menu, and Explorer will launch that shortcut when you press the hotkey, but not if you are in a fullscreen application, like when you set Paint into fullscreen mode, or when you are displaying a PowerPoint presentation. Why are shortcut hotkeys ignored when a fullscreen application is running?

    This feature was added by customer request.

    The issue is that you're playing a game like World of Warcraft or Unreal Tournament, and you're mashing your keys like crazy, and whoops you slip and hit Ctrl+Alt+C instead of Ctrl+Shift+C, and now you inadvertently launched Calc or something. Whack, you get taken out of your game into Calc, and by the time you get back into the game, your character has been mortally wounded, and you've lost the game.

    To prevent this from happening, Explorer temporarily ignores custom hotkeys for Start menu shortcuts when a fullscreen application is running.

    Note, however, that there is a bug in older versions of Windows: If you press a hotkey for a Start menu shortcut when there is a fullscreen application running, the hotkey is blocked. (So far so good.) But even if you clear the fullscreen application, the hotkey is still blocked because the hotkey got "stuck" in the blocked state. To unstick the hotkey, press a system hotkey involving the Windows key, then try the original hotkey again. (I use Win+B since it is relatively harmless.)

  • The Old New Thing

    What are the scoping rules for variables in nested courtyards?

    • 7 Comments

    I dreamed that I had to fix a bug caused by variable redeclaration inside a nested courtyard. Had to move the horses out of the way first.

    And then it got weird.

    I won't go into the orchestra rehearsal that turned out to be just a recording played on a loudspeaker in an empty concert hall. Because you wouldn't believe that part.

  • The Old New Thing

    How do I customize the console properties for a shortcut to a console application?

    • 9 Comments

    You already know how to create a shortcut:

    #include <windows.h>
    #include <tchar.h>
    #include <shlobj.h>
    #include <atlbase.h>
    
    // class CCoInitialize incorporated here by reference
    
    int __cdecl _tmain(int argc, TCHAR **argv)
    {
     // error checking elided for expository purposes
     CCoInitialize init;
     CComPtr<IShellLink> spsl;
     spsl.CoCreateInstance(CLSID_ShellLink);
     spsl->SetPath(TEXT("C:\\Windows\\system32\\cmd.exe"));
     CComQIPtr<IPersistFile>(spsl)->Save(L"Here.lnk", TRUE);
     return 0;
    }
    

    If you double-click the resulting shortcut from Explorer, it will run the command processor in a default console window.

    Today's Little Program customizes the other console properties, so you can control settings like the console buffer size and whether QuickEdit is enabled by default.

    We use the IShell­Data­List interface to attach "bonus data" to the shell link. The data we are interested in here is the NT_CONSOLE_PROPS. Remember, Little Programs perform little to no error checking, use hard-coded paths, and all that other stuff that make them unsuitable for shipping-quality code.

    int __cdecl _tmain(int argc, TCHAR **argv)
    {
     CCoInitialize init;
     CComPtr<IShellLink> spsl;
     spsl.CoCreateInstance(CLSID_ShellLink);
     spsl->SetPath(TEXT("C:\\Windows\\system32\\cmd.exe"));
    
     NT_CONSOLE_PROPS props;
     ZeroMemory(&props, sizeof(props));
     props.dbh.cbSize = sizeof(props);
     props.dbh.dwSignature = NT_CONSOLE_PROPS_SIG;
     props.wFillAttribute = FOREGROUND_BLUE | FOREGROUND_GREEN |
                            FOREGROUND_RED; // white on black
     props.wPopupFillAttribute = BACKGROUND_BLUE | BACKGROUND_GREEN |
                                 BACKGROUND_RED | BACKGROUND_INTENSITY |
                                 FOREGROUND_BLUE | FOREGROUND_RED;
                                 // purple on white
     props.dwWindowSize.X = 132; // 132 columns wide
     props.dwWindowSize.Y = 50; // 50 lines tall
     props.dwScreenBufferSize.X = 132; // 132 columns wide
     props.dwScreenBufferSize.Y = 1000; // large scrollback
     props.uCursorSize = 25; // small cursor
     props.bQuickEdit = TRUE; // turn QuickEdit on
     props.bAutoPosition = TRUE;
     props.uHistoryBufferSize = 25;
     props.uNumberOfHistoryBuffers = 4;
     props.ColorTable[ 0] = RGB(0x00, 0x00, 0x00);
     props.ColorTable[ 1] = RGB(0x00, 0x00, 0x80);
     props.ColorTable[ 2] = RGB(0x00, 0x80, 0x00);
     props.ColorTable[ 3] = RGB(0x00, 0x80, 0x80);
     props.ColorTable[ 4] = RGB(0x80, 0x00, 0x00);
     props.ColorTable[ 5] = RGB(0x80, 0x00, 0x80);
     props.ColorTable[ 6] = RGB(0x80, 0x80, 0x00);
     props.ColorTable[ 7] = RGB(0xC0, 0xC0, 0xC0);
     props.ColorTable[ 8] = RGB(0x80, 0x80, 0x80);
     props.ColorTable[ 9] = RGB(0x00, 0x00, 0xFF);
     props.ColorTable[10] = RGB(0x00, 0xFF, 0x00);
     props.ColorTable[11] = RGB(0x00, 0xFF, 0xFF);
     props.ColorTable[12] = RGB(0xFF, 0x00, 0x00);
     props.ColorTable[13] = RGB(0xFF, 0x00, 0xFF);
     props.ColorTable[14] = RGB(0xFF, 0xFF, 0x00);
     props.ColorTable[15] = RGB(0xFF, 0xFF, 0xFF);
     CComQIPtr<IShellLinkDataList>(spsl)->AddDataBlock(&props);
    
     CComQIPtr<IPersistFile>(spsl)->Save(L"Here.lnk", TRUE);
     return 0;
    }
    
  • The Old New Thing

    What happens if I manually post an auto-generated message into my message queue?

    • 18 Comments

    As we well know, the window manager generates various messages on-demand rather than posting them into the queue at the time the event occurs. But what happens if you manually post one of these messages, like Post­Message(hwnd, WM_PAINT, 0, 0)? Does that clear the internal flag that says "This window needs a paint message?"

    Nope.

    The window manager does not have a prank call detector. If you post a fake WM_PAINT message, then a fake WM_PAINT message shows up in the message queue. The part of the window manager which manages the "Does this window need to be repainted?" does not wiretap every telephone call to see if somebody is prank-calling a window with the WM_PAINT message, and then say, "You know what? I'm going to make that prank call a reality and make it act like a real WM_PAINT message."

    Imagine if the act of prank-calling somebody caused Prince Albert to stop by for a visit!

    In particular, the posted message goes into the posted message queue and will rise to the front of the queue as messages are retrieved. On the other hand, the auto-generated message will remain in its proto-message state until the queue is finally empty.

    It then follows as a consequence that if you post one of these auto-generated messages, like WM_TIMER message, you don't get any magical coalescing behavior because the coalescing happens as part of the auto-generation, not as part of the Post­Message. These prank-call messages also appear in the message queue in posted order rather than being generated at low priority like normal auto-generated messages.

  • The Old New Thing

    Even though mouse-move, paint, and timer messages are generated on demand, it's still possible for one to end up in your queue

    • 8 Comments

    We all know that the generated-on-demand messages like WM_MOUSE­MOVE, WM_PAINT, and WM_TIMER messages are not posted into the queue when the corresponding event occurs, but rather are generated by Get­Message or Peek­Message when they detect that they are about to conclude that there is no message to return and the generated-on-demand message can be returned. When this happens, the window manager creates the message on the fly, posts it into the queue, and hey, how about that, the Get­Message or Peek­Message function now has a message to return!

    Note that this auto-generate can happen even though the queue is not empty, because the message filters control what messages in the queue can be returned. For example, suppose the message queue contains the following messages:

    • { hwnd1, WM_CLIP­BOARD­UPDATE }
    • { hwnd2, WM_LBUTTON­DOWN }

    (Note that the above diagram is not strictly correct, because the WM_LBUTTON­DOWN message goes into the input queue, not the message queue, but the distinction is not important here.)

    Suppose you now call Get­Message(&msg, hwnd1, WM_MOUSE­FIRST, WM_MOUSE­LAST). None of the messages in the queue satisfy the message filter: The first message meets the window filter, but the message is not in range. The second message meets the message range filter, but does not meet the window filter. The Get­Message function is about to give up and say "I guess I need to wait for a message," but before it finally concedes defeat, it says, "Hang on there. I see a note that tells me that I should auto-generate a WM_MOUSE­MOVE message for window hwnd1. And that message satisfies the message filter. I'll generate it now!"

    The Get­Message function posts the { hwnd1, WM_MOUSE­MOVE } message into the queue (assigning it the current time as the timestamp), and then it says, "Hey, lookie here! A message that satisfies the filter!" It then removes the message from the queue and returns it.

    (Note that this algorithm is conceptual. It doesn't actually work this way internally. In particular, the window manager does not literally talk to itself, at least not out loud.)

    Okay, so in the Get­Message case, even if the message conceptually goes into the queue, it comes right back out immediately, so you never actually observe it there.

    Now repeat the exercise with the Peek­Message function. As before, the WM_MOUSE­MOVE message is posted into the queue with the current time as the timestamp. If the PM_REMOVE flag is passed, then the message is removed from the queue and returned, just like Get­Message. If the PM_NO­REMOVE flag is passed, then things get interesting: The message is returned but not removed from the queue.

    You now have a WM_MOUSE­MOVE message physically residing in the queue!

    This is the answer to the puzzle: If auto-generated messages are generated on demand, how is it possible for them to end up sitting in your message queue?

    I recall a bug investigation from nearly two decades ago which basically boiled down to this issue: Somebody PM_NO­REMOVE'd an auto-generated message and not only left it in the queue, but kept generating new ones without processing the old ones. Eventually, the message queue filled up.

    (Note that this is also the answer to the puzzle: If WM_MOUSE­MOVE is generated on demand, how can it be possible to retrieve a WM_MOUSE­MOVE message with a timestamp different from the current time?)

  • The Old New Thing

    How do I get a window back on the screen when it moved far, far away? Windows 7 (and 8) edition

    • 30 Comments

    Some time ago, I showed how to get a window back on the screen when it moved far, far away. That technique still works in Windows 7 and 8, but there's an easier shortcut that takes advantage of window arrangement features added in Windows 7.

    First, you switch to the application by whatever means. Then hit Win+UpArrow to maximize the window. That should put the window on-screen, albeit at the wrong size. Now you just grab the title bar of the window with the mouse and drag it off the top edge of the screen. Bingo, the window returns to its original position, and you can use the mouse to put it wherever you like.

    This trick doesn't work for windows that cannot be resized (such as Calculator), but for those windows, you can use the old version of the trick.

  • The Old New Thing

    A question about proper disposal of unwanted items with an unhelpful answer

    • 13 Comments

    On an internal mailing list about home maintenance and ownership, somebody asked:

    I have a handful of items that I need to get rid of and probably should not toss into the regular garbage. Any thoughts?

    • Older TV — 32″
    • Propane tanks (full)
    • Lighter fluid
    • Carpet glue adhesive
    • Lawn fertilizer

    The best reply was an unhelpful one.

    You've pretty much got all the components you need to build a bomb. Why dispose of them?

Page 1 of 3 (30 items) 123