September, 2011

  • The Old New Thing

    Microspeak: The bug farm


    In its most general sense, the term bug farm refers to something that is a rich source of bugs.

    It is typically applied to code which is nearly unmaintainable. Code can arrive in this state through a variety of means.

    • Poor initial design.
    • An initial design that has been pushed far beyond its original specification (resulting in features built on top of other features in weird ways).
    • Overwhelming compatibility constraints such that the tiniest perturbation is highly likely to cause some application somewhere to stop working.
    • Responsibility for the code residing in people whom we shall euphemistically describe as "failing to meet your personal standards of code quality."

    The term is most often used as a cautionary term, calling attention to areas where there is high risk that code you're about to write is going to result in a bug farm.

    Aren't we setting ourselves up for a bug farm?
    This could easily lead to a bug farm from different lifetimes for this various state objects.

    The term is quite popular at Microsoft (pre-emptive snarky comment: because Microsoft software is all one giant bug farm). Here are some citations just from

    Layout runs under disable processing. The reason we did that is because, well, reentrant layout is a bug farm.
    A lot of testers suddenly realized that case sensitivity is a veritable bug farm on a project that thinks it is ready to go, but has not yet tried it.
    That type of implicit vs. explicit inference also turned out to be a bug farm.
    Did you forget to handle an entire set of test cases? Is the features implementation overly complex and going to be a bug farm?
  • The Old New Thing

    The clipboard viewer linked list is no longer the responsibility of applications to maintain, unless they want to


    Commenter Nice Clipboard Manager (with drop->clipboard) wonders why Windows still uses a linked list to inform programs about clipboard modifications. If any clipboard viewer fails to maintain the chain, then some windows won't get informed of the change, and if a clipboard viewer creates a loop in the chain, an infinite loop results.

    Well, sure, that's what happens if you use the old clipboard viewer chain. So don't use it. The old clipboard viewer chain remains for backward compatibility, but it's hardly the best way to monitor the clipboard. (This is another example of people asking for a feature that already exists.)

    Instead of using the clipboard viewer chain, just add yourself as a clipboard format listener via AddClipboardFormatListener. Once you've done that, the system will post you a WM_CLIPBOARDUPDATE message when the contents of the clipboard have changed, and you can respond accordingly. When you're done, call RemoveClipboardFormatListener.

    By using the clipboard format listener model, you let Windows worry about keeping track of all the people who are monitoring the clipboard, as Clipboarder Gadget suggested. (Mind you, Windows doesn't go so far as making each clipboard viewer think that it's the only viewer in the chain, because there may be applications which break the chain on purpose. Changing the chain behavior will break compatibility with those applications.)

    Let's turn our scratch program into a clipboard format listener.

    SniffClipboardContents(HWND hwnd)
     SetWindowText(hwnd, IsClipboardFormatAvailable(CF_TEXT)
                 ? TEXT("Has text") : TEXT("No text"));
    OnCreate(HWND hwnd, LPCREATESTRUCT lpcs)
     SniffClipboardContents(hwnd); // set initial title
     return AddClipboardFormatListener(hwnd);
    OnDestroy(HWND hwnd)
    ... add to window procedure ...
     case WM_CLIPBOARDUPDATE: SniffClipboardContents(hwnd); break;

    And that's it. Much, much simpler than writing a clipboard viewer, and much more robust since you aren't dependent on other applications not screwing up.

    There's another alternative to registering a clipboard listener and that's using the clipboard sequence number. The window manager increments the clipboard sequence number each time the contents of the clipboard change. You can compare the sequence number from two points in time to determine whether the contents of the clipboard have changed while you weren't looking.

    Now you have a choice. Do you use the notification method (clipboard format listener) or the polling method (clipboard sequence number)? The notification method is recommended if you want to do something as soon as the clipboard contents change. On the other hand, the polling method is more suitable if you perform calculations based on the clipboard contents and cache the results, and then later you want to verify that your cached results are still valid.

    For example, suppose you have a program with a Paste function, and pasting from the clipboard involves creating a complex data structure based on the clipboard contents. The user clicks Paste, you create your complex data structure, and insert it into the document. Your research discovers that a common operation is pasting the same contents several times. To optimize this, you want to cache the complex data structure so that if the user clicks Paste five times in a row, you only have to build the complex data structure the first time and you can just re-use it the other four times.

    void DocumentWindow::OnPaste()
     if (m_CachedClipboardData == NULL ||
         GetClipboardSequenceNumber() != m_SequenceNumberInCache) {
      delete m_CachedClipboardData;
      m_SequenceNumberInCache = GetClipboardSequenceNumber();
      m_CachedClipboardData = CreateComplexDataFromClipboard();
     if (m_CachedClipboardData) Paste(m_CachedClipboardData);

    When the OnPaste method is called, we see if we have clipboard data cached from last time. If not, then clearly we need to create our complex data structure from the clipboard. If we do have clipboard data in our cache, we see if the clipboard sequence number has changed. If so, then the cached data is no longer valid and we have to throw it away and create it from scratch. But if we have cached data and the sequence number hasn't changed, then the cache is still valid and we can avoid calling CreateComplexDataFromClipboard.

    The old clipboard viewer is like DDE: please feel free to stop using it.

  • The Old New Thing

    Why can't I PostMessage the WM_COPYDATA message, but I can SendMessageTimeout it with a tiny timeout?


    After receiving the explanation of what happens to a sent message when Send­Message­Timeout reaches its timeout, a customer found that the explanation raised another question: If the window manager waits until the receiving thread finishes processing the message, then why can't you post a WM_COPY­DATA message? "After all, Send­Message­Timeout with a very short timeout isn't all that different from Post­Message."

    Actually, Send­Message­Timeout with a very short timeout is completely different from Post­Message.

    Let's set aside the one crucial difference that, unlike messages posted by Post­Message, which cannot be recalled, the Send­Message­Timeout function will cancel the message entirely if the receiving thread does not process messages quickly enough.

    Recall that messages posted to a queue via Post­Message are retrieved by the Get­Message function and placed in a MSG structure. Once that's done, the window manager disavows any knowledge of the message. It did its job: It placed the message in the message queue and produced it when the thread requested the next message in the queue. What the program does with the message is completely up in the air. There's no metaphysical requirement that the message be dispatched to its intended recipient. (In fact, you already know of a common case where messages are "stolen" from their intended recipients: Dialog boxes.)

    In principle, the message pump could do anything it wants to the message. Dispatch it immediately, steal the message, throw the message away, eat the message and post a different message, even save the message in its pocket for a rainy day.

    By contrast, there's nothing you can do to redirect inbound non-queued messages. They always go directly to the window procedure.

    The important difference from the standpoint of messages like WM_COPY­DATA is that with sent messages, the window manager knows when message processing is complete: When the window procedure returns. At that time, it can free the temporary buffers used to marshal the message from the sender to the recipient. If the message were posted, the window manager would never be sure.

    Suppose the message is placed in a MSG structure as the result of a call to GetMessage. Now the window manager knows that the receiving thread has the potential for taking action on the message and the buffers need to be valid. But how would it know when the buffers can be freed? "Well you can wait until the exact same parameters get passed in a MSG structure to the Dispatch­Message function." But what if the message loop discards the message? Or what if it decides to dispatch it twice? Or what if it decides to smuggle it inside another message?

    Posted messages have no guarantee of delivery nor do they provide any information as to when the message has been definitely processed, or even if it has been processed at all. If the window manager let you post a WM_COPY­DATA message, it would have to use its psychic powers to know when the memory can be freed.

  • The Old New Thing

    Some preliminary notes from //build/ 2011


    Hey everybody, I'm down at the //build/ conference. (The extra slash is to keep the d from falling over.) I'm not speaking this year, but you can find me in the Apps area of the Expo room today until 3:30pm (except lunchtime), and Friday morning before lunch. I'll also be at Ask the Experts tonight.

    There are so many great sessions to choose from. The one I would attend if I weren't working that time slot would be Bring apps to life with Metro style animations in HTML5. Instead, I'll probably go to Building high performance Metro style apps using HTML5. Fortunately, the sessions are being recorded, so I can catch up later.

    (At PDC 2008, I learned of a class of conference attendee known as the overflow vulture. These people decide which sessions to attend by looking for the ones that are close to filling up, on the theory that "500 people can't be wrong." These people often fail to take into account the room size. A talk in a 200-person room which fills up is not necessarily more popular than a talk in a 500-person room which doesn't.)

    Here are my observations so far:

    • At the airport, I heard a page for "Katy Perry". Normally, my reaction would be, "Oh, that poor woman has the same name as the singer." But since I'm in Los Angeles, I have to give consideration to the possibility that it really is the singer.
    • On the ride from the airport to the hotel, I observed part of a police car chase, or at least two police cars rushing through traffic with lights on. Welcome to Los Angeles.
    • I decided to walk from my hotel to the convention center rather than taking the shuttle bus. Along the way, I spotted a bus coming down the street. The driver parked the bus in the right-hand lane (a lane which is normally used for driving), got off, and walked into the Carl's Jr. I took a peek inside, and he was at the counter ordering breakfast. I guess he figured the bus wouldn't fit in the drive-through. Welcome to Los Angeles.
    • I thought it would have been funny if Michael Anguilo had said, "And we're making these devices available to attendees for just $500. [beat] Just kidding. You're each getting one for free." Or pulled an Oprah. "Everybody, look under your chair! Ha-ha, made you look!"
    • You spend a good amount of time listening to the music that plays before the keynote begins. Imagine having that as your job. "I write music for conferences. My music is peppy, but not too much; hopeful, but with a little bit of attitude. And not so good you want to dance to it. And I have to write a dozen different versions, each one exactly fifteen seconds longer than the previous one. Oh, and it needs to segue into a higher-energy version when the speaker arrives on stage."
    • The City National Grove at Anaheim is not a city, not national, and not a grove. I do concede, however that it is in Anaheim.
    • If you look closely at the //build/ logo, you'll also notice that the second slash has partially decapitated the b. I tried reproducing the effect here, but my CSS-fu isn't powerful enough.
    • Bonus: The hotel I'm staying at is hosting a conference on hotel conference security. I wonder who provides security for that conference.
  • The Old New Thing

    What happens to a sent message when SendMessageTimeout reaches its timeout?


    The Send­Message­Timeout function tries to send a message, but gives up if the timeout elapses. What exactly happens when the timeout elapses?

    It depends.

    The first case is if the receiving thread never received the message at all. (I.e., if during the period the sender is waiting, the receiving thread never called GetMessage, PeekMessage, or a similar message-retrieval function which dispatches inbound sent messages.) In that case, if the timeout is reached, then the entire operation is canceled; the window manager cleans up everything and makes it look as if the call to SendMessageTimeout never took place. The message is removed from the list of the thread's non-queued messages, and when it finally gets around to calling GetMessage (or whatever), the message will not be delivered.

    The second case is if the receiving thread received the message, and the message was delivered to the destination window procedure, but the receiving thread is just slow to process the message and either return from its window procedure or call Reply­Message. In that case, if the timeout is reached, then the sender is released from waiting, but the message is allowed to proceed to completion.

    Since people seem to like tables, here's a timeline showing the two cases.

    Sending thread Case 1 Case 2
    SendMessageTimeout(WM_X) called ... not responding ... ... not responding ...
    ... not responding ... ... not responding ...
    ... not responding ... GetMessage() called
    ... not responding ... WndProc(WM_X) called
    ... not responding ... WndProc(WM_X) still executing
    timeout elapses ... not responding ... WndProc(WM_X) still executing
    SendMessageTimeout(WM_X) returns ... not responding ... WndProc(WM_X) still executing
    ... not responding ... WndProc(WM_X) returns
    GetMessage() called
    (message WM_X not received)

    Notice that in case 2, the window manager has little choice but to let the window procedure continue with the message. After all, time travel has yet to be perfected, so the window manager can't go back in time and tell the younger version of itself, (Possibly with a slow-motion "Nooooooooooooo" for dramatic effect.) "No, don't give him the message; he won't finish processing it in time!"

    If you are in case 2 and the message WM_X is a system-defined message that is subject to marshaling, then the data is not unmarshaled until the window procedure returns. It would be bad to free the memory out from under a window procedure. On the other hand, if the message is a custom message, then you are still on the hook for keeping the values valid until the window procedure is done.

    But wait, how do I know when the window procedure is done? The Send­Message­Timeout function doesn't tell me! Yup, that's right. If you need to do cleanup after message processing is complete, you should use the Send­Message­Callback function, which calls you back when the receiving thread completes message processing. When the callback fires, that's when you do your cleanup.

  • The Old New Thing

    A common control for associating extensions is well overdue


    Mark complained that a common control for associating extensions is *well* overdue.

    This is a recurring theme I see in the comments: People complaining that Windows lacks some critical feature that it in fact already has. (In the case, Windows had the feature for over two years at the time the question was asked. Maybe the SDK needs a ribbon? j/k)

    Windows Vista added the Default Programs UI as a control panel program, and it also has a programmable interface. You can use IApplication­Association­Registration to query and set default associations, and you can use IApplication­Association­Registration­UI to invoke the control panel itself on a set of associations associated with your program.

  • The Old New Thing

    Why are the building numbers on Microsoft main campus so erratic?


    Carrie complains that the building numbers on Microsoft main campus are completely random. Why is building 22 near buildings 40 and 41, far, far away from building 24?

    Because the Microsoft campus evolved.

    Many many years ago, the space on which the central Microsoft campus resides was a mill. Eventually it became an office park, and when Microsoft decided to move its headquarters there, it carved out a little wooded area and constructed four buildings, logically numbered 1 through 4.

    Later, the campus expanded, and plans were drawn up for three more buildings, logically numbered (and placed) 5 through 7. Two of those buildings were constructed, but the third was not built for reasons shrouded in mystery. When the campus expanded a third time, the new buildings were numbered 8 through 11. Presumably, at this point, there were still plans to construct Building 7 someday, so the number remained assigned to the planned-but-not-yet-built building. (Even if the Building 7 plans had been abandoned, the number had already been used in the plans submitted to the City of Redmond, and revising them would have entailed additional paperwork for no real benefit aside from satisfying some anal-retentive compulsion to ensure that every number was used. People who worry about this probably are also waiting for DirectX 4.)

    The campus grew, and each time new buildings were added, they received the next available number. The result of this was that buildings with consecutive numbers could very well end up far apart on campus.

    When the Microsoft main campus expanded across highway 520, the people in charge of assigning numbers decided to assign numbers starting at 100 for buildings on the other side of the highway. Mind you, they didn't stick to that plan rigidly, as there are some buildings numbered in the high 90's on that part of the campus.

    Once the idea of assigning non-consecutive numbers was breached, the number-assigning people went to town. There is a cluster of buildings in the 40's, another in the 50's (with Building 50 being an outlier), and another in the 80's.

    So at least the numbers for newer buildings are a bit less crazy. But if you're looking for an older building, you're going to have a rough time of it.

    Maybe if the original building-numbering people had had the foresight to name the buildings after their GPS coordinates.

    Bonus chatter: In 2009, the building-numbering people tried to rename Buildings 116 through 119 to Studios E through H, presumably because they were across the street from Studios A through D. This "Rebranding Project" was largely mocked. (And of course, just to make things confusing, the new names appear to have been assigned randomly.)

    Bonus chatter 2: The original Building 100 was demolished to make way for The Commons. The soon-to-be-displaced residents of Building 100 had a "demolition party" on their last day in the building, wherein they went around spraying graffiti, smashing walls with sledgehammers, that sort of thing.

  • The Old New Thing

    Is this a really bug with CreateWindowEx or am I just confused?


    Somebody sent me email pointing out strange behavior in the MessageBox function if you fail a window creation by returning −1 from the WM_CREATE message. On the other hand, returning FALSE from WM_NCCREATE seems to work just fine. "So why the difference with WM_CREATE?"

    #include <windows.h>
    WndProc(HWND hWnd, UINT uMsg, WPARAM wParam, LPARAM lParam)
            case WM_CREATE:
                return -1;
            case WM_DESTROY:
                return 0;
        return DefWindowProc(hWnd, uMsg, wParam, lParam);
        LPSTR lpCmdLine, int nShowCmd)
        MSG msg;
        HWND hWnd;
        WNDCLASS wc = { 0 };
        wc.lpfnWndProc   = WndProc;
        wc.hInstance     = hInst;
        wc.hIcon         = LoadIcon(NULL, IDI_APPLICATION);
        wc.hCursor       = LoadCursor(NULL, IDC_ARROW);
        wc.hbrBackground = (HBRUSH)(COLOR_WINDOW + 1);
        wc.lpszClassName = "TestApp";
            MessageBox(NULL, "Error creating class",
                "Test App", MB_ICONERROR);
            return 1;
        hWnd = CreateWindow(wc.lpszClassName, "Test App",
            MessageBox(NULL, "Error creating window",
                "Test App", MB_ICONERROR);
            return 1;
        ShowWindow(hWnd, nShowCmd);
        while(GetMessage(&msg, NULL, 0, 0))
        return (int)msg.wParam;

    You already know enough to solve this puzzle. You just need to connect the dots.

    (In fact, the person who sent me this topic did so a year after I already answered it. But I'm repeating it here because the original answer was accidentally destroyed.)

  • The Old New Thing

    Throwing garbage on the sidewalk: The sad history of the rundll32 program


    During the development of Windows Vista, the application comaptibility team traced a bunch of issues back to people corrupting the stack by using the rundll32 program to call functions that were not designed to be called by rundll32.

    The problems were often subtle. For example, a batch file which used rundll32 incorrectly ended up hanging because the rundll32 process never returned. The misaligned stack resulted in registers being restored from the stack incorrectly, and then the cleanup code inside rundll32 ends up getting confused and wedging itself. The programs got away with it on previous versions of Windows by sheer luck. The version of the compiler used by Windows Vista contains different optimizations, and it ended up arranging stack variables and using registers differently, and what in previous versions of Windows was some corruption that went largely unnoticed became corruption that resulted in the program getting stuck in an infinite loop. Lucky no longer.

    I was asked to come up with a solution for this problem, to fix the rundll32 program so it was more resilient to people who used it incorrectly. To fix other people's bugs for them.

    The solution: Before calling the function, push a hundred bytes of garbage onto the stack (in case the called function pops too many bytes off the stack) and save the stack pointer in a global variable. After the function returns, restore the stack pointer, in case the called function pops too many or too few bytes off the stack. I think I may even have saved the processor registers in global variables, I forget.

    Do not consider this free license to continue abusing the rundll32 program. When the pet store opens on Sundays, that doesn't mean that it's okay to keep throwing garbage on the sidewalk.

  • The Old New Thing

    Why waste your money on the car when it's the sound system you care about?


    There is apparently a subculture of people who decide to economize on the car part of the "loud stereo in car" formula (since they really don't care about the car—it's all about the music) and put their loud stereo on the back of a bicycle instead.

    This quotation from the article caught my attention:

    "People say, 'It's the next best thing to having a system in a car.' But it's better because you don't even have to roll down the windows."

    I had been unsure what to think about people who drive down the street with their stereos blaring. Are they audiophiles who prefer their music loud? Or are they jerks who like to annoy other people with loud music? That quotation sort of settles it.

Page 2 of 3 (26 items) 123