April, 2011

  • The Old New Thing

    Not quite understanding why you wash your hands before playing the piano

    • 15 Comments

    My niece wanted to play my piano, and I asked her to wash her hands. She said, "I don't need that, I have Magic Soap," and she produced a bottle of hand sanitizer.

    Um, the purpose of washing your hands isn't so the piano doesn't get sick.

    My piano-instructor cousin-in-law tells me that her young students often say, "My hands are not dirty. They are just a little sticky because I just ate a chocolate muffin on my way here. I don't think I need to wash my hands. Why do you always ask me to wash my hands before the piano lesson?"

  • The Old New Thing

    Why is there a RestoreLastError function that does the same thing as SetLastError?

    • 18 Comments

    Matt Pietrek noticed that Set­Last­Error and Restore­Last­Error do exactly the same thing and wondered why there's a separate function for it.

    It's to assist in debugging and diagnostics.

    Say you're debugging a problem and when you call Get­Last­Error you get ERROR_ACCESS_DENIED. It would really help a lot if you could figure out who set the error code to ERROR_ACCESS_DENIED. If you set a breakpoint on Set­Last­Error, you find that people call Set­Last­Error for two different reasons:

    1. To report an error.
    2. To restore the error code to what it was before they did something that might change the last error code.

    That second one needs a little explanation. You might have a logging function that goes like this:

    // Remember, code in italics is wrong
    void LogSomething(blah blah)
    {
     DWORD dwError = GetLastError();
     ... do logging stuff ...
     SetLastError(dwError);
    }
    
    // or if you prefer RAII
    
    class PreserveLastError
    {
    public:
        PreserveLastError() : m_dwLastError(GetLastError()) {} 
        ~PreserveLastError() { SetLastError(m_dwLastError); }
    private:
        DWORD m_dwLastError;
    };
    
    void LogSomething(blah blah)
    {
     PreserveLastError preserve;
    
     ... do logging stuff ...
    }
    

    It's important that functions which perform logging, assertion checking, and other diagnostic operations are nonintrusive. You don't want a bug to go away when you turn on logging because the logging code somehow perturbed the system. Therefore, your logging function saves the value of Get­Last­Error() and sets that back as the error code when it's done, so that any errors that took place during logging do not escape and inadvertently affect the rest of the program.

    Now let's go back to the code that's trying to figure out who set the error code to ERROR_ACCESS_DENIED. You set up your debugging diagnostic tool and tell it to record everybody who calls Set­Last­Error() and pay particular attention to everybody who sets the error to ERROR_ACCESS_DENIED. You then run your scenario, your program encounters the failure you're trying to debug, and you ask the diagnostic tool, "Tell me who set the error code to ERROR_ACCESS_DENIED." The diagnostic tool says, "Ah, I have that in my history. The function that set the error code to ERROR_ACCESS_DENIED is... Log­Something!"

    Of course, Log­Something wasn't really the originator of the ERROR_ACCESS_DENIED; it was just restoring things to how it found them. The real ERROR_ACCESS_DENIED came from somebody else, and the log function was just being careful not to disturb it.

    ...
      if (!FunctionX()) {
        LogSomething("Function X failed");
      } else {
        LogSomething("Function X succeeded");
       FunctionY(); // also does some logging
      }
      FunctionZ(); // also does some logging
      Assert(EverythingOkay()); // assertion fires
      // GetLastError() returns ERROR_ACCESS_DENIED
    ...
    

    All those calls to logging functions in between called Get­Last­Error() and got ERROR_ACCESS_DENIED back, then when the logging was complete, they called Set­Last­Error(ERROR_ACCESS_DENIED) to put things back. Your diagnostic error-tracing tool gleefully points the finger at your logging function: "Look! Look! This guy set the error code to ERROR_ACCESS_DENIED!"

    Enter Restore­Last­Error. This function does the same thing as Set­Last­Error, but its use is a message to diagnostic tools that "Sure, you may see me set an error code, but it wasn't my idea. I'm just trying to put things back the way I found them. Keep looking backwards in your history."

    (The message also works forward in time: If you want to catch ERROR_ACCESS_DENIED in the act, you might set a breakpoint on Set­Last­Error, and then get frustrated that the breakpoint keeps getting hit by your logging function. Switching the logging function to Restore­Last­Error keeps the breakpoint on Set­Last­Error from firing spuriously.)

    The corrected version of the Log­Something function is therefore something like this:

    void LogSomething(blah blah)
    {
     DWORD dwError = GetLastError();
     ... do logging stuff ...
     RestoreLastError(dwError);
    }
    
    // or if you prefer RAII
    
    class PreserveLastError
    {
    public:
        PreserveLastError() : m_dwLastError(GetLastError()) {} 
        ~PreserveLastError() { RestoreLastError(m_dwLastError); }
    private:
        DWORD m_dwLastError;
    };
    
  • The Old New Thing

    Microspeak: Hipo

    • 23 Comments

    A friend of mind was asked out of the blue, "What does hypo mean?"

    She started to flash back to high school English class and Greek word roots.

    "I've started to hear it everywhere. Like Everyone in that meeting is a hypo or We need to reach out to hypos."

    My friend realized that she had mis-heard the question. It was not about the Greek root hypo but rather the bizarro Microspeak word hipo, shorthand for high-potential employee.

    As I researched this term (which I had never encountered before), I found that it fell into that special category of Microspeak known as if you have to ask, I'm not going to tell you. Identifying and developing high-potential employees is one of the charter activities of the ExPo project. And if you look through the ExPo Web site, you'll find that nowhere do they tell you what HiPo and ExPo stand for. "If you have to ask, I'm not going to tell you."

    My friend suggested that "ExPos are people who have a lot of potential and enjoy showing it off to others."

  • The Old New Thing

    How do I pin a program directly to the Start menu rather than a shortcut?

    • 34 Comments

    Anonymous bemoans the fact that pinning programs to the Start menu (in Windows XP) actually pins the shortcut rather than the program itself. This means that if you right-click on a shortcut to pin it, then delete the shortcut, the pinned item stops working. How do you pin the program directly?

    You pin the program directly by right-clicking on the program directly. If you right-click on a shortcut and select Pin to Start menu, then you're pinning the shortcut. This is sort of obvious, because after all, that's what you right-clicked on. If you want to pin the raw program, then dive into the Program Files folder and right-click the EXE and select Pin to Start menu. Now you pinned the program executable rather than a shortcut.

    The Windows XP Start menu behaves this way instead of tunnelling through the shortcut to the executable because the shortcut itself contains valuable information such as name, command line parameters, hotkey, working directory, and icon. If you have a shortcut called "Development command prompt" that has custom console colors and runs CMD.EXE with a working directory set to your development tree and with a special startup batch file (via the /K command line switch) to set up your environment variables, and you right-click on that shortcut and selected Pin to Start menu, then you would probably be upset if the thing that actually got pinned to your Start menu was a boring CMD.EXE command prompt with no options and no customization.

    "It would be nice if Windows were a bit more intuitive about this."

    Anonymous didn't provide any specific suggestion as to what would be more intuitive. Pinning the executable and throwing away the shortcut properties is definitely not intuitive.

    The Windows 7 folks thought about this and came up with something that hopefully meets the a bit more intuitive criterion: When you pin a shortcut to the taskbar or the Start menu, they make a copy of the shortcut and pin the copy.

  • The Old New Thing

    No, you can't ask whether your notification icon is visible or not

    • 43 Comments

    A customer asked what started out as a simple question, but by now you know that this sort of simple question only raises new questions:

    Is there a way that we can detect that the user has hidden our notification icon?

    No, there is no way to ask whether your notification icon is hidden or visible. Users decide which icons they want to see, and you don't get to snoop in on the users' decisions.

    Questions like this scare me, because it sounds like this customer's program is going to respond to the user's choice to hide their icon by doing something even more annoying. This is another demonstration of Le Chatelier's Principle as interpreted by John Gall: Every system resists its proper functioning. In this case, a design to make programs less annoying and intrusive merely drives those programs to even more annoying and intrusive behavior.

    The customer's response confirmed this suspicion.

    We have a long-running process with multiple stages. It starts with a dialog box and a progress bar, and after the dialog finishes, a long operation takes place in the background. In some cases during this background operation, we need to ask the user to take an action or make a decision. Our previous version of the program displayed a dialog box under these circumstances, but we switched to a notification balloon to make the experience less annoying. But if the user has hidden our notification, then we want to go back to showing the dialog. (Ideally, we would make the long background operation part of the initial dialog box, but we do not control the initial dialog box.)

    If you use a notification icon, then you have to accept that the user can hide notification icons. If you still want a notification icon, you can start by displaying a dialog box with a progress bar, and give it a button that says "Hide this dialog and display a notification when attention is required." With this design, converting the dialog box to a notification icon is an active step the user must take, and if the user hides the notification icon, it's their own fault.

    Or abandon the notification icon entirely. According to the notification area design guidance, you shouldn't use notification icons if immediate action is required.

    Is the status critical? Is immediate action required? If so, display the information in a way that demands attention and cannot be easily ignored, such as a dialog box.

    Windows 7 lets you communicate the progress of the background operation unobtrusively by displaying the progress bar as part of the taskbar icon.

  • The Old New Thing

    There's only so much you can do to stop running code from simulating UI actions

    • 33 Comments

    Commenter KiwiBlue asks whether Captcha-style tests were considered to prevent unsigned drivers from programmatically clicking the 'Install anyway' button.

    I'm sure somebody considered it, but Captcha has its own problems.

    "Type the (distorted) letters below"-type Captcha cannot be used by people with visual impairments, people who are dyslexic, or people who simply are not familiar with the Latin alphabet. (Believe it or not, the vast majority of people on the planet have a native language which does not use the Latin alphabet.) Using an audio captcha runs into the problem of different accents, letters whose readings vary (zee/zed anyone?), and computers without a sound card (like most servers).

    And yes, there are other types of Captchas (dog/cat, for example), but the strongest argument against Captcha is probably that it's just adding more locks to the front door while leaving the service entrance wide open. Once you make it computationally infeasible to programmatically solve the Captcha, unscrupulous driver vendors would simply inject a DLL into the "Install this unsigned driver?" process and patch the call to Did­User­Answer­Captcha­Correctly so it always returns TRUE.

    Or even easier, just programmatically set the Driver Signing Options to Install the software anyway.

    If somebody is running code with administrative privileges, then they already own your machine. Any roadblocks you put up they can find a way to drive over. The goal is not so much putting up stronger and stronger roadblocks (because eventually people will simply drive around them) but rather making it clear to the developer that what they're doing is driving around a roadblock.

  • The Old New Thing

    Even if you have a lock, you can borrow some lock-free techniques

    • 8 Comments

    Even if you prefer to use a lock (after all, they are much easier to program), you can borrow some lock-free techniques. For example, consider this:

    CRITICAL_SECTION g_cs;
    GORILLADATA g_data;
    
    void PokeGorilla(double intensity)
    {
      EnterCriticalSection(&g_cs);
      DeformGorilla(intensity, &g_data);
      Reticulate(&g_data.spline);
      int stress = CalculateTension(&g_data.spline);
      if (stress < 25)      g_data.mood = RELAXED;
      else if (stress < 50) g_data.mood = ANNOYED;
      else                  g_data.mood = ANGRY;
      DeleteObject(g_data.hbmGorilla);
      g_data.hbmGorilla = RenderGorilla(&g_data);
      LeaveCriticalSection(&g_cs);
    }
    

    There are some concerns here. First of all, there's the lock hierarchy issue: If reticulating a spline takes the geometry lock, that may violate our lock hierarchy.

    If the lock g_cs is a hot lock, you may be concerned that all this gorilla stuff will hold the lock for too long. Maybe rendering a gorilla is a slow and complicated operation because it's hard to get the fur just right.

    These issues become less onerous if you switch to a lock-free algorithm, but that's an awful lot of work, and it's hard to get right. But maybe you can do just 20% of the work to get 80% of the benefit.

    void PokeGorilla(double intensity)
    {
      // Capture
      EnterCriticalSection(&g_cs);
      GORILLADATA data = g_data; // typo fixed
      LeaveCriticalSection(&g_cs);
    
      // Recalculate based on captured data
      DeformGorilla(intensity, &data);
      Reticulate(&data.spline);
      int stress = CalculateTension(&data.spline);
      if (stress < 25)      data.mood = RELAXED;
      else if (stress < 50) data.mood = ANNOYED;
      else                  data.mood = ANGRY;
      data.hbmGorilla = RenderGorilla(&data);
    
      // Commit
      EnterCriticalSection(&g_cs);
      HBITMAP hbmToDelete = g_data.hbmGorilla;
      g_data = data;
      LeaveCriticalSection(&g_cs);
      DeleteObject(hbmToDelete);
    }
    

    Here, we use the capture/try/commit model. We capture the state of the gorilla into a local variable, then perform our update based on that captured state. The spline reticulation takes place without any locks held, which avoids introducing a lock hierarchy violation. And rendering the gorilla is done without any locks held, which avoids introducing a choke point on the lock. After the calculations are done, we then re-enter the lock and commit the changes.

    This pattern uses a last-writer-wins model. If another thread pokes the gorilla while we are still calculating the previous gorilla state, we will overwrite that gorilla state when we complete. For some scenarios, that's acceptable. But maybe the gorilla's emotional state needs to be an accumulation of all the times he's ben poked. We want to detect that somebody has poked the gorilla while we were busy calculating so that we can incorporate that new information into the final result.

    To do that, we introduce a change counter.

    LONG g_lCounter;
    
    void PokeGorilla(double intensity)
    {
      BOOL fSuccess;
    
      do {
        // Capture
        EnterCriticalSection(&g_cs);
        GORILLADATA data = g_data; // typo fixed
        LONG lCounter = g_lCounter;
        LeaveCriticalSection(&g_cs);
    
        // Recalculate based on captured data
        DeformGorilla(intensity, &data);
        Reticulate(&data.spline);
        int stress = CalculateTension(&data.spline);
        if (stress < 25)      data.mood = RELAXED;
        else if (stress < 50) data.mood = ANNOYED;
        else                  data.mood = ANGRY;
        data.hbmGorilla = RenderGorilla(&data);
    
        // Commit
        EnterCriticalSection(&g_cs);
        HBITMAP hbmToDelete;
        if (lCounter == g_lCounter)
        {
          hbmToDelete = g_data.hbmGorilla;
          g_data = data;
          g_lCounter++;
          fSuccess = TRUE;
        } else {
          hbmToDelete = data.hbmGorilla;
          fSuccess = FALSE;
        }
        LeaveCriticalSection(&g_cs);
        DeleteObject(hbmToDelete);
      } while (!fSuccess);
    }
    

    In addition to the regular gorilla data, we also associate a change counter that is incremented each time somebody pokes the gorilla. In real life, you might want to make the change counter part of the GORILLA­DATA structure. (Actually, in real life, you probably shouldn't poke a gorilla.) In a lock-free algorithm, we would Interlocked­Compare­Exchange­Release the lock counter to see if the lock counter changed (and if not, to update it with the new lock counter). But since a GORILLA­DATA structure cannot be updated atomically, we have to use our critical section to perform the comparison-and-update.

    Even though we used a lock, we still follow the lock-free pattern. If the gorilla has been poked while we were busy processing our own poke, then we throw away the results of our computations and start over, so that our poke can be accumulated with the previous pokes.

    Exercise: What constraints must be applied to the GORILLADATA structure for this technique to work?

  • The Old New Thing

    The performance improvements of a lock-free algorithm is often not in the locking

    • 9 Comments

    GWO wonders what the conditions are under which the lock-free version significantly outpeforms a simple critical section.

    Remember that switching to a lock-free algorithm should be guided by performance measurements. Switching from a simple algorithm to a complex one shouldn't be done unless you know that the simple algorithm is having trouble.

    That said, here are some non-obvious advantages of a lock-free algorithm over one that uses a simple lock. (Later, we'll see how you can take advantage of these techniques without actually going lock-free.)

    Consider a program that uses a simple critical section to perform something like the singleton constructor. Instead of a fancy lock-free algorithm, we use the much simpler version:

    CRITICAL_SECTION g_csSingletonX;
    X *g_px = NULL;
    
    X *GetSingletonX()
    {
        EnterCriticalSection(&g_csSingletonX);
        if (g_px == NULL)
        {
            g_px = new(nothrow) X();
        }
        LeaveCriticalSection(&g_csSingletonX);
        return g_px;
    }
    

    This simple code can run into trouble if the constructor function itself requires some locks, because now you have to impose a lock hierarchy in order to avoid a deadlock. (And this becomes impossible if the constructor function belongs to code outside your control.)

    When working out what your lock hierarchy should be, you may discover that you need to consolidate some locks. This avoids the inversion problem, but it also reduces your lock granularity. You might decide to use a single lock to cover all singletons, and then you later discover that you also have to extend the lock that protects X's constructor to cover other operations on X.

    CRITICAL_SECTION g_csCommon;
    
    // (updated to remove double-check lock because that just raises
    // more questions that distract from the point of the article)
    X *GetSingletonX()
    {
        EnterCriticalSection(&g_csCommon);
        if (g_px == NULL)
        {
            g_px = new(nothrow) X();
        }
        LeaveCriticalSection(&g_csCommon);
        return g_px;
    }
    
    Y *GetSingletonY()
    {
        EnterCriticalSection(&g_csCommon);
        if (g_py == NULL)
        {
            g_py = new(nothrow) Y();
        }
        LeaveCriticalSection(&g_csCommon);
        return g_py;
    }
    
    void X::DoSomething()
    {
        EnterCriticalSection(&g_csCommon);
        .. something ..
        LeaveCriticalSection(&g_csCommon);
    }
    

    Over time, your quiet little singleton lock has turned into a high-contention lock in your system.

    One nice thing about a lock-free algorithm is that since there is no lock, it can't create inversion in a lock hierarchy. (Of course, you have to be careful not to use the interlocked operations to build a private lock, because that puts you back where you started.)

    Another nice consequence of a lock-free algorithm is that, since there is no lock, you don't have to handle the WAIT_ABANDONED case. The data structure is never inconsistent; it passes atomically from one consistent state to another. Therefore, there's no need to write code to clean up leftover inconsistency. This came in handy in a case we looked at earlier, so that an application which crashes at an inopportune time will not corrupt the shared data and require a server reboot.

  • The Old New Thing

    Corrections to Patterns for using the InitOnce functions

    • 3 Comments

    Adam Rosenfield pointed out that it is not possible to fail an asynchronous initialization; if you pass INIT_ONCE_INIT_FAILED when completing an asynchronous initialization, the function fails with ERROR_INVALID_PARAMETER. (Serves me right for writing an article the night before it goes up.) A more correct version is therefore

    ITEMCONTROLLER *SingletonManager::Lookup(DWORD dwId)
    {
     ... same as before until we reach the "singleton constructor pattern"
    
     void *pv = NULL;
     BOOL fPending;
     if (!InitOnceBeginInitialize(&m_rgio[i], INIT_ONCE_ASYNC,
                                  &fPending, &pv)) return NULL;
    
     if (fPending) {
      ITEMCONTROLLER *pic = m_rgsi[i].pfnCreateController();
      if (!pic) return NULL;
      if (InitOnceComplete(&m_rgio[i], INIT_ONCE_ASYNC, pic)) {
       pv = pic;
      } else {
       // lost the race - discard ours and retrieve the winner
       delete pic;
       InitOnceBeginInitialize(&m_rgio[i], INIT_ONCE_CHECK_ONLY,
                               X&fPending, &pv);
      }
     }
     return static_cast<ITEMCONTROLLER *>(pv);
    }
    

    In other words, the pattern is as follows:

    • Call Init­Once­Begin­Initialize in async mode.
    • If it returns fPending == FALSE, then initialization has already been performed and you can go ahead and use the result passed back in the final parameter.
    • Otherwise, initialization is pending. Do your initialization, but remember that since this is a lock-free algorithm, there can be many threads trying to initialize simultaneously, so you have to be careful how you manipulate global state. This pattern works best if initialization takes the form of creating a new object (because that means multiple threads performining initialization are each creating independent objects).
    • If initialization fails, then abandon the operation.
    • Call Init­Once­Complete with the result of your initialization.
    • If Init­Once­Complete succeeds, then you won the initialization race, and you're done.
    • If Init­Once­Complete fails, then you lost the initialization race and should clean up your failed initialization. In that case, you should call Init­Once­Begin­Initialize one last time to get the answer from the winner.

    While I'm here, I may as well answer the exercises.

    Exercise: Instead of calling Init­Once­Complete with INIT_ONCE_INIT_FAILED, what happens if the function simply returns without ever completing the init-once?

    Answer: The INIT_ONCE structure is left in an asynchronous initialization pending state. This is fine, because the next attempt to initialize will simply join the race. (And it will win since we already quit the race!)

    Exercise: What happens if two threads try to perform asynchronous initialization and the first one to complete fails?

    Answer: If two threads both begin initialization and the first one to come to a result concludes that the initialization fails, then it will abandon the initialization. The second thread will then come to its own conclusion. If that conclusion is also failure, then it too will abandon the initialization as well. If that conclusion is that initialization was successful, then its completion will succeed and the INIT_ONCE will enter the initialized state.

    Exercise: Combine the results of the first two exercises and draw a conclusion.

    Answer: It is fine to abandon a failed initialization (and indeed, given what we learned above, it is indeed mandatory).

    There is a documentation update coming soon to clarify that you cannot combine INIT_ONCE_ASYNC and INIT_ONCE_INIT_FAILED.

  • The Old New Thing

    Endorsement: Aaron Margosis's Unintended Consequences of Security Lockdowns talk at TechEd

    • 6 Comments

    At TechEd 2011 North America in Atlanta, Aaron Margosis is presenting a talk on Unintended Consequences of Security Lockdowns. I've seen the internal version of his talk and I give it two thumbs up. If you're going to be at TechEd North America, consider adding it to your schedule.

Page 1 of 3 (28 items) 123