• The Old New Thing

    Sure, I'm supposed to pass WT_EXECUTELONGFUNCTION if my function takes a long time, but how long is long?


    A customer contacted me to tell a story and ask a question. The customer discovered that in their code base, all calls to Queue­User­Work­Item passed the WT_EXECUTE­LONG­FUNCTION flag, regardless of whether the function actually took a long time or not. Their program creates a large number of work items at startup, and the result of passing WT_EXECUTE­LONG­FUNCTION for all of them was that the thread pool created a new thread for each queued work item, resulting in a bloated thread pool that thrashed the CPU.

    When he asked the other people on his team why they were passing the WT_EXECUTE­LONG­FUNCTION flag unconditionally, they pointed to this article from 2005 on the importance of passing the WT_EXECUTE­LONG­FUNCTION flag to the Queue­User­Work­Item function.

    As I've mentioned before, Good advice comes with a rationale so you can tell when it becomes bad advice, but the people who applied my advice didn't understand the rationale and merely concluded, "It is important always to pass the WT_EXECUTE­LONG­FUNCTION flag!"

    The WT_EXECUTE­LONG­FUNCTION flag is two-edged. If you pass the flag when queueing a task, then the thread pool will more aggressively create a new thread when that task is running. The upside is that other tasks don't get stuck waiting for your long-running task. The downside is that this creates more threads. And if you set the flag for all of your tasks, then you don't really have a thread pool at all, since you basically told the thread pool, "Run every task on its own thread, stat!"

    But this raises the question of "How long is long?" How long does a task need to run before you declare it a long-running task?

    There is no magic number here.

    The definition of a long-running task depends on the nature of your application. Let's consider, for concreteness, a task that takes one second. If this task is not marked as a long-running task, then the thread pool will wait for it to complete rather than creating a new thread. What are the consequences for your application of the thread pool choosing to wait for one second rather than creating a new thread? If your application doesn't generate tasks at such a high rate that a one-second pause would be a significant problem, then it's not a long-running task.

    On the other hand, if your application is a service that is handling thousands of requests per second, then waiting for a one-second tasks means that a thousand tasks pile up in the meantime, and that may be enough to push your service to the brink of death because it has started falling behind on its processing and may never catch up.

    Which category does your application fall in? That's for you to determine.

  • The Old New Thing

    What does it mean when my program exits with the message "This application has requested the Runtime to terminate it in an unusual way"?


    You're running your program, and then it suddenly exits with the message This application has requested the Runtime to terminate it in an unusual way. What happened?

    That message is printed by the C runtime function abort, the same function that also causes your program to terminate with exit code 3.

    Your program might call abort explicitly, or it might end up being called implicitly by the runtime library itself.

    The C++ standard spells out the conditions under which terminate is called, and it's quite a long list, so I won't bother repeating them here. Consult your favorite copy of the C++ standard for details. (The most common reason is throwing an unhandled exception.)

  • The Old New Thing

    GetParent, just as confusing as EnumClaw, but it's an actual function!


    The function Enum­Claw, documented as returning "the child or the parent of the window", was a joke, but there's a function whose behavior is just as confusing as the joke function Enum­Claw: Get­Parent.

    The Get­Parent function returns the parent window, or owner window, or possibly neither. All that's left is for it to be a floor wax and it'll have everything covered.

    The idea behind Get­Parent is that it returns the parent window. Only child windows have parents, so what happens if you pass something that isn't a child window? Well, we shouldn't let a parameter go to waste, right? So let's have it return the owner window if you pass in a top-level window. But wait, we're not going to return the owner window for all top-level windows, just for top-level windows which have the WS_POPUP style.

    This last bit regarding the WS_POPUP style is a leftover from Windows 1.0, where there was a distinction made between "tiled windows" and "popup windows." Popup windows could overlap, whereas tiled windows could not, and the function was only interested in windows that can overlap. Of course, now that all windows can overlap, the rejection of tiled windows is just a compatibility remnant.

    Anyway, the algorithm for Get­Parent goes like this:

    • If the window is a child window, then return the parent window.
    • Else, the window is a top-level window. If WS_POPUP style is set, and the window has an owner, then return the owner.
    • Else, return NULL.

    Here it is in tabular form, since I've discovered that people like tables:

    Get­Parent return values WS_CHILD
    Set Clear
    WS_POPUP Set N/A Owner
    Clear Parent NULL

    The upper left entry of the table (corresponding to WS_CHILD and WS_POPUP both set) is left as N/A because that combination of styles is illegal.

    Fortunately, you don't have to deal with all the craziness of the Get­Parent function. There are ways to get the parent or owner separately and explicitly without having to deal with Get­Parent's quirks.

    • To get the parent window, call GetAncestor(hwnd, GA_PARENT).
    • To get the owner window, call GetWindow(hwnd, GW_OWNER).

    Compatibility requirements prevent Get­Parent from ever going away, but that doesn't mean you are forced to continue using it. Use one of the less confusing alternatives. That's why they're there.

  • The Old New Thing

    A feature I didn't even know existed much less had a name: Color hot-track


    I hadn't even noticed this until somebody pointed it out: When you hover your mouse over a button in the Windows 7 taskbar which corresponds to a running application, the taskbar button lights up in a color that matches the colors in the icon itself. (And even more subtly, the lighting effect is centered on the mouse.)

    This feature even has a name: Color hot-track. (Gentlemen, start your photocopiers.)

    Some people ask how it's done. It's really nothing special. The code just looks for the predominant color in the icon. (And, since visual designers are sticklers for this sort of thing, black, white, and shades of gray are not considered "colors" for the purpose of this calculation.)

  • The Old New Thing

    Why does my program still show up in the Programs and Features folder after my uninstaller exits?


    A customer reported that they were having problems with their program entry in the Programs and Features folder (formerly known as Add and Remove Programs, formerly known as Add/Remove Programs). When the user goes to the Programs and Features folder, selects their program, and clicks Uninstall, the uninstaller runs, but after the uninstaller exits, their entry remains in Programs and Features. They have to manually refresh the folder to get it to disappear. Why won't the item disappear?

    The Programs and Features folder launches your uninstaller and waits for it to exit, at which point the Programs and Features folder looks to see if the uninstall entry for your program still exists in the registry. (There is no standard method for an uninstaller to report success or failure back to their caller, so this is the only technique available.) If the registry entry has been deleted, then the Programs and Features folder concludes that the program was successfully uninstalled and removes the item from view.

    The trick is the "waits for it to exit" part. If your uninstaller in turn launches another program to do the real work of uninstalling, the Programs and Features folder doesn't know or care. It just waits for your uninstaller to exit. If your uninstaller exits before the uninstall is complete (because it handed off the work to some helper process), then when the uninstaller exits, the Programs and Features folder looks for the uninstall entry in the registry, finds that it's still there (because your helper is still running), and concludes that the uninstall was unsuccessful.

    Sadly, as is not uncommon, the customer replied with additional information that indicated that they didn't understand the explanation.

    Thanks. We logged all accesses to our uninstall key, and we found that the key is accessed by two processes: explorer.exe and msiexec.exe. The last access is from msiexec.exe. A log file has been attached for reference.

    Yes, this is completely consistent with the uninstaller exiting before the helper (here: msiexec.exe) finishes its work. The Programs and Features folder (running inside explorer.exe) checks the key, finds that it's still there, and leaves the item in the view. And then msiexec.exe finishes its work and deletes the key. But by then it's too late; the Programs and Features folder already concluded that the uninstall was unsuccessful.

    Bonus chatter: Why doesn't the Programs and Features folder wait until the uninstaller and all its child processes exit?

    Because that doesn't work either. It is common for installers to launch child processes that it intends to outlive the installer. For example, it might end with a dialog box that says, "Congratulations, your install was successful. Do you want to run the program now?" Or it might launch some background task to create an annoying notification icon.

    Anything an installer does, an uninstaller can do, because one of the options in the uninstaller might be Repair or Reinstall. If the Programs and Features folder waited for all child processes to exit, then it might end up concluding that the uninstaller never finished because some random background task is still running.

  • The Old New Thing

    Don't let more than one process try to read from stdin at the same time


    A customer reported a problem with a program that ran a series of other programs in parallel.

    We have a main program (call it main.exe) that runs a bunch of child processes with stdout and stderr redirected. (We are not redirecting stdin.) We've found that some of the child processes get stuck inside the C runtime startup code on a call to Get­File­Type on the stdin handle. What could be the reason for this? Is there something we can do that doesn't require us to modify the child processes? (They are third party code we do not have control over.)

    This is one of those once you've debugged this problem you never forget it type of problems.

    Notice that each of the child processes inherits the same stdin from main.exe, since you aren't redirecting stdin. Since the stdin handle was not opened as overlapped, all I/O to the handle is serialized.

    The C runtime calls Get­File­Type at startup to determine whether or not to use buffering. When each child process starts up, it calls Get­File­Type, enters its main, and goes about its business. Everything is great until one of them tries to read from stdin. At that point, everything falls apart.

    The next child process to start calls Get­File­Type, but instead of returning with a result, it waits for the previous I/O request (the read) to complete because the handle is marked synchronous, and synchronous handles permit only one operation at a time. The user, of course, doesn't realize that the first program is waiting for input (the prompt got redirected), so the user sits and waits for the program while the program sits and waits for the user.

    To solve this problem, you first need to decide what you want to happen to stdin. Right now, you gave stdin to a dozen child processes, and each line of input the user types will be randomly assigned to one of those programs. In this case, the customer's answer is "I don't care about stdin; these programs aren't supposed to be reading from stdin anyway", in which case you can redirect stdin of the child processes to NUL.

    Bonus chatter: This is also why, when you hit Ctrl+C to exit a console program which launched child processes with CREATE_NEW_PROCESS_GROUP, the command prompt that comes back sometimes behaves kind of strangely. Since the child processes were launched in a separate process group, the Ctrl+C killed the main program but left the child processes running. If any of those child processes read from stdin, you get the "Randomly assign input" effect described above because you have two programs racing to read from stdin: the orphaned child process and cmd.exe.

  • The Old New Thing

    Our code needs to run on multiple platforms with different rules, so we follow none of them!


    A customer was encountering sporadic crashes in their 64-bit application, and upon investigation, the problem was traced to a misaligned RSP register. We saw some time ago that the Windows x64 calling convention requires the RSP register to be 16-byte aligned.

    The customer traced the source of the misalignment to a third-party library they were using. They contacted the vendor, who acknowledged that they were not following the Windows x64 calling conventions, but explained that their code needs to run on multiple x64 operating systems, and since each operating system has different calling conventions, they adhere to none of them!

    I was kind of boggled by this remark. Yes, it's frustrating that different operating systems have different calling conventions, but that doesn't mean that you are welcome to ignore them. Every region of the world has different laws regarding the operation of motorized vehicles, but that doesn't mean "My truck driver has to drive through all these different jurisdictions with different rules, so he follows none of them!"

  • The Old New Thing

    If you protect a write with a critical section, you may also want to protect the read


    It is common to have a critical section which protects against concurrent writes to a variable or collection of variables. But if you protect a write with a critical section, you may also want to protect the read, because the read might race against the write.

    Adam Rosenfield shared his experience with this issue in a comment from a few years back. I'll reproduce the example here in part to save you the trouble of clicking, but also to make this entry look longer and consequently make it seem like I'm actually doing some work (when in fact Adam did nearly all of the work):

    class X {
     volatile int mState;
     HANDLE mEvent;
    X::Wait() {
     while(mState != kDone) {
       WaitForSingleObject(mEvent, INFINITE);
    X::~X() {
    X::SetState(int state) {
     // do some state logic
     mState = state;
     X x;
     ... spawn off thread 2 ...
    Thread2(X* px)

    There is a race condition here:

    • Thread 1 calls X::Wait and waits.
    • Thread 2 calls X::SetState.
    • Thread 2 gets pre-empted immediately after calling Set­Event.
    • Thread 1 wakes up from the Wait­For­Single­Object call, notices that mState == kDone, and therefore returns from the X::Wait method.
    • Thread 1 then destructs the X object, which destroys the critical section.
    • Thread 2 finally runs and tries to leave a critical section that has been destroyed.

    The fix was to enclose the read of mState inside a critical section:

    X::Wait() {
     while(1) {
       int state = mState;
       if(state == kDone)
       WaitForSingleObject(mEvent, INFINITE);

    Forgetting to enclose the read inside a critical section is a common oversight. I've made it myself more than once. You say to yourself, "I don't need a critical section here. I'm just reading a value which can safely be read atomically." But you forget that the critical section isn't just for protecting the write to the variable; it's also to protect all the other actions that take place under the critical section.

    And just to make it so I actually did some work today, I leave you with this puzzle based on an actual customer problem:

    class BufferPool {
     BufferPool() { ... }
     ~BufferPool() { ... }
     Buffer *GetBuffer()
      Buffer *pBuffer = FindFreeBuffer();
      if (pBuffer) {
       pBuffer->mIsFree = false;
      return pBuffer;
     void ReturnBuffer(Buffer *pBuffer)
      pBuffer->mIsFree = true;
     Buffer *FindFreeBuffer()
      Buffer *pBuffer = NULL;
      for (int i = 0; i < 8; i++) {
       if (mBuffers[i].mIsFree) {
        pBuffer = &mBuffers[i];
      return pBuffer;
     Buffer mBuffers[8];

    The real class was significantly more complicated than this, but I've distilled the problem to its essence.

    The customer added, "I tried declaring mIs­Free as a volatile variable, but that didn't seem to help."

  • The Old New Thing

    Things I've written that have amused other people, Episode 8


    In a technical discussion, I opened a reply with

    Bob's paper which I haven't yet read points out that...

    Some people wrote to me to say that the've added this quote to their file in the hopes of being able to use it themselves someday.

    For those who are curious, I found the technical discussion in question. It had to do with whether the following code is thread-safe:

    // initial conditions
    int x = 1, y = 0;
    int *p = &x;
    // Thread 1 executes this
    y = 1;
    p = &y;
    // Thread 2 executes this
    print *p;

    Question: Can this code legitimately print 0?

    Surprisingly, the answer is yes!

  • The Old New Thing

    Why is CLIPFORMAT defined to be a WORD rather than a UINT?


    Commenter Ivo wants to know if the Register­Clipboard­Format function returns a UINT, why is the CLIP­FORMAT data type defined to be a WORD? Since a WORD is smaller than a UINT, you have to stick in a cast every time you assign the result of Register­Clipboard­Format to a CLIP­FORMAT.

    Rewind to 16-bit Windows. Back in those days, a UINT and a WORD were the same size, namely, 16 bits. As a result, people got lazy about the distinction. Six of one, a half dozen of the other. (People are lazy about this sort of distinction even today, assuming for example that UINT and DWORD are the same size, and in turn forcing UINT to remain a 32-bit integer type even on 64-bit Windows.) The Register­Clipboard­Format function came first, and when the OLE folks wanted to define a friendly name for the data type to hold a clipboard format, they said, "Well, a clipboard format is a 16-bit integer, so let me use a 16-bit integer." A WORD is a 16-bit integer, so there you go.

    This mismatch had no effect in 16-bit code, but once Win32 showed up, you had a problem since 32-bit Windows expanded the UINT type to 32 bits. Not only does keeping a CLIP­FORMAT in a WORD create the need for all this casting, it also leaves two bytes of padding in the FORMAT­ETC structure. Strike two.

    Yeah, basically, it sucks.

Page 124 of 464 (4,640 items) «122123124125126»