• The Old New Thing

    What happens to lost timer messages if I don't process them fast enough?


    Some time ago, I noted that if your WM_TIMER handler takes longer than the timer period, your queue will not fill up with WM_TIMER messages. The WM_TIMER message is generated on demand, and if your handler doesn't check for messages, then there is no demand for timer messages. But what happens when your thread finally starts processing messages again? What happens to the timers that elapsed while you were busy? Do they accumulate?

    Here's a sketch of how timers work. (Note that the timers under discussion here are the timers set by the Set­Timer function.)

    When a timer is created, it is initially not ready.

    Every n milliseconds (where n is the period of the timer), the timer is marked ready. This is done regardless of the state of the UI thread. Note that ready is a flag, not a counter. If the timer is already ready, then it stays ready; there is no such thing as "double ready". The QS_TIMER flag is set on the queue state, indicating that there is now a pending timer for the thread. This in turn may cause a function like Get­Message or Msg­Wait­For­Multiple­Objects to wake up.

    When the appropriate conditions are met (as discussed earlier), the window manager checks whether there are any timers for the thread that are marked ready. If so, then the corresponding WM_TIMER message is generated and the ready flag is cleared.

    Let's illustrate this with our scratch program. Make the following changes:

    #include <strsafe.h>
    DWORD g_tmStart;
    void SquirtTime()
     TCHAR sz[256];
     StringCchPrintf(sz, 256, "%d\r\n", GetTickCount() - g_tmStart);

    This adds a function that prints the number of milliseconds which have elapsed since g_tmStart. Note that we use simple subtraction and rely on unsigned arithmetic to handle timer rollover issues.


    Our timer tick handler merely prints the elapsed time to the debugger.

    OnCreate(HWND hwnd, LPCREATESTRUCT lpcs)
     g_tmStart = GetTickCount();
     SetTimer(hwnd, 1, 500, OnTimer);
     return TRUE;

    Finally, we create a 500ms timer on our window, but we also intentionally stall the thread for 1750ms.

    Can you predict the output of this program?

    Here's what I got when I ran the program:


    Let's see if we can explain it.

    Since the timer is set to fire at 500ms intervals, every 500ms, the timer gets marked ready.

    • At T = 500ms, the timer is marked ready.
    • At T = 1000ms, the timer is marked ready. This is redundant, since the timer is already ready.
    • At T = 1500ms, the timer is marked ready. Again, this is redundant.
    • At T = 1750ms, the program finally wakes up from its Sleep and eventually gets around to processing messages.
    • Hey look, there is a ready timer, so we generate a WM_TIMER message and clear the ready flag.
    • At T = 1797ms, the timer message is processed.
    • The program returns to its message loop, where there are no further messages to process, so it sits and waits.
    • At T = 2000ms, the timer is marked ready. This causes the Get­Message to wake up generate a WM_TIMER message and clear the ready flag.
    • At T = 2000ms, the timer message is processed.
    • At T = 2500ms, the timer is marked ready. This causes the Get­Message to wake up generate a WM_TIMER message and clear the ready flag.
    • At T = 2500ms, the timer message is processed.
    • And so on, with a new timer message every 500ms that is processed immediately.

    Observe that when the program begins processing messages at T = 1750ms, it receives only one timer message right away, even though three timer periods have elapsed.

    I guess another way of looking at this is to think of your timer as setting a frame rate. If your thread is busy when it's time to render the next frame, then the next frame is late. And if your thread is really busy, it may drop frames entirely.

  • The Old New Thing

    Why doesn't my keyboard hook get called for keyboard messages I manually posted?


    A customer was generating synthesized keyboard input by posting WM_KEY­DOWN and related messages to a window. They found that a WH_KEYBOARD keyboard hook was not triggering for these keyboard messages and they wanted to know why.

    You already know enough to answer this question, though the pieces are scattered about over several years for you to put together.

    First of all, you can't simulate keyboard input with Post­Message because you are going straight to the end product without going through the other paperwork that leads to that end product.

    It's like sending a letter to a friend by putting it directly in his mailbox, and then saying, "My friend filed a change of address with the postal service, and anything sent to his address is supposed to be redirected to his new address, but my letter is still sitting in his mailbox at his old address." Well, yeah, because you didn't actually mail the letter. You bypassed the mail system and merely replicated the end product (a letter in the mailbox). You will also find that if you go to the postal service's Web site and ask for a delivery confirmation of the letter, the Web site is going to say, "Sorry, we have no record of having delivered that letter to that mailbox."

    The window manager does not have a fake message detector that looks to see if you posted a fake input message, and if so, reverse-engineers the logic that led to that fake input message and internally simulates the actions of that reverse-engineered logic. If you post a WM_CHAR message, it's not going to say, "Well, let me see, to get to that message, the user needed to have pressed the Shift key, then pressed the A key, then released the A key, then released the Shift key, so I'll retroactively send out WH_KEYBOARD hook notifications for those events, and if any of the hooks blocks the event, then I created a time paradox, and I have to go back in time and kill the program that posted the WM_CHAR message." (It's also not going to insert the posted message in the processing queue in the correct order relative to true input messages.)

    If you want to simulate input, you need to send it through the input system with functions like Send­Input.

  • The Old New Thing

    How do I call SetTimer with a timer ID that is guaranteed not to conflict with any other timer ID?


    Suppose you want to create a timer for a window, but you don't know what timers that window already has running. You don't want your timer to overwrite any existing timer, so you need to find a unique ID for your timer. If this were for a window message, you could use Register­Window­Message, but there is no corresponding Register­Timer­Id function. How do you generate a unique timer ID?

    Every window gets to decide how to assign its own timer IDs, which are 32-bit integers on 32-bit Windows or 64-bit integers on 64-bit Windows. If you are an outsider, then you will have to negotiate with the window for a timer ID. There is no standard for this, so you will have to talk to whoever wrote the code for the window and come to some sort of agreement. "Okay, Bob, I'll let you have timers in the range 1000 to 1999."

    But what if you don't personally know the person who wrote the code for the window, and the documentation for the window class does not specify how to negotiate a timer ID?

    If you don't have a way to negotiate a timer ID for a window, then just create your own window and put your timer there. Since you created your own window, you control the window class, and you can set the rules for how timer IDs are assigned for that window class.

    It's hardly a coincidence that the timer ID space is the same size as the address space. You can allocate some memory to track the timer (you probably have already done this), and use the address of that memory as the timer ID. Another common design is to use the handle to the object associated with the timer as the timer ID.

  • The Old New Thing

    It rather involved being on the other side of this airtight hatchway: Elevation from Administrator to SYSTEM


    A security vulnerability report arrived that took the following form:

    I have discovered a critical security vulnerability in Windows which I intend to present at the XYZ conference. It allows any user with administrator privileges to perform operation Q, something that should be available only to SYSTEM.

    I think you know how this story ends. If you have administrator privileges, then you are already on the other side of the airtight hatchway. That you can use administrator privileges to pwn the machine is not interesting, because by virtue of being an administrator you already pwn the machine.

    There is formally a distinction between Administrator and SYSTEM, seeing as they are some things which are ACL'd so that SYSTEM can do them and not arbitrary adminitrators, but that distinction is formal and not practical. An administrator who wanted to get some code running as SYSTEM could install a service that runs as SYSTEM. Or use Debug Privilege to take over a process (say, a service) running as SYSTEM. Or simply open a command prompt as SYSTEM and go to town. No need to go through the complex operation Q to get SYSTEM access.

    So yes, a user with administrator privileges can use operation Q to do things that are normally limited to SYSTEM. But so what? Users with administrator privileges already have plenty of easier ways of doing things that are normally limited to SYSTEM. The distinction between the SYSTEM and Administrator accounts is a roadblock to make it harder to mess up your system by mistake. You can still mess up your system, but you have to try harder.

    Before dismissing these reports, you have to verify that the attack is effective only against the current machine. In other words, that obtaining administrator privileges on the computer gets you nothing more than administrator privileges on the computer. And in this case, that was true. The attack described gives the user access to the local machine, but had no effect on other machines.

  • The Old New Thing

    Thanks for informing us that you know about the compatibility issue in your application, so we don't have to fix it


    During the public prerelease period of a version of Windows some time ago, I remember a bug came in from an application developer. It went something like this:

    Hi, we just tried our program on the most recent public prerelease version of Windows, and we noticed that you changed the undocumented XYZ function. Our program uses that undocumented function, and your change to the function breaks us. can you change it back? Yes, we understand that there are no guarantees surrounding the use of undocumented APIs, but still we'd like you to continue supporting the old version. It should just be a few dozen lines of code on your side.

    How very nice of the application vendor to identify their compatibility issues for us. Now we don't need to create a compatibility shim, because not only do we know that the application vendor is still actively supporting the program in question, but that they have already been alerted to the problem with plenty of advance warning for them to issue a patch to their customers before the next version of Windows is released.

    The product team rejected this change request with the simple comment, "Unsupported APIs are unsupported." Presumably the customer liaison translated this into something a bit more polite before communicating it back to the vendor.

  • The Old New Thing

    How can I tell if Windows Update is waiting for the system to reboot?


    Today's Little Program shows how to ask whether Windows Update is waiting for the system to reboot. You might want to check this in your installer, so that you don't try to install your program while the system is in a mixed state where an update is partially-installed.

    Testing this from script is easy. MSDN contains sample code to do that. Let's do it from C++. Today's smart pointer class is (rolls dice) _com_ptr_t! Remember that Little Programs do little to no error checking.

    #include <windows.h>
    #include <comdef.h> // for _COM_SMARTPTR_TYPEDEF
    #include <wuapi.h>  // for ISystemInformation
    #include <stdio.h>  // for printf (horrors! mixing stdio and C++!)
    _COM_SMARTPTR_TYPEDEF(ISystemInformation, __uuidof(ISystemInformation));
    int __cdecl main(int argc, char** argv)
     CCoInitialize init;
     ISystemInformationPtr info;
     VARIANT_BOOL rebootRequired;
     printf("Reboot required? %d\n", rebootRequired);
     return 0;

    Remember that VARIANT_BOOL uses -1 to represent VARIANT_TRUE, so if a reboot is required, you will see -1. Personally, I would treat any nonzero value as logically true.

  • The Old New Thing

    Why are there all these processes lingering near death, and what is keeping them alive?


    A customer (via their customer liaison) asked for assistance debugging a failure that occurs when their system remains up for several weeks.

    The customer seems to have a complicated system where they create and kill processes, and I am seeing hundreds of processes in the following state.

    PROCESS fffffa80082a7960
        SessionId: 0  Cid: 1490    Peb: 7efdf000  ParentCid: 2614
        DirBase: 1b3fd0000  ObjectTable: 00000000  HandleCount:   0.
        Image: contoso.exe
        VadRoot 0000000000000000
        DeviceMap fffff8a000008ca0
        Token                             fffff8a00dbf9060
        ElapsedTime                       1 Day 01:25:38.983
        UserTime                          00:00:01.903
        KernelTime                        00:00:00.265
        QuotaPoolUsage[PagedPool]         0
        QuotaPoolUsage[NonPagedPool]      0
        Working Set Sizes (now,min,max)  (5, 50, 345) (20KB, 200KB, 1380KB)
        PeakWorkingSetSize                17981
        VirtualSize                       222 Mb
        PeakVirtualSize                   246 Mb
        PageFaultCount                    29532
        MemoryPriority                    BACKGROUND
        BasePriority                      8
        CommitCharge                      0
    No active threads
            THREAD fffffa800a358b50  Cid 1490.2704  TERMINATED

    This got me curious: Why are there so many of these near-death processes, and what could be keeping them alive?

    I'll let you puzzle on this for a little bit. But you already know the answer.




    First thing you should observe is that this process is not actually alive. It has already exited. "No active threads." The one thread that is still associated with the process has terminated.

    Why would you have a terminated thread hanging around inside a terminated process?

    Because there is still an outstanding handle to the thread.

    Even though the thread has exited, the thread object can't go away until all handles to it have been closed.

    Now, when you create a process with the Create­Process function, you get a PROCESS_INFORMATION structure back which contains four pieces of information:

    1. A handle to the created process.
    2. A handle to the initial thread in the process.
    3. The ID of the created process.
    4. The ID of the initial thread in the process.

    Of those things, you are probably interested in the process handle, because that's the thing you can wait on to learn when the process has exited. And you probably ignore the thread handle.


    You need to close the thread handle, or the thread cannot go away. It may have stopped executing, but the fact that you have a handle to it means that you can still do things like check if the thread has exited (yes, already!), ask for the exit code, ask for the thread ID, ask how much CPU the thread consumed during its lifetime, and so on, and all those statistics are kept in the thread object. And since thread and process IDs need to remain unique as long as there is still a handle to the object, the object needs to hang around so that it "occupies space" so that no other thread can grab its ID.

    The customer called this process near death, but the more conventional term for it is zombie. In fact, zombie isn't a good term either, because this process and this thread are well and truly dead, never to walk again.

    A better name would be corpse. The process and thread are dead. They're just lying there, rotting away in memory, waiting for all references to be released so they can disappear entirely.

    Since the customer liaison said that the customer has "a complicated system where they create and kill processes", it's entirely possible that somewhere in the complicated system, somebody loses track of the thread handle, causing it to leak. It also calls into question whether they need this complicated system at all. Maybe their complicated system exists to work around some other problem, and we should be trying to solve that other problem.

    Just for completeness, another possibility for the thread lying dead in the process is that some kernel driver has taken a reference to the thread and has gotten stuck.

    We left the customer liaison with that information. We didn't hear back, so either our guess about thread handles was correct, or the customer decided we weren't being helpful enough and decided to stop talking to us.

  • The Old New Thing

    How can I start my service as soon as possible, before any other service?


    A customer wanted to know how they could make their service start before any other service.


    Of course, a simple application of the What if two programs did this? principle shows that this is not possible.

    Upon closer questioning, the real problem was that they had a service which configured some piece of hardware, and they wanted to make sure this configuration was completed before starting the service that uses the hardware. "But we don't want to create a dependency between the two services."

    Okay, so first of all, they didn't actually want to be FRIST. They merely wanted to come ahead of the other service. And the way to do that is to create a service dependency so that the other service depends on their custom service. But then they added a remark where they pre-emptively rejected the solution to their problem.

    The way to control the order in which services start up is to create dependencies among them. Rejecting the correct solution only leads you down the path to creating incorrect solutions. And incorrect solutions tend to create all sorts of problems.

    So do the right thing. Create the service dependency.

  • The Old New Thing

    Why doesn't GetAddrInfo work from behind a proxy?


    A customer was having a problem with the Get­Addr­Info function when running inside a corporate proxy environment.

    We are trying to get the IP address of, say, www.microsoft.com by using the Get­Addr­Info function. This works fine if the computer is not behind a proxy, but if it is run in a corporate proxy environment, the call fails with WSAHOST_NOT_FOUND.

    The Get­Addr­Info function operates at a level below proxies. When you have a Web proxy, the computer never actually talks to www.microsoft.com directly. Instead, you connect to the proxy and tell the proxy, "Please contact www.microsoft.com for me, thanks." That's why it's called a "proxy".

    You never see the IP address of www.microsoft.com; the only IP address you see is that of the proxy. Besides, since you are inside a corporate proxy environment, even if you had the IP address for www.microsoft.com, it is if no use to you since you cannot connect to it.

    There are products that try to smooth over this boundary, so that programs think that they are connected directly to the Internet when in fact they are talking through the proxy.

  • The Old New Thing

    A process inherits its environment from its parent, and the consequences of this simple statement


    A customer reported that they changed some system-level enviornment variables (include PATH). They also have a batch file that, among other things, runs a program that requires that the PATH be set a certain way. The batch file is set to run automatically every five minutes, and the customer observed that the batch file is not picking up the changes to the system-wide environment variables.

    The customer developed two theories as to why this was happening.

    Theory number one was that batch files inherit the environment from its parent process (unless the parent process passes an explicit environment to Create­Process). "Since we changed a global environment variable and the parent process is continually running, it does not pick up the new environment variables. Then it passes this outdated set of environment variables to the batch file."

    Theory number two is that changes to global environment variables will eventually become applied to running processes, but the effect is not immediate; you have to wait a while for the change to propagate. "We started a new command prompt, and the new command prompt did not have the new environment variables, even though the new values did appear in the System control panel. After setting the environment variable a few more times, eventually the new values started showing up in the command prompt."

    The first theory is correct. Processes inherit their initial environment from their parents. Of course, once the process is running, it is free to change its environment variables by calling Set­Environment­Variable, and those modified environment variables are passed to any child processes launched after the new variable is set. (A parent process can also pass a custom environment to the child process.)

    Now, a child process inherits its initial environment from its parent, but it only gets a snapshot of that environment. If the parent subsequently modifies its environment, the child environment is not updated.

    When the customer said that they "changed some global environment variables", that was a rather vague statement, because as we saw above, there is technically no such thing as a "global environment variable". Every environment variable is local to a process. What the customer actually did was make a chance to the template environment that is used to get the ball rolling.

    When you log on, the system constructs an initial environment from the template specified in the Environment Variables control panel. That initial environment is then given to Explorer, where it becomes Explorer's initial environment, and from there, it becomes inherited by anything that gets launched by Explorer, and then anything that gets launched by anything that gets launched by Explorer, and so on. If you can trace your lineage back to Explorer, then your initial environment was based on the copy in Explorer.

    Now, each process along the way may have edited its environment before spawning a child process, in which case those edited values are inherited by the children.

    In order for the changes to the environment template to take effect in a process, that process needs to support the "Throw away all your environment variables and create a new set of environment variables based on the current template" message. That message is the WM_SETTING­CHANGED message, where the lParam points to the string "Environment".

    The only program in common use that actually responds to that message is Explorer.

    This means that when you change the template for the initial environment from the Environment Variables control panel, the only program that reacts immediately to those changes is Explorer. Any other programs that are already running will continue to operate with whatever environment variables they had when they started (or subsequently edited).

    If you run a program from Explorer, it will get the updated environment because Explorer updated its environment in response to the message.

    If you run a program from Task Manager or a command prompt or anything else, then it will not get the updated environment because Task Manager, the command prompt, and pretty much anything else does not update its environment in response to the message.

    If you want to regenerate your environment in response to the WM_SETTING­CHANGED message with an lParam of Environment, you can use the Create­Environment­Block function. You can enumerate the contents of this environment block and copy it to your current environment.

    But personally, I wouldn't bother, because pretty much nobody else bothers either.

Page 6 of 464 (4,639 items) «45678»