May, 2005

  • The Old New Thing

    Using modular arithmetic to avoid timing overflow problems

    • 21 Comments

    In an earlier article, I presented a simple way of avoiding timing overflows which seemed to create a bit of confusion.

    The short version: Given a starting time start, an ending time end and an interval interval, the way to check whether the interval has elapsed is to use the expression end - start >= interval. The naive expression end >= start + interval suffers from integer overflow problems.

    To simplify the discussion, let's operate in base-100 instead of base-232. The same logic works, but I think operating in base-100 will be easier to follow.

    Base-100 means that we remember only the last two digits of any number. Consider a starting time of start = 90 and an interval of interval = 10. Using the wrong expression yields end >= start + interval = 90+10 = 100 = 0. In other words, end >= 0 which is always true since end has the range 0...99. As a result, the wrong expression will think that the interval has expired prematurely.

    Using the correct expression, we have end - 90 >= 10. Of the numbers 0..99, the ones that give a difference less than 10 are 90 through 99. Once end = 0, the result is 0 - 90 = 10, which correctly indicates that 10 ticks have elapsed since 90 once the timer reaches 0.

    You can work through a similar mistake using start = 89 instead of start = 90; in this case, the wrong expression becomes end >= start + interval = 89 + 10 = 99, or in other words, end >= 99. This has the opposite problem from the previous case, namely that the expression will fail to detect that the interval has expired once the timer rolls over.

    But why does the end - start expression work? It's very simple: You just have to remember your rules of arithmetic from elementary school.

    (x - c) - (y - c) = x - c - y + c = x - y

    In other words, subtracting the same value from both terms of a difference does not affect the final value. This rule applies even to modular arithmetic (because, as the mathematicians like to say, the set of integers modulo n form an additive group).

    This rule is useful because it lets you delay the overflow as long as possible by subtracting the starting point from all your time markers; it has no effect upon time intervals. Wouldn't it be great if start = 0? Then the overflow won't happen for 100 ticks. Well, you can act "as if" the starting point were start = 0 by simply subtracting start from all your time markers.

    Those who prefer a graphical view can think of time passing as the hands around a clock (which wraps around at 60 minutes, say). When you decide to record your start point, rotate the clock so that the "12" precisely lines up with wherever the hand happens to be. You can now read off the elapsed time directly from your rotated clock. Rotating your clock is the same as subtracting (or adding) a constant to all time markers.

    Of course, this trick falls apart once you have to measure time intervals that come close to the wraparound time of your timer. In our 100-tick timer, for example, trying to measure the passage of 90 ticks is very difficult because there is only a 10-tick window where the inequality is satisfied. If we fail to catch the timer during that window, we miss it and have to wait another 90 ticks.

    So don't do that. In practical terms, this means that you shouldn't use GetTickCount to measure time intervals longer than 15 days.

  • The Old New Thing

    Shocked (shocked!) that patronage exists in Chicago politics

    • 4 Comments

    NPR reported on a startling discovery in Chicago: That government jobs go not to those best qualified to perform them, but rather to those with the best connections. Who'd-a thunk it?

    Shocked by this discovery, the Daley administration vowed to end it.

    The City of Chicago's top lawyer Mara Georges told incredulous City Hall reporters yesterday that the hiring system will now be completely fair, objective, and free of political influence. "City hiring will be on the square, yes."

    The Associated Press notes that "City officials have long denied that political favoritism or patronage had anything to do with who got jobs." The Chicago Sun-Times captures the sentiment quite succinctly: Who knew politics as usual in Chicago is against the law?

  • The Old New Thing

    You can't simulate keyboard input with PostMessage

    • 15 Comments

    Some people attempt to simulate keyboard input to an application by posting keyboard input messages, but this is not reliable for many reasons.

    First of all, keyboard input is a more complicated matter than those who imprinted on the English keyboard realize. Languages with accent marks have dead keys, Far East languages have a variety of Input Method Editors, and I have no idea how complex script languages handle input. There's more to typing a character than just pressing a key.

    Second, even if you manage to post the input messages into the target window's queue, that doesn't update the keyboard shift states. When the code behind the window calls the GetKeyState function or the GetAsyncKeyState function, it's going to see the "real" shift state and not the fake state that your posted messages have generated.

    The SendInput function was designed for injecting input into Windows. If you use that function, then at least the shift states will be reported correctly. (I can't help you with the complex input problem, though.)

  • The Old New Thing

    When is x/2 different from x>>1?

    • 25 Comments

    Everyone "knows" that the following pairs of expressions are equivalent:

    x*2 ≡ x<<1
    x/2 ≡ x>>1

    Too bad they aren't.

    In the C language standard, there is no requirement that the internal representation of signed integers be two's complement. All the permissible representations agree for positive numbers, but negative numbers can have different representations. If x is negative, then x*2 and x<<1 are quite different on a sign/magnitude system.

    However, Win32 requires a two's complement machine, in which case the first equivalence x*2 ≡ x<<1 is indeed always true.

    Of course, the compiler is free to recognize this and rewrite your multiplication or shift operation. In fact, it is very likely to do this, because x+x is more easily pairable than a multiplication or shift. Your shift or multiply-by-two is probably going to be rewritten as something closer to an add eax, eax instruction.

    As for the second so-called equivalence, the C language specification originally did not specify whether division of a negative number by a positive number rounds towards or away from zero, but in 1999, the specification was revised to require rounding towards zero. Furthermore, the result of a right-shift of a negative value is unspecified, so the expression x>>1 has an unspecified result if x is negative.

    Even if you assume that the shift fills with the sign bit, The result of the shift and the divide are different if x is negative.

    (-1) / 2 ≡ 0
    (-1) >> 1 ≡ -1

    The moral of the story is to write what you mean. If you want to divide by two, then write "/2", not ">>1".

  • The Old New Thing

    Why does Add or Remove Programs show a large blank space?

    • 43 Comments

    Some people have noticed that certain programs cause the Add or Remove Programs control panel to create an enormous amount of blank space. What's going on?

    These are programs that have bad custom uninstall icon registrations.

    If you go to the registry key HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\CurrentVersion\Uninstall, you'll find a list of programs that have registered for appearing in the Add or Remove Programs control panel. Some of them might have been so kind as to provide a "DisplayIcon" value, thereby saving the control panel the indignity of having to guess at an appropriate icon.

    Unfortunately, if they put a bad icon registration in that registry value, the result is a bunch of blank space since the control panel is trying to reserve space for a bogus icon.

    The format of the icon registration is a filename, optionally followed by a comma and a decimal number.

    C:\full\path\to\icon\file.dll
    C:\full\path\to\icon\file.dll,123
    

    Since this is not a command line, quotation marks are not necessary (although they are tolerated). Furthermore, the number can be any value except for -1. Why is -1 forbidden? Because the ExtractIcon function treats the value -1 specially.

    If the icon file does not exist in the icon file, or if the icon number is -1, then the icon specification is invalid and the Add or Remove Programs control panel will reserve an odd amount of space for an icon that doesn't exist.

    Perhaps the Add or Remove Programs control panel should be more tolerant of invalid icon registrations? Or should it stay the way it is, adhering to the "Don't bend over backwards to fix buggy programs; force the program authors to fix their own bugs" policy that so many of my readers advocate? (Noting furthermore that refusing to accomodate invalid icon registrations makes it look like Add or Remove Programs is the buggy one.)

  • The Old New Thing

    The effect of SetCursor lasts only until the next SetCursor

    • 8 Comments

    Of course the effect of the SetCursor function for a thread lasts only until that thread changes the cursor to something else. Any moron knows that, right?

    The tricky part is that the SetCursor may come from an unexpected place.

    THe most common place people run into this is when they do something like this:

    // Put up the hourglass
    HCURSOR hcurPrev = SetCursor(hcurWait);
    ... do some processing ...
    // Restore the original cursor
    SetCursor(hcurPrev);
    

    This puts up the hourglass during the processing. But if you pump messages (or if a function you call pumps messages), then the hourglass will go away and return to the normal arrow.

    That's because when you pump messages, this opens the gates for messages like WM_NCHITTEST and WM_SETCURSOR. The latter in particular will typically result in the cursor changing, either to a cursor selected by the window itself or to the class cursor if the message makes it all the way to DefWindowProc.

    If you want to keep the hourglass up even while pumping messages, you need to let the window know that "If you are asked to set the cursor, please put up an hourglass instead of what you would normally display as the cursor." That window would then have to alter its WM_SETCURSOR handling to take this setting into account.

    case WM_SETCURSOR:
     if (ForceHourglass()) {
       SetCursor(hcurWait);
       return TRUE;
     }
     ...
    

    Note that forcing the hourglass is only the tip of the iceberg. Even though the cursor is an hourglass, the window is still active and can receive other message, such as mouse clicks and keypresses. If your program is not ready to receive new input during this phase, you need to detect this case and not go into some recursive state if the user, say, impatiently clicks the "Compute!" button while you are still computing.

  • The Old New Thing

    Understanding ternary raster operations

    • 9 Comments

    It's perfectly logical, which doesn't mean that it's easy to understand.

    A ternary raster operation describes how three boolean values should combine to form an output boolean. I was going to write up a description but found that the existing documentation pretty much covers it, so I refer you to that. In particular, look at the table that explains where PSo and DPSoo come from.

    Now, one question that has come up is "What is the algorithm for deriving the RPN raster operation description string from the operation index?" The answer is, "You stare at it and use your brain's power of pattern-matching." There is no single "correct" expression of an operation index as RPN. For example, "DPSoo" could have been written as "DSPoo" or "DSoPo" or as the equivalent but pointlessly esoteric "DPoDnPnSaao". All of these expressions have the same boolean truth table.

  • The Old New Thing

    Boil first, then mash

    • 40 Comments

    Last year, British schoolchildren (ages six and seven) went to a farm and were baffled by the long, pointy, orange things. (Those who specialize in plant biology have a special term for these strange objects: carrots.)

    Their older siblings don't seem to be faring much better. A three-year study revealed that modern Scottish schoolchildren lacked skills such as understanding clothing care instructions, sewing a button, and knowing that you boil the potatoes before you mash them.

    Okay, but let's look at those clothing care instructions. Do you know what it means if you have a square with a solid circle inside and two underlines? What about a triangle with two diagonal stripes? I don't know either. (They mean "Tumble dry, gentle cycle, no heat" and "Use non-chlorine bleach as needed", respectively. More symbols deciphered here.)

    And while it's true that I can sew a button, I am unable to sew a straight line or with constant tension.

    I'm sure every generation bemoans lost skills. Our ancestors probably shook their heads in disappointment when their grandchildren demonstrated themselves unable to churn butter, wash clothes by hand, or operate a mangle.

    On the other hand, you really ought to know what a carrot looks like and that you boil first, then mash. I doubt carrots and mashed potatoes are going to go out of style any time soon.

  • The Old New Thing

    Why are DLLs unloaded in the "wrong" order?

    • 15 Comments

    When a program starts or when a DLL is loaded, the loader builds a dependency tree of all the DLLs referenced by that program/DLL, that DLL's dependents, and so on. It then determines the correct order in which to initialize those DLLs so that no DLL is initialized until after all the DLLs upon which it is dependent have been initialized. (Of course, if you have a circular dependency, then this falls apart. And as you well know, calling the LoadLibrary function or the LoadLibraryEx function from inside a DLL's DLL_PROCESS_ATTACH notification also messes up these dependency computations.)

    Similarly, when you unload a DLL or when the program terminates, the de-initialization occurs so that a DLL is de-initialized after all its dependents.

    But when you load a DLL manually, crucial information is lost: Namely that the DLL that is calling LoadLibrary depends on the DLL being loaded. Consequently, if A.DLL manually loads B.DLL, then there is no guarantee that A.DLL will be unloaded before B.DLL. This means, for example, that code like the following is not reliable:

    HSOMETHING g_hSomething;
    typedef HSOMETHING (WINAPI* GETSOMETHING)(void);
    typedef void (WINAPI* FREESOMETHING)(HSOMETHING);
    
    GETSOMETHING GetSomething;
    FREESOMETHING FreeSomething;
    
    // Ignoring race conditions for expository purposes
    void LoadB()
    {
     HINSTANCE hinstB = LoadLibrary(TEXT("B.DLL"));
     if (hinstB) {
      GetSomething = (GETSOMETHING)
              GetProcAddress(hinstB, "GetSomething");
      FreeSomething = (FREESOMETHING)
              FreeProcAddress(hinstB, "FreeSomething");
     }
    }
    
    // Ignoring race conditions for expository purposes
    HSOMETHING CacheSomethingFromB()
    {
     if (!g_hSomething &&
         GetSomething && FreeSomething) {
      g_hSomething = GetSomething();
     }
     return g_hSomething;
    }
    
    BOOL CALLBACK DllMain(HINSTANCE hinst,
          DWORD dwReason, LPVOID lpReserved)
    {
     switch (dwReason) {
     ...
     case DLL_PROCESS_DETACH:
      if (g_hSomething) {
       FreeSomething(g_hSomething); // oops
      }
      break;
     }
     return TRUE;
    }

    At the line marked "oops", there is no guarantee that B.DLL is still in memory because B.DLL does not appear in the dependency list of A.DLL, even though there is a runtime-generated dependency caused by the call to LoadLibrary.

    Why can't the loader keep track of this dynamic dependency? In other words when A.DLL calls LoadLibrary(TEXT("B.DLL")), why can't the loader automatically say "Okay, now A.DLL depends on B.DLL"?

    First of all, because as I've noted before, you can't trust the return address.

    Second, even if you could trust the return address, you still can't trust the return address. Consider:

    // A.DLL - same as before except for one line
    void LoadB()
    {
     HINSTANCE hinstB = MiddleFunction(TEXT("B.DLL"));
     if (hinstB) {
      GetSomething = (GETSOMETHING)
              GetProcAddress(hinstB, "GetSomething");
      FreeSomething = (FREESOMETHING)
              FreeProcAddress(hinstB, "FreeSomething");
     }
    }
    
    // MIDDLE.DLL
    HINSTANCE MiddleFunction(LPCTSTR pszDll)
    {
     return LoadLibrary(pszDll);
    }
    

    In this scenario, the load of B.DLL happens not directly from A.DLL, but rather through an intermediary (in this case, MiddleFunction). Even if you could trust the return address, the dependency would be assigned to MIDDLE.DLL instead of A.DLL.

    "What sort of crazy person would write a function like MiddleFunction?", you ask. This sort of intermediate function is common in helper/wrapper libraries or to provide additional lifetime management functionality (although it doesn't do it any more, though it used to).

    Third, there is the case of the GetModuleHandle function.

    void UseBIfAvailable()
    {
     HINSTANCE hinstB = GetModuleHandle(TEXT("B"));
     if (hinstB) {
      DOSOMETHING DoSomething = (DOSOMETHING)
              GetProcAddress(hinstB, "DoSomething");
      if (DoSomething) {
       DoSomething();
      }
     }
    }
    

    Should this call to GetModuleHandle create a dependency?

    Note also that there are dependencies among DLLs that go beyond just LoadLibrary. For example, if you pass a callback function pointer to another DLL, you have created a reverse dependency.

    A final note is that this sort of implicit dependency, as hard as it is to see as written above, is even worse once you toss global destructors into the mix.

    class SomethingHolder
    {
    public:
     SomethingHolder() : m_hSomething(NULL);
     ~SomethingHolder()
      { if (m_hSomething) FreeSomething(m_hSomething); }
     HSOMETHING m_hSomething;
    };
    
    SomethingHolder g_SomethingHolder;
    ...
    

    The DLL dependency is now hidden inside the SomethingHolder class, and when A.DLL unloads, g_SomethingHolder's destructor will run and try to talk to B.DLL. Hilarity ensues.

  • The Old New Thing

    I'd like to register my stolen car, please

    • 7 Comments

    On NPR's All Things Considered Robert Siegel reported on the curious status of legally stolen cars in Gaza.

    Gaza license plates can be red for official, green for taxis, and white for private vehicles. The lower the number on the red plates, the higher the position of the official. The number 30 designates a truck. All this is pretty conventional stuff for licence plates. But then...

    "And then the cars which are written in Arabic the letters M and F, those are the stolen cars. And then there is these plates with MHF, it is a stolen car, but working for the authorities. It is a stolen government car. There is also another kind, but it is the same plate, just the numbers are different. If the number starts with 25, then it is a stolen car, but it is allowed to work as a taxi."

    [Raymond is currently on vacation; this message was pre-recorded.]

Page 1 of 3 (25 items) 123