July, 2007

  • The Old New Thing

    Why is the limit of window handles per process 10,000?


    If your program runs haywire, you will find that it manages to create about 10,000 window manager objects and then the system won't let it have any more. Why stop at 10,000?

    The first answer is "If you have to ask, you're probably doing something wrong." Programs shouldn't be creating anywhere near ten thousands window manager objects in the first place.

    Furthermore, as we saw last time, the maximum number of window manager objects that can be created is around 32,700. Giving a program 10,000 is already a third of the total amount available. That's already pretty darned generous, if you ask me. Preventing a program from running away and consuming all of the window manager objects is an attempt to contain the damage caused by a runaway program. Even if a program goes haywire, there's still around 20,000 objects available for the other programs to use. One of them might even be Task Manager, which the user fired up in order to kill the runaway program.

  • The Old New Thing

    The real cost of compatibility is not in the hacks; the hacks are small potatoes


    Commenter Myron A. Semack asks how much faster Windows would be if you took out the backward compatibility stuff. Myron is so anxious about this that he asked the question a second time. Asking a question twice typically counts as a reason not to answer it, but since I had already written up the answer, I figured I'd post it anyway. Oh great, and now he asked it a third time. Myron is so lucky I already wrote up the answer, because if I hadn't I would've just skipped the topic altogether. I don't respond well to nagging.

    The answer is, "Not much, really."

    Because the real cost of compatibility is not in the hacks. The hacks are small potatoes. Most hacks are just a few lines of code (sometimes as few as zero), so the impact on performance is fairly low. Consider a compatibility hack for programs that mess up IUnknown::QueryInterface:

    ITargetInterface *pti = NULL;
    HRESULT hr = pobj->QueryInterface(
                     IID_ITargetInterface, (void**)&pti);
    if (SUCCEEDED(hr) && !pti) hr = E_FAIL;

    The compatibility hack here was just two lines of code. One to set the pti variable to NULL and another to check for a common application error and work around it. The incremental cost of this is negligible.

    Here's an example of a hack that takes zero lines of code:

    HINSTANCE ShellExecute(...)
     return (HINSTANCE)42;

    I count this as zero lines of code because the function has to return something. You may as well return a carefully-crafted value chosen for compatibility. The incremental cost of this is zero.

    No, the real cost of compatibility is in the design.

    If you're going to design a feature that enhances the window manager in some way, you have to think about how existing programs are going to react to your feature. These are programs that predate your feature and naturally know nothing about it. Does your feature alter the message order? Does it introduce a new point of re-entrancy? Does it cause a function to begin dispatching messages that previously did not? You may be forced to design your feature differently in order to accommodate these concerns. These issues aren't things you can "take out"; they are inherently part of the feature design.

    Consider for example color NTSC. (Videophiles like to say that NTSC stands for "never twice the same color.")

    The NTSC color model is backward compatible with the existing system for black-and-white television. How much cheaper would your color television be if you could take out the backward compatibility circuitry? That question misses the point. The backward compatibility is in the design of the NTSC color signal. It's not a circuit board (or, to be more historically accurate, a set of vacuum tubes) that you can pull out. You can't "take out" the compatibility stuff from your television set. The compatibility is fundamentally part of the way the NTSC color signal works.

  • The Old New Thing

    Don't just grab the foreground window and host UI on it


    I was asked to look at an application compatibility bug in a program from a major vendor. But that's actually irrelevant; what I'm writing about today has nothing to do with application compatibility. That's just what drew my attention to the program in the first place.

    At some point during the install, the setup program encountered an error and wanted to display an error message. When it called DialogBox to display the error message, it didn't pass the setup program's main window as the hwndParent. Instead it passed GetForegroundWindow(). They chose the wrong owner for modal UI. (I've also seen people try GetTopWindow(0).)

    It so happened that the foreground window was Task Manager, since I had switched to Task Manager to look at various statistics of their installer as it ran.

    I hope you can see where this is going.

    They passed Task Manager as their modal owner, and since modal dialog boxes disable the owner, they ended up disabling Task Manager. (Meanwhile, their main setup program remained enabled, so I could have clicked on the Cancel button if I wanted to, which would have led to the "stack without support" problem.)

    Now I can't terminate their broken setup program from Task Manager since they inadvertently disabled Task Manager.

    But why did the programmers choose to use the foreground window anyway?

    One possibility is the programmer's version of the politician's fallacy.

    • We must pass a window.
    • The foreground window is a window.
    • Therefore, we must pass it.

    Another possibility is that they did this on purpose in order to ensure that their error message steals focus. Because their program is the most important program in the history of mankind.

    Unfortunately, I see this a lot. People who think their program is so important that they will abuse the rest of the system in order to get what they want instead of just waiting their turn. Of course, these people also fail to realize that setting a window as the owner for UI creates its own problems. As already noted, you disabled a random program. What's more, you've now attached the two input queues and tied your fates together. If the program that owns the foreground window stops responding to messages, then your program will also stop responding to messages.

    But primarily it's just rudeness. You took somebody else's window and started acting as if you owned the place. It's like looking up somebody's address in the phone book and using it as your own. That's not your house, and that's not your window.

  • The Old New Thing

    If you want a modeless dialog, then create a modeless dialog already


    Somebody I'd never heard of sent me email with a general programming question. (This is one of the top ways of instantly making me not interested in talking to you.)

    I have a modal dialog box which I hide with ShowWindow(SW_HIDE), and I want to enable the parent window so the parent window will get focus and keyboard input. Is this possible? If so, how do I do it? MSDN says "Although a dialog box procedure could potentially enable the owner window at any time, enabling the owner defeats the purpose of the modal dialog box and is not recommended." How do I go about enabling the parent window?

    This is like asking for a cheeseburger and then trying to peel every last bit of cheese off the patty to turn it into a plain hamburger. Since the cheese has already melted into the crevices of the burger, you're going to be picking off tiny flecks of cheese for a long time. If you want a modeless dialog box, then just make a modeless dialog box. (This is what MSDN is trying to tell you in slightly more polite language.)

  • The Old New Thing

    How are window manager handles determined in 16-bit Windows and Windows 95?


    (Prefatory remark: The following information is of the "behind the scenes" nature and does not constitute formal documentation. Actually, nothing I write counts as formal documentation, so I shouldn't have needed to write that, but people sometimes intentionally play stupid and interpret all forms of the future tense as if I were making some sort of "guarantee" on behalf of Microsoft Corporation. I assure you that I have no such authority! It's times like that that I'm tempted to just give up writing.)

    Let's start with 16-bit window handles. Those are simple: They are just pointers into the window manager's data segment, cast to the HWND data type. Since the window manager had a single 64KB data segment, all of these pointers were 16-bit values.

    In Windows 95, the window manager moved several categories of objects out of the default data segment into their own custom heaps. (And those were 32-bit heaps so they could be bigger than 64KB.) Window classes, menus, and windows each got their own "big" heap. There may have been other categories of objects that moved out of the default data segment, but those are the ones I remember.

    But since Windows 95 still had to support 16-bit programs, it needed a way to return 16-bit window handles back to those programs. To do this, the window manager allocated the memory in the 32-bit heap as "movable", which as we learned some time ago isn't actually movable. The purpose of allocating it as movable memory was to get that local memory handle, the HLOCAL.

    No, wait, but that doesn't actually solve the problem, because a local handle in a 32-bit heap is still a 32-bit value. How do we get a 16-bit value out of that?

    When the window manager created the 32-bit heap, it asked the 32-bit heap manager very nicely if it could give back 16-bit handles instead of 32-bit handles. The heap manager did this by pre-allocating a 64KB block of memory and allocating its handles out of that memory block, using the offset into the block as the handle.

    Since each entry in the handle table is four bytes (a 32-bit pointer), the 64KB handle table can hold up to 16384 entries. This is why the documentation for CreateWindowEx includes the following remark:

    Windows 95/98/Me: The system can support a maximum of 16,384 window handles.

    Actually, it was a little bit less than that because some of the entries were lost to bookkeeping overhead. For example, the handle value of zero could not be used because that would be confused with NULL.

    Now, you may have asked, "Well, if all the window handles are multiples of four, why not divide by four and then you can get the full range of 65535 window handles?"

    Well, remember that Windows 3.1 could handle only around 700 windows. Increasing this to 16,384 was enormous progress already. I mean, it's more than 23 times as much as what you had before. A hundred windows was already considered excessive at the time, so the window manager already could accommodate 163 abusive, badly-written programs. There's really no reason to bump that up to 655 badly-written programs. That'd just be encouraging programs to behave badly.

    Both the 16-bit Windows technique and the Windows 95 technique did suffer from the problem of handle re-use. When a window is destroyed, its memory is freed (as well as its handle on Windows 95). When a new window is created, there's a good chance that the memory or handle will get re-used, and consequently the numerical value of the window handle once again becomes valid, but refers to a different window.

    It so happens that boatloads of programs (and "boatloads" is a technical term) contain bugs where they use window handles after the window has been destroyed. When a window handle is re-used, that program sends a message to the window it thinks is still there, but instead it sends the message to a completely unrelated window. This doesn't bode well for the program, and it usually doesn't bode well for the new window that received the message by mistake either.

    Next time, we'll look at how the Windows NT folks addressed this problem of window handle re-use.

    Nitpicker's corner

    "Boatloads" is not a technical term. That was a joke.

    The initial version of this article accidentally omitted the word "not" from the opening sentence. Kudos to the people who were able to exercise their brain and figure this out from context instead of robotically taking everything at face value. There may be hope for the world yet.

  • The Old New Thing

    Tips for doing the Seattle to Portland (STP) in two days: What I learned in 2007


    Two weekends ago, I participated in the 28th annual Seattle to Portland bicycle ride, wherein I joined up with 8999 of my closest friends for a friendly ride through western Washington and Oregon. Earlier this year I provided tongue-in-cheek bad advice for preparing for STP. Today I restore the balance with proper advice.

    This was my first STP, and I was somewhat apprehensive over whether I was up to the task, since I had never ridden more than 60 miles in a day prior to this. Here are some notes I'm recording for the benefit of future generations, since I couldn't find much in the way of this type of advice on the Web. (Note: These remarks apply to two-day riders. I refer one-day riders to Eric Gunnerson's STP 2006 blog. Obligatory disclaimers: Every person is different. These tips may not work for you. Consult your doctor before starting a major exercise program. Objects in mirror are closer than they appear.)

    It's not as hard as you think. Sure, it may be longer than you've ever ridden before, but if you're like me, your training rides were 50 to 60 miles without any significant break. In that sense, STP is easier than a training ride, since STP has a rest area every 15 miles or so. It's not really two 100-mile rides; it's more like a dozen short rides.

    It's an endurance ride, not a race. If you're like me, your usual bicycling is done at a decent clip, 16–18 mph on level ground. When I told one of my colleagues that I was wiped out from a 60-mile ride and he learned that I went 17–19 mph on the flats, he was horrified. Riding fast is the wrong strategy for STP. Keep it down to 13–14 mph. If you find yourself exerting, then you're going too fast, because you won't be able to keep it up the whole way. It turns out that if you're used to going 16–18 mph all the time, then going 14 mph takes almost no effort at all. You can do it all day without even breathing hard. And that's the idea.

    Watch your pit time. As with auto racing, the amount of time you spend in pit stops is important. Although our group managed an average speed of around 13.3 mph on the first day, we also hung around for over four hours at the various rest stops, stretching that first 125-mile leg into a grueling 13½-hour day. (That's right, we had more downtime than this guy, and at the end of the day, he was in Portland!) Some of our extended stops were triggered by mechanical troubles, but others were just dawdling, or at least they felt like dawdling to me; perhaps the others in the group really needed the break time. If you've been moderating your pace per the previous tip, you might very well not be tired at all and need only stop for a bathroom break and a water refill. Besides, if you stop for too long, your muscles may start to stiffen. A lot of short breaks is better than a small number of long ones.

    You don't have to stop at every mini-stop. Stop at the major stops, but if you feel fine when you reach a mini-stop and don't need a water refill, then just keep going. This is just a special case of the previous tip, where the pit time is zero. My colleague who had a suboptimal training regimen told me that he had to take frequent breaks, and when he got back on the road, he found himself passing the same person each time. (Said person was easy to recognize because he was riding a Razor Scooter. I experienced the same thing in reverse on this ride: I would recognize the same person passing me over and over.) It's the tortoise vs hare. If you go too fast, you'll need to take a long break to recover, and you end up going no faster overall than the person who goes slower but takes shorter breaks.

    One slow leg isn't the end of the world. Even if it looks like you got dropped by the rest of your group, it's not as bad as it looks. At one point, a subset of our group wanted to do a fast leg, and off they went. I stayed back with the rest of the group, but lost track of them in the crowd. I assumed the others were ahead of me, so I picked up the pace a bit and was able to sustain 17–18 mph without any real effort. (It's easy to go fast on Highway 507 between Spanaway and Roy.) Eventually, I caught up with the fast group and realized that I had left the main group behind. I pulled over and waited for the others to catch up. They did, six minutes later.

    This was over about two thirds of the leg, so the total difference between a fast pace and a relaxed pace on a single leg is just ten minutes. Being ten minutes late isn't the end of the world.

    It's not your legs that will hurt. Your legs will be fine, since you won't be pushing yourself much at all. What will hurt are your hands and butt. (And for me on the first day, my toes, since I didn't lace up well and my toes rubbed against the inside of my shoe.) Vary your hand and seat position to shift the weight to different parts of your body. At the start of the day, you applied butt cream, but since you don't know where it's going to hurt yet, you kind of covered everything and hoped for the best. Towards the end of the first day, you'll start to figure out where you should've applied it. Pull over and apply butt cream to the trouble spots. Yes, you may look like you're getting a bit too familiar with yourself standing by the side of the road with your hand in your pants, but everybody else going past you will say "ah, reapplying butt cream" and not "what a pervert", because by this point, they're probably thinking of doing the same thing. (Note: Attempt only along isolated country roads. In populated areas, seek a private place like a rest room.)

    Get your bike ready early. Don't think you can get your bike tuned up with only a month before STP; the local bike shops will be hammered with people who got the same brilliant idea. Also, don't make major changes to your set-up, like a new saddle, a new handlebar configuration, or (heaven forfend) a new bike! It takes a while to adapt to a new configuration, and you don't want to ride STP while you're still adjusting to the new saddle.

    Other quick tips:

    • Bring your bicycle mirror. Saves you from having to turn your head to see what's behind you. Use it to make sure you're not pulling away from the rest of your group, to wait for a break in traffic so you can pass somebody, or to spot the car approaching from behind. If you turn your head, you stop riding straight, and you slow down. I broke my mirror at the start of Day Two, and life was significantly more difficult.
    • Bring a bell. You can ding your bell to announce your presence instead of having to shout "On your left!" all the time. You can also ring your bell to celebrate crossing the finish line.
    • Remove your bicycle computer when you finish so it doesn't count your post-race puttering.
    • If you are into performance statistics, you can bring a digital camera (or use the one built into your phone) and take a picture as you arrive at each stop, and again as you leave. The timestamp on the photo combined with the mileage on the route map will let you compute your average speed for each leg, as well as calculating how much time you spent resting. If you don't trust the mileage on the route map, you can take a picture of your bike computer to record split statistics for posterity. If your phone doesn't have a camera, you can leave yourself a voicemail message saying, for example, "Arrived at Seward Park, mileage 10.8." The voicemail system will automatically timestamp the message.
    • If you're a guy and you just need to make a tinkle, then check out the line for the "men-only" portable toilets; it is often shorter.

    You may not need (but since you can toss it in your luggage, it probably won't hurt to bring anyway):

    • Book, deck of cards, or other light entertainment. I brought a book and didn't even crack it open, I was so tired. (On top of that, we overnighted in a high school, so there were plenty of books in the library to choose from.)
    • Disposable fork, knife, spoon, cup. The people who serve you dinner will immediately notice when they run out of plates, but they are less likely to notice right away that they ran out of utensils and cups. If you bring your own, you won't be stuck standing there for five minutes with a plate of food and no way to eat it.
    • Pillowcase. Wadded-up clothes + pillowcase = pillow.
    • Sleeping mask. You may want to go to sleep before Nature decides to turn off the lights.

    Okay, those are the tips. Trip report begins next time.

    Nitpicker's corner: Numbers have been rounded for simplicity of presentation.

  • The Old New Thing

    Not my finest hour: Driving a manual transmission


    Sara Ford messed up her leg last month; you can see her on ScooterCam-1 at TechEd 2007. The best part of the name ScooterCam-1 isn't the ScooterCam; it's the 1. That just gives it that extra little kitchy space technology feel that turns a good name into a great name.

    Anyway, she asked around if somebody with a car with an automatic transmission would be willing to swap with her manual transmission car until her leg got better. To me, driving a stick shift is like changing a flat tire: It's something that everybody should know how to do, but it's not something that anybody is expected to enjoy doing. I offered my car and (long boring part of story deleted) we met at her place and exchanged keys. I got into her car, backed out of the parking space, and... stalled right in front of her.

    Great job there. I really know how to instill confidence.

  • The Old New Thing

    QueryPerformanceCounter is not a source for unique identifiers


    This article happened to catch my eye:

    I needed to generate some unique number in my application. I could use GUID, but it was too large for me (I need to keep lots of unique identifiers). I found something like this:

    private static extern int
        QueryPerformanceFrequency(ref System.Int64 frequency);
    private static extern int
        QueryPerformanceCounter(ref System.Int64 performanceCount);
    public static long GenerateUniqueId()
          System.Int64 id = 0;
          QueryPerformanceFrequency(ref id);
          QueryPerformanceCounter(ref id);
          return id;

    This code generates Int64 (long) unique number (at least I hope it is unique). The uniqueness is in the scope of process. So two processes can generate the same number, but it should be always unique in a single process (I am not sure about two threads invoking the same GenerateUniqueId() method.

    QueryPerformanceCounter retrieves the current value of the high-resolution performance counter, but there is no guarantee that every call to the function will return a different number.

    The frequency of the high-resolution performance counter is determined by the HAL. You might think that the RDTSC instruction would be perfect for this purpose, since it returns the number of CPU clock ticks, a value that always increases at a very high rate. But there are many problems with RDTSC. For example, variable-speed processors mean that the rate at which CPU clock elapse varies over time. A million clock ticks might take one millisecond when the computer is running on wall power, but two milliseconds when running on battery power.

    If the HAL can't use RDTSC, what does it use instead? Well, as I said, it's up to the HAL to find something suitable. Older motherboards have to make do with the programmable interval timer which runs at 1,193,182 ticks per second (approximately 0.8 microseconds per tick). Newer motherboards can use the ACPI timer which runs at 3,579,545 ticks per second (approximately 0.3 microseconds per tick).

    One of the machines in my office uses the ACPI timer for its high-resolution performance counter, so I threw together a quick program to see how close I can get to outracing the ACPI timer by calling QueryPerformanceCounter in rapid succession. With a 1.80GHz processor, the computer manages to call QueryPerformanceCounter quickly enough that only four ticks of the ACPI timer elapse between consecutive calls. We're getting into shouting range of being able to call QueryPerformanceCounter twice and getting the same value back from the ACPI timer. Of course, if the computer had been using the programmable interval timer, it would have been within spitting distance, and upgrading to a 3GHz processor would have taken us over the top.

    In other words, you may be lucky today that your CPU isn't fast enough to call QueryPerformanceCounter twice and get the same value back, but it sure looks like we're threatening to get there soon.

    Then again, all this back-of-the-envelope calculation is superfluous. All you need is a machine with multiple processors. Get two of the processors to call QueryPerformanceCounter at the same time (or nearly so), and they'll get the same timer value back.

    If you want to generate unique 64-bit values, you can just use InterlockedIncrement64.

  • The Old New Thing

    How are window manager handles determined in Windows NT?


    We left off our story last time by raising the problem of programs that send messages to windows that have already been destroyed and how window handle re-use exacerbates the problem. Although this is clearly a bug in the programs that use window handles after destroying the window, the problem is so widespread that the window manager folks in Windows NT decided to take a more proactive approach.

    (Reiterating in case you couldn't figure it out: This entry goes "behind the scenes". Behavior described here falls into the category of "implementation detail" and is subject to change at any time.)

    In Windows NT, the 32-bit HWND was broken into two parts. The bottom 16 bits acted like the window handle of Windows 95: It was an index into a table. But whereas Windows 95 left the upper 16 bits zero for compatibility with 16-bit programs, Windows NT used the top bits as a "uniquifier".

    For example, the first time entry 0x0124 in the table was used, the handle corresponding to that entry was 0x00010124. After that window is destroyed and a new one created in slot 0x0124, the handle for the new entry is 0x00020124. Each time the entry is re-used, the uniquifier increments by one.

    But what about those 16-bit programs?

    When a 16-bit program used a window handle, the window manager would just use that value as the index into the table. There was no uniquifier, so the window manager just ran with whatever was in that slot. After all, the uniquifier is wasn't essential to correct operation of the window manager; it was added merely to cover for buggy programs. Consequently, 16-bit programs were just as susceptible to the "handle re-use problem" as they were in Windows 3.1 and Windows 95. Only 32-bit programs got the extra protection.

    But since the window handle was an index, the full 16-bit range of window handles became available (minus special values like NULL), for a theoretical maximum of around 65,000 windows. Well, except that the actual theoretical maximum is is half that, around 32,700 windows. Why did we lose half of the window handles? Because there are programs out there that assume that window handles are always even numbers!

    Next time, we'll look at the magical 10,000 per-process handle limit.

  • The Old New Thing

    Raymond's tips on selecting a memorable greeting card


    My favorite greeting card is one I got from :: Wendy ::. It was a Christmas card with a picture of kids running around while snow fell all around them. Wendy added the message, "Watch out for that 'rogue' snowball from midfield..." I still have that card on my windowsill years later.

    My recommendation for choosing a memorable greeting card is to go to the wrong aisle in the greeting card store. I always start with the "blank cards" aisle since you can look for a picture that sets an interesting scene (like Wendy's snowball fight) and then add a short message inside.

    Last year, I went to the Sympathy aisle to get a birthday card for a friend who was trying not to show his concern over turning 30. After some searching, I found the perfect card. It was your usual sappy sympathy card with flowers and birds on the cover and some text about peace and comfort. And on the inside, it simply said, "Hope you feel better soon." I added, "Happy birthday."

Page 1 of 5 (43 items) 12345