November, 2008

  • The Old New Thing

    Sensor development kits were flying off the shelves


    After the Sensor and Location Platform PDC presentation, people were stopping by the booth and grabbing sensor hardware and development kits like they were candy. Then again, to geeks, this stuff is candy.

    (And technically, they weren't flying off shelves. They were flying out of bins. Well, and technically they weren't flying either.)

    Other notes from the last day of the 2008 PDC:

    • PDC content honcho Mike Swanson announced that PDC 2008 sessions and keynotes are now up for download, and will be available indefinitely. Available in Silverlight for live streaming, or in iTunes format, WMV, high-quality WMV (this one includes video of the speakers), and Zune for offline viewing. In response, Nigel Parker created the PDC session firehose.
    • Sometimes, when you make the sign too big, nobody can see it. Someone came up to me asking where the Azure booth was. I pointed at the insanely huge banner. "Probably somewhere over there."
    • The reason why room 406A is on the third floor but you press 2 to get there: The room numbers at the convention center are numbered based on room clusters. The 100's in one cluster, the 400's are in another cluster, etc. The "4" in 406 doesn't mean fourth floor; it means fourth cluster. And why do you push 2 to get to the third floor? Because Level 1 is a split level which occupies two floors. Therefore, in the elevator, there are two buttons to get to Level 1, depending on whether you want the upper half or lower half. Cluster 4 is on Level 2, so the button for that is 3.
    • A video of carrots. If you had come to my PDC talk, you'd understand. (The carrot joke elicited a mild chuckle from the main room, but I'm told that in the overflow room it totally killed. I'm not sure what that says about people in the overflow room. It really wasn't that great a joke, people.)
    • At Ask the Experts, I started out at one of the Windows 7 tables, and we speculated as to what the most obscure technical question would be. One of my colleagues suggested that it would be a BitLocker question. And what would you know, the very first person to come to our table had a BitLocker question. Later in the evening, I moved to another Windows 7 table and posed the same question. At the second table, we figured that the most obscure technical question would be about the dispatcher spinlock. About fifteen minutes later, somebody stopped by with a question about the dispatcher spinlock. I decided to stop playing this game. It was too much like a Twilight Zone episode.
    • One attendee was shy about speaking English (not being a native speaker) and came up with a clever solution: The attendee walked up to our table, said, "I have some questions, can you read them?" and then opened a laptop computer where a Word document awaited with the four questions already typed up in a large font so everybody could read them. We answered the questions, and everybody seemed pleasantly surprised with how well this approach worked.
    • At the end of the conference the Channel 9 folks held a giveaway for their giant beanbags. Like anybody would be able to take a giant beanbag home on the airplane. (Second prize: Two beanbags!)
    • James Senior has the PDC by the numbers. I have trouble believing the number of bananas consumed, though. Twenty-five bananas in four days? Perhaps we should start calling James Monkey Boy.
  • The Old New Thing

    The cost-benefit analysis of bitfields for a collection of booleans


    Consider a class with a bunch of BOOL members:

    // no nitpicking over BOOL vs bool allowed
    class Pear {
     BOOL m_peeled;
     BOOL m_sliced;
     BOOL m_pitted;
     BOOL m_rotten;

    You might be tempted to convert the BOOL fields into bitfields:

    class Pear {
     BOOL m_peeled:1;
     BOOL m_sliced:1;
     BOOL m_pitted:1;
     BOOL m_rotten:1;

    Since a BOOL is typedef'd as INT (which on Windows platforms is a signed 32-bit integer), this takes sixteen bytes and packs them into one. That's a 93% savings! Who could complain about that?

    How much did that savings cost you, and how much did you save anyway?

    Let's look at the cost of that savings. Code that updated the plain BOOL m_sliced member could do it by simply storing the result into the member. Since it was a normal field, this could be accomplished directly:

      mov [ebx+01Ch], eax ; m_sliced = sliced

    On the other hand, when it's a bitfield, updating it becomes trickier:

      add eax, eax        ; shift "sliced" into the correct position
      xor eax, [ebx+01Ch] ; merge "sliced" with other bits
      and eax, 2
      xor [ebx+01Ch], eax ; store the new bitfield

    Exercise: Figure out how the above trick works.

    Converting a BOOL to a single-bit field saved three bytes of data but cost you eight bytes of code when the member is assigned a non-constant value. Similarly, extracting the value gets more expensive. What used to be

     push [ebx+01Ch]      ; m_sliced
     call _Something@4    ; Something(m_sliced);


     mov  ecx, [ebx+01Ch] ; load bitfield value
     shl  ecx, 30         ; put bit at top
     sar  ecx, 31         ; move down and sign extend
     push ecx
     call _Something@4    ; Something(m_sliced);

    The bitfield version is bigger by nine bytes.

    Let's sit down and do some arithmetic. Suppose each of these bitfielded fields is accessed six times in your code, three times for writing and three times for reading. The cost in code growth is approximately 100 bytes. It won't be exactly 102 bytes because the optimizer may be able to take advantage of values already in registers for some operations, and the additional instructions may have hidden costs in terms of reduced register flexibility. The actual difference may be more, it may be less, but for a back-of-the-envelope calculation let's call it 100. Meanwhile, the memory savings was 15 byte per class. Therefore, the breakeven point is seven. If your program creates fewer than seven instances of this class, then the code cost exceeds the data savings: Your memory optimization was a memory de-optimization.

    Even if you manage to come out ahead in the accounting ledger, it may be a win of just a few hundred bytes. That's an awful lot of extra hassle to save a few hundred bytes. All somebody has to do is add an icon to a dialog box and your savings will vanished.

    When I see people making these sorts of micro-optimizations, sometimes I'll ask them, "How many instances of this class does the program create?" and sometimes the response will be, "Oh, maybe a half dozen. Why do you ask?"

    But wait, there's more. Packing all these members into a bitfield has other costs. You lose the ability to set a hardware write breakpoint on a specific bit, since hardware breakpoints are done at the byte level (at a minimum). You also lose atomicity: An update to m_sliced will interfere with a simultaneous update to m_peeled on another thread, since the update process merges the two values and stores the result non-atomically. (Note that you also lose atomicity if you had used a byte-sized bool instead of a 32-bit BOOL because some CPU architectures such as the original Alpha AXP cannot access memory in units smaller than a DWORD.)

    These are just a few things to take into account when considering whether you should change your fields to bitfields. Sure, bitfields save data memory, but you have to balance it against the cost in code size, debuggability, and reduced multithreading. If your class is going to be instantiated only a few times (and by "a few" I'm thinking less than a few thousand times), then these costs most likely exceed the savings.

  • The Old New Thing

    The great thing about priorities is that you can always go one higher


    The phenomenon I call priority inflation has spread to product planning documents as well. Back in the old days, there were three priority levels:

    • Priority 1: must have. If you don't accomplish a priority 1 item, you may as well just cancel the project because it ain't shipping.
    • Priority 2: should have. If you don't accomplish a priority 2 item, the product is significantly weaker, but you can still ship it.
    • Priority 3: nice to have. If you don't accomplish a priority 3 item, it's not quite as awesome as it could have been, but it's still a good product.

    Over the past few years, I've seen a shift in the labelling of priorities in planning documents. A new priority has been introduced: Priority Zero. Nobody has explained to me what Priority 0 means, but I assume somebody invented it to emphasize that the feature is even more critical than priority 1. Mind you, I'm not sure what could be more important to a project than "If we don't do this, we're all fired." Maybe "If we don't do this, the earth will explode."

    As you might expect, priority inflation has a trickle-down effect. People whose features had been assigned priority 1 said, "Hey, how come my feature isn't priority 0? It's just as critical as that other guy's feature." Soon, everything that was priority 1 got reclassified as priority 0. Nature abhors a vacuum, so all the priority 2 items got reclassified as priority 1, and the priority 3 items got reclassified as priority 2.

    In the end, nothing changed aside from the names on the buckets. It's been years since I've seen a planning document with any priority 3 items. It's all zero, one, and two now.

    Wait, I lied. The meaning of the last bucket (the former priority 3, now named priority 2) has changed. It used to be things that would be nice to have, but now it appears to be used for something other people suggested which I didn't think was important, but I didn't want to be mean and reject it outright, so I'm listing it here to make those people feel better and showing that their "voice was heard," but don't kid yourself; we're not going to do it. In other words, priority 2 means No.

    I give it three years before somebody decides that an issue is even more critical than priority 0 and labels it Priority −1.

    Epilogue: After I originally wrote this entry, I've learned that some teams have indeed come up with a priority level even more important than Priority 0. It's called Priority Now.

  • The Old New Thing

    Just because a method is called Refresh doesn't mean that it refreshes what you want


    Here's a question from a customer:

    I made some changes related to my shell extension [details omitted], but the changes don't show up in the Explorer window when I refresh it. Any suggestions on how to solve this problem?

    When we asked how they were refreshing the Explorer window, we were expecting something like pressing F5 or calling SHChangeNotify with SHCNE_UPDATEDIR, or maybe calling IShellView::Refresh or possibly even calling WebBrowser.Refresh from script. But we definitely didn't expect this response:

    I'm invoking the Process.Refresh() method from the System.Diagnostics namespace.

    Just because a method is called Refresh doesn't mean that it refreshes what you want. I think this is somebody who just saw a method name, perhaps inspired by IntelliSense and— Boom! You have it!—assumed it did what was needed without actually reading the documentation to check. But you don't even need to read the documentation to know that Process.Refresh has no chance of working.

    Since it's a method on the Process class, the method is applicable to all processes. But certainly there is no generic way to tell a process to refresh. This magical Refresh method would have to know how to refresh Explorer windows, Firefox web pages, iTunes mp3 tags... And what would it even mean to refresh, say, a Notepad window? Does that mean to throw away all changes and reload the original document?

    How do you know that there is no generic way to tell a process to refresh? Well, for one thing, a single process like Explorer can be viewing multiple objects that can be refreshed; which one are you refreshing? Second, when you write your own program, how do you implement refresh? Do you respond to some standard system Refresh message? Or do you just add a Refresh option to your program's main menu and give it some arbitrary command ID? If there's not even a standard way to refresh your program's window, then how can there be a standard way to refresh all program windows?

    In this specific case, the Process.Refresh method refreshes the Process object's internal cache of process properties. It doesn't actually do anything to the process itself. How could it?

    It's like thinking that the Matrix.Rotate method rotates the entries in a matrix.


    Actually, I'm scared by this customer's question for another reason: The fact that they even mentioned Process.Refresh suggests to me that they wrote their shell extension in managed code, which we already know is strongly disrecommended.

  • The Old New Thing

    Microspeak: Represent


    The more conventional definition of represent is along the lines of "to act as a proxy for". An attorney represents his or her client in court. Your legislator represents you in the assembly. A token on a board represents your position in the game.

    At Microsoft, the word represent takes on a stranger meaning. Here are some usages inspired by actual sentences:

    • We need someone to represent this bug at the morning meeting.
    • Can somebody represent which OS versions are affected by this issue?

    In the first case, I'm guessing that the word represent means "to act as an advocate for" or possibly just "to serve as a source of information on".

    In the second case, it appears that the word represent just means "tell us in an official capacity".

    These new senses may be influenced by rap lyrics and what is genteelly referred to as urban culture.

    If somebody actually knows what the word "represent" means at Microsoft, please add a comment. I asked the person who made the second statement for a definition, but what came back was merely the "act as a proxy for" definition, which clearly doesn't make sense. (How can a person act as a proxy for a list of operating systems?)

  • The Old New Thing

    Rachmaninov had big hands: An illustration


    Rachmaninov's Prelude in C# minor, Op. 3, No. 2, performed as it is written, by classical music comedy duo Igudesman & Joo.

    I tried to learn that piece once. I didn't last long.

  • The Old New Thing

    Self-esteem gone overboard: The perils of a global namespace


    There are items with overly generic names. HANDLE, CloseHandle, GetObject, DIFFERENCE, query.exe. But their functionality doesn't live up to their name. HANDLE refers only to kernel handles, CloseHandle can only close kernel handles, GetObject only gets information about GDI objects, DIFFERENCE applies only to the numerical difference between group resources and standalone resources, and query.exe only queries information about Terminal Services Remote Desktop Services.

    Why do functions that operate only inside a specific realm have names that suggest a broader scope?

    Self-esteem gone bad.

    You're on the kernel team. You have a handle manager. What should you call your handles? Well, since they're handles, just call them HANDLE. Because that's what they are, right? And naturally the function that closes HANDLEs should be called CloseHandle. Sure, there are other types of handles out there, but they don't exist in your world. Your world is the kernel, and in the kernel world, you can call them HANDLEs and everybody will know that you're talking about kernel handles because that's why you're in the kernel in the first place! Why would somebody pass a handle to a non-kernel object to a kernel function? That makes no sense!

    Similarly, the GDI folks came up with their own object system, and naturally the way you get information about an object is to call GetObject. There's no confusion here, right? I mean, this is GDI, after all. What other types of objects are there?

    The Terminal Services Remote Desktop Services folks thought the same thing when they created their query.exe program. Hey, this is a computer set up to run Remote Desktop Services; of course you want to query information about Remote Desktop Services.

    Of course, when your symbol exists in a shared namespace, the context of your naming decision becomes lost, and your generic-sounding function name (which worked just great for generic operations in the world in which it was created) ends up carrying more meaning than you originally intended.

    Commenter Sean W. tries to explains that Unix doesn't have this problem. "A Unix-flavored close() system call can close any file descriptor." This explanation ends up being its own counter-argument. When you say that it can close any file descriptor, you're admitting that it can't close anything. You can't use close() to close the objects opened by opendir() or dbm_open() or XtOpenDisplay.

    "Well, yeah, but it can close any file descriptor regardless of where it came from." And CloseHandle works the same way: It can close any kernel handle regardless of where it came from.

    Sean W. later clarified that "the scope of close() is the system kernel, so it's reasonable to expect that it applies to kernel data and no other data, whereas the scope of CloseHandle is all of Win32, including at least KERNEL/USER/GDI/ADVAPI." Um, actually, the scope of CloseHandle is also the kernel.

    And in the category of "suggesting things that are already done" goes this comment from Daniel, who suggests that the documentation explain which HANDLEs can be closed by CloseHandle. Actually, if you look at each function that creates a handle, it also tells you the function to use to close it. Not quite the same thing, but since you have to open something in order to close it, you'll find the information even sooner.

  • The Old New Thing

    You can't fight in here; this is the War Room!


    (Video clip for those who don't get the reference.)

    The term War Room is attributed to Winston Churchill, used to describe the underground meeting room where the war effort was directed. It is the "crucial meeting room where important decisions are made" sense of the term that is used at Microsoft.

    Many software products (or even just components of software products) have a meeting room designated as its War Room. Depending on the stage of the project, War Meetings might be held weekly, daily, or even several times a day. The meeting is run by the War Team, with representatives from all product components in attendance. Progress reports are given, the schedule is managed, critical bugs (or, if things are coming down to the wire, all bugs) are reviewed. For projects that are spread out over multiple locations, fancy teleconferencing equipment may be installed, and there may be clocks on the walls for time zones around the world.

    Since the same people tend to attend these meetings over and over, there's a good chance that you'll find long-standing inside jokes and other rituals. For example, in the Windows War Room, there are pieces of paper on the walls with various catch phrases printed on them. I have no idea what they mean, but they are obviously a source of amusement for the people who spend hour after hour in the War Room.

    If you're a programmer or tester, you don't have much direct interaction with the War Meetings unless you are responsible for a bug that has caught the attention of the War Team, at which point you are suddenly thrust into the frenzy. Before each War Meeting, your manager will want a status report, and if it's a new bug, you may even have to explain the situation in person. For example, suppose the project is in its final days, and a new bug just turned up. You may be called upon to describe the bug to the War Team, assess how risky the fix is and its impact on performance, application compatibility, and all that other information that the release management team will need to know in order to decide whether they are going to approve the fix. (Another phrase you may hear is taking a bug to War, which means bringing the bug before the War Team to request that they approve the fix for the product.)

    And now that you've learned all this vocabulary surrounding the term War Room, you have to forget it all, because the new term is Ship Room, since that is the room where decisions about shipping the product are made. (Though it seems that the security folks still call their meeting room the War Room.)

    Aside: The White House War Room was renamed the Situation Room in 1961.

    I don't know the reason for sure, but I suspect the old term is deprecated because it sounds too belligerent and uncooperative.

    (And happy Veteran's Day.)

    Bonus track: Inside the SQL 2008 Ship room. And no, the music is not part of the meeting.

  • The Old New Thing

    If everything is top priority, then nothing is top priority


    Last time, I mentioned that eventually everything is top priority. A similar topic is what I'm calling priority inflation, which takes more than one form.

    Today's priority inflation is the introduction of new "top priority" items. (Chris Becker has some thoughts on this topic as well.)

    "XYZ is very important to our project. Please make it your top priority." A few weeks later, "ABC is very important to our project. It should take priority over all other issues." When this happens, I like to ask, "Is this even more important than XYZ?" I've done it so much that my management has changed the way it introduces new top priorities: Instead of just saying "Please make ABC your top priority," they list out all the existing top priorities... in priority order.

    ABC is very important to our project. There are just a handful of ABC issues remaining, and we would like to close them out by the end of the month. If you have an ABC issue, please make it your top priority. To summarize:

    1. ABC issues.
    2. XYZ issues.
    3. DEF issues.

    I like this approach because it forces management to understand and acknowledge where their priorities are. If you're going to ship a product, you have to make hard choices, and one of them is deciding where your priorities are.

    If everything is top priority, then nothing is top priority.

    Update: Sometimes, the answer to "Is this even more important than XYZ?" was "No, XYZ is still more important than ABC." So it's not a gimme that if somebody says that ABC is top priority, it replaces what used to be the top priority. That's why it's important to keep track.

  • The Old New Thing

    Why is the maximum boot.ini delay 11 million seconds?


    I mentioned in passing that the maximum delay you can specify in boot.ini is about 11 million seconds. I'm disappointed but sadly not surprised that everybody focused on that number and completely missed the point of the article.

    First of all, the value of 11 million was not a conscious limitation. It's just an artifact of other limitations. The delay is specified in seconds in boot.ini, but internally it is converted to BIOS clock ticks. (Remember, this is a boot loader; there's not much infrastructure available yet.) The conversion is done in 32-bit arithmetic, and 4 billion BIOS clock ticks at 18.2 ticks per second is about 110 million seconds.

    Wait, but you said 11 million seconds, not 110 million seconds.

    Well, yes, but the conversion is done in 32-bit integer arithmetic: BiosTicks = BootIniSeconds * 182 / 10. A value for BootIniSeconds larger than about 11 million will result in a signed integer overflow in the intermediate product.

    That's why the limit is 11 million seconds. It's a storage limitation, not an arbitrary cutoff. And it is indeed the limit of a natural data type, just hidden behind some other computations. (SB figured this out, though I fear that this shout-out will only serve to encourage people to speculate even more wildly than they already do in the hopes of earning a future shout-out.)

    Now you'd think this would be plenty enough of a boot delay. I mean, it's already much longer than the maximum amount you can delay with a single call to Sleep or WaitForSingleObject. And who would ever need a computer to pause four months before booting? Yet there's one commenter who thought 11 million seconds wasn't enough: "What if I want a timeout of 129 days?" If you need a timeout of 129 days, then go buy a custom hardware timer device that waits 129 days after the onset of power before allowing current to flow. This same commenter has difficulty comprehending that just because you're capable of delaying something for a specific length of time and you're capable of never doing it at all, that doesn't mean that you are also capable of delaying something for an arbitrary length of time.

    One commenter suggested that the problem could be solved by using a bignum library. Before you solve a problem, you first have to be sure that what you have really is a problem in the first place. I have yet to hear of any customers who are running into the 11 million second limit. Using bignums here would be a solution in search of a problem. "Hey, let's consume valuable developer and tester resources in order to add a bunch of code to a tightly-constrained environment to cover a case that nobody cares about." (And I failed to live up to my promise to Ulric merely to incorporate this lecture by reference in the future.)

Page 1 of 4 (33 items) 1234