December, 2007

  • The Old New Thing

    Management-speak: Norming around mechanisms

    • 48 Comments

    This is the entire text of an actual piece of email I received from a high-level manager in response to some feedback I sent.

    Thanks.

    There is a lot of norming around any of these mechanisms as well as a certain amount of ability to hold ones ground in these interactions in addition to the admin of the rule being good enough as discussed.

    Bob (not the manager's actual name)

    I've heard rumors that this message was written in English, but I'm not convinced.

  • The Old New Thing

    'Tis the season for top ten lists, and manipulation of top ten lists

    • 6 Comments

    A few years ago, Marketplace radio reported on the sub-industry of top ten lists, specifically the lists intended to be used as gift guides. Marketing companies drool over these lists, since placement on them can mean a tremendous boost in sales, and they're anxious to do whatever it takes to get on the list. For example, The Da Vinci Code was a relatively unknown title until the American Booksellers Assocation placed it on their own "best picks" list.

    In the story, c|net comes off looking good, standing their ethical ground against marketers looking for an endorsement. The Today Show doesn't fare as well.

    On the Media reports on the subject as well.

  • The Old New Thing

    Consequences of the scheduling algorithm: Low priority threads can take 100% CPU

    • 33 Comments

    I see variations on this question occasionally. "Why is my low priority thread consuming 100% CPU?"

    Setting a thread to low priority doesn't mean that it won't consume lots of CPU. It just means that it doesn't get to run as long as there is a higher-priority thread ready to run. But if there is a CPU looking for something to do, and there is no higher-priority thread that is runnable, your low-priority thread will run, and if your low-priority thread is CPU-intensive, it will get all the CPU.

    Priority merely controls which threads get first dibs on CPU time, but if you arrange so that your thread is the only one who wants to run, then it get all the CPU. The chicken at the bottom of the pecking order gets to eat all it wants if no higher-rank chickens are around. You paid for that CPU. There's no point withholding it just out of spite.

  • The Old New Thing

    How do I mark a shortcut file as requiring elevation?

    • 25 Comments

    Specifying whether elevation is required is typically something that is the responsibility of the program. This is done by adding a requestedExecutionLevel element to your manifest. (Bart De Smet shows you how. Calvin Hsia does the same for your Visual FoxPro programs.) But if the program you're running doesn't have such a manifest—maybe it's an old program that you don't have any control over—you can create a shortcut to the program and mark the shortcut as requiring elevation.

    To do this, you set the SLDF_RUNAS_USER flag in the shortcut attributes. Here's a skeleton program that sets the flag on the shortcut whose path is passed on the command line. For expository purposes, I've skimped on the error reporting, and just to shake things up, I've used ATL smart pointers.

    #include <windows.h>
    #include <shlobj.h>
    #include <atlbase.h>
    
    void MarkShortcutRunAs(LPCWSTR pszShortcut)
    {
     CComPtr<IPersistFile> sppf;
     if (FAILED(sppf.CoCreateInstance(CLSID_ShellLink))) return;
     if (FAILED(sppf->Load(pszShortcut, STGM_READWRITE))) return;
     CComQIPtr<IShellLinkDataList> spdl(sppf);
     if (!spdl) return;
     DWORD dwFlags;
     if (FAILED(spdl->GetFlags(&dwFlags))) return;
     dwFlags |= SLDF_RUNAS_USER;
     if (FAILED(spdl->SetFlags(dwFlags))) return;
     if (FAILED(sppf->Save(NULL, TRUE))) return;
     wprintf(L"Succeeded\n");
    }
    
    int __cdecl wmain(int argc, wchar_t *argv[])
    {
     if (argc == 2 && SUCCEEDED(CoInitialize(NULL))) {
      MarkShortcutRunAs(argv[1]);
      CoUninitialize();
     }
     return 0;
    }
    

    There's not really much to this program. It creates a shell link object (CLSID_ShellLink) and asks it to load from the file whose path is given on the command line. It then uses IShellLinkDataList::GetFlags and IShellLinkDataList::SetFlags to fetch the old flags and set new flags that include SLDF_RUNAS_USER. Once that's done, it saves the result back out.

    The hard part was knowing that the SLDF_RUNAS_USER flag existed in the first place.

    (I fear that most people will read this article and say, "Awesome! My program requires elevation, and this is how I can mark my Start menu shortcut to prompt for elevation. Thanks, Raymond!" These people will have completely ignored the opening paragraph, which explains that that is the wrong thing to do.)

  • The Old New Thing

    Book review: Advanced Windows Debugging (Mario Hewardt and Daniel Pravat)

    • 19 Comments

    Ever so often, somebody sends me a book, and most of the time I glance through it and say, "Eh."

    But not this time.

    Advanced Windows Debugging will make you the envy of your friends (if your friends are computer nerds). Even the section with the "Oh come on every moron knows this already" title Basic Debugger Tasks has stuff that I didn't know. Fortunately, you don't have to slog through the stuff you already do know in order to find it, because the nifty new debugger commands are set off in the snippets of debugger conversation. (And by debugger conversation, I mean output of a debugger based on the Windows debug engine, debuggers like ntsd, kd and windbg.)

    Once you get past the "basics", you still have loads more ahead of you. The book covers debugging scenarios like a corrupted heap, a deadlock, or 100% CPU usage, as well as debugging tasks, like following the trail of an LPC request from the client to the server, peeking at the token count of a semaphore, and reconstructing a partially-corrupted stack—and illustrates each investigation with both discussion and annotated debugger output. All the things that seasoned developers take for granted (because they have become instinctual after years of experience) are spelled out for you. Learn more from the book's web site, not unsurprisingly named advancedwindowsdebugging.com.

    I'm keeping this book on my shelf. You can borrow it, but I'm going to insist that you return it when you're done.

  • The Old New Thing

    Not every first-chance exception is a security vulnerability

    • 20 Comments

    In the category of dubious vulnerability, I submit the following (paraphrased) report:

    If I call the FormatMessage function, I can cause a buffer overflow exception if I provide an insertion that is more than 2000 characters long.

    The FormatMessage function in Windows NT, 2000 and XP used the dynamically expanding buffer technique to allocate memory for the resulting message. If the resulting string was more than one page in length (4KB on an x86 system), there was an exception thrown when the FormatMessage function tried to write to the 4096th byte of the buffer. This looks like a buffer overflow, and in a sense it is, but it's a controlled overflow (the bytes beyond the end of the buffer are under the program's control), the exception is entirely expected, and it is correctly handled.

    Using intentionally invalid pages to trigger just-in-time memory commit is a rare technique, so it's not surprising that people aren't familiar with it. In fact, to avoid these sorts of false alarm security vulnerability reports, the kernel folks rewrote the FormatMessage function in Windows Vista so it doesn't use this technique any more.

    It's an odd Catch-22. You remove something that is frequently mistaken for a security vulnerability so that people stop mistakenly reporting it, but the fact that you remove it only confirms in the mind of the people who filed the false alarms that they found something for real!

    (For further reading, may I recommend this blog entry from Larry Osterman.)

  • The Old New Thing

    Wall Street bonus season's trickle-down

    • 10 Comments

    Marketplace covers the businesses who are indirect beneficiaries of the Wall Street bonus season. Ted Fisher, who sells custom-tailored clothing, sees 35 to 40% of his business come in during the three months of bonus season. I suspect business near Redmond are similarly affected by Microsoft review season, though perhaps not as much now as before.

  • The Old New Thing

    How did wildcards work in MS-DOS?

    • 48 Comments

    The rules were simple but led to complicated results.

    MS-DOS files were eleven characters long with an implicit dot between characters eight and nine. Theoretically, spaces were permitted anywhere, but in practice they could appear only at the end of the file name or immediately before the implicit dot.

    Wildcard matching was actually very simple. The program passed an eleven-character pattern; each position in the pattern consisted either of a file name character (which had to match exactly) or consisted of a question mark (which matched anything). Consider the file "ABCD····TXT", where I've used · to represent a space. This file name would more traditionally be written as ABCD.TXT, but I've written it out in its raw 11-character format to make the matching more obvious. Let's look at some patterns and whether they would match.

    PatternResultExplanation
    ABCD····TXT Match exact
    ??????????? Match all positions are a wildcard so of course they match
    ABCD????··· No match space (position 9) does not match T
    A?CD····??? match perfect match at A, C, D, and the spaces; wildcard match at the question marks

    The tricky part is converting the traditional notation with dots and asterisks into the eleven-character pattern. The algorithm used by MS-DOS was the same one used by CP/M, since MS-DOS worked hard at being backwards compatible with CP/M. (You may find some people who call this the FCB matching algorithm, because file names were passed to and from the operating system in a structure called a File Control Block.)

    1. Start with eleven spaces and the cursor at position 1.
    2. Read a character from the input. If the end of the input is reached, then stop.
    3. If the next character in the input is a dot, then set positions 9, 10, and 11 to spaces, move the cursor to position 9, and go back to step 2.
    4. If the next character in the input is an asterisk, then fill the rest of the pattern with question marks, move the cursor to position 12, and go back to step 2. (Yes, this is past the end of the pattern.)
    5. If the cursor is not at position 12, copy the input character to the cursor position and advance the cursor.
    6. Go to step 2.

    Let's parse a few patterns using this algorithm, since the results can be surprising. In the diagrams, I'll underline the cursor position.

    First, let's look at the traditional "ABCD.TXT".

    InputPatternDescription
    ··········· Initial conditions
    A A·········· Copy to cursor and advance the cursor
    B AB········· Copy to cursor and advance the cursor
    C ABC········ Copy to cursor and advance the cursor
    D ABCD······· Copy to cursor and advance the cursor
    . ABCD······· Blank out positions 9, 10, and 11 and move cursor to position 9
    T ABCD····T·· Copy to cursor and advance the cursor
    X ABCD····TX· Copy to cursor and advance the cursor
    T ABCD····TXT  Copy to cursor and advance the cursor

    The final result is what we expected: ABCD····TXT.

    Let's look at a weird case: the pattern is ABCDEFGHIJKL.

    InputPatternDescription
    ··········· Initial conditions
    A A·········· Copy to cursor and advance the cursor
    B AB········· Copy to cursor and advance the cursor
    C ABC········ Copy to cursor and advance the cursor
    D ABCD······· Copy to cursor and advance the cursor
    E ABCDE······ Copy to cursor and advance the cursor
    F ABCDEF····· Copy to cursor and advance the cursor
    G ABCDEFG···· Copy to cursor and advance the cursor
    H ABCDEFGH··· Copy to cursor and advance the cursor
    I ABCDEFGHI·· Copy to cursor and advance the cursor
    J ABCDEFGHIJ· Copy to cursor and advance the cursor
    K ABCDEFGHIJK  Copy to cursor and advance the cursor

    Sure, this was extremely boring to watch, but look at the result: What you got was equivalent to ABCDEFGH.IJK. The dot is optional if it comes after exactly eight characters!

    Next, let's look at the troublesome A*B.TXT.

    InputPatternDescription
    ··········· Initial conditions
    A A·········· Copy to cursor and advance the cursor
    * A??????????  Fill rest of pattern with question marks and move to position 12
    B A??????????  Do nothing since cursor is at position 12
    . A???????··· Blank out positions 9, 10, and 11 and move cursor to position 9
    T A???????T·· Copy to cursor and advance the cursor
    X A???????TX· Copy to cursor and advance the cursor
    T A???????TXT  Copy to cursor and advance the cursor

    Notice that the result is the same as you would have gotten from the pattern A*.TXT. Any characters other than a dot that come after an asterisk have no effect, since the asterisk moves the cursor to position 12, at which point nothing changes the parse state except for a dot, which clears the last three positions and moves the cursor.

    I won't work it out here, but if you stare at it for a while, you'll also discover that *.* is the same as * by itself.

    In addition to the rules above, the MS-DOS command prompt had some quirks in its parsing. If you typed DIR .TXT, the command prompt acted as if you had typed DIR *.TXT; it silently inserted an asterisk if the first character of the pattern was a dot. This behavior was probably by accident, not intentional, but it was an accident that some people came to rely upon. When we fixed the bug in Windows 95, more than one person complained that their DIR .TXT command wasn't working.

    The FCB matching algorithm was abandoned during the transition to Win32 since it didn't work with long file names. Long file names can contain multiple dots, and of course files can be longer than eleven characters, and there can be more than eight characters before the dot. But some quirks of the FCB matching algorithm persist into Win32 because they have become idiom.

    For example, if your pattern ends in .*, the .* is ignored. Without this rule, the pattern *.* would match only files that contained a dot, which would break probably 90% of all the batch files on the planet, as well as everybody's muscle memory, since everybody running Windows NT 3.1 grew up in a world where *.* meant all files.

    As another example, a pattern that ends in a dot doesn't actually match files which end in a dot; it matches files with no extension. And a question mark can match zero characters if it comes immediately before a dot.

    There may be other weird Win32 pattern matching quirks, but those are the two that come to mind right away, and they both exist to maintain batch file compatibility with the old 8.3 file pattern matching algorithm.

  • The Old New Thing

    Unwittingly enveloped in the Santarchy

    • 11 Comments

    Last Saturday, my friends and I were heading for a subway station in Brooklyn and found ourselves behind a man and a woman both dressed in Santa Claus outfits. They were kind enough to wave us onto the train with a "hurry, hurry!", and we made it on board just as the subway doors closed.

    And then we discovered that we were on a subway train filled with people dressed as Santa Claus, noisy but well-behaved. What parallel universe had we mistakenly stepped into? Was there some subway dress code announcement we had missed?

    Nope. The announcement we missed was for the 2007 New York City SantaCon. (Here they are swarming Grand Central Terminal.)

    Many cities are having their SantaCon tomorrow, including Seattle, so if you see roving bands of red-suited revelers, you'll at least know what's going on.

  • The Old New Thing

    The compatibility constraints of your side effects: Beeping

    • 30 Comments

    Why does the Welcome screen beep if you hit the space bar after the computer has finished starting up?

    For compatibility with a side effect of the Windows XP Welcome screen.

    A beta tester asked us why we removed the beeps that were generated if you started typing at the Welcome screen after the computer finished starting up. Well, we didn't remove the beeps consciously. That the old Welcome screen beeped was just a side effect of the way the old Welcome screen positioned focus at startup and how it handled unexpected input. When the Welcome screen was rewritten for Windows Vista, focus and input were handled differently, and it so happened that the new way didn't beep under the initial conditions.

    The tester in this case was blind and relied upon the beeping to know when the system was ready. The Welcome screen folks had to go back and take extra care to ensure that if you start the computer and wait for the Welcome screen, then hit the space bar, you get a beep.

Page 2 of 4 (37 items) 1234