• The Old New Thing

    Why does the BackupWrite function take a pointer to a modifiable buffer when it shouldn't be modifying the buffer?


    The Backup­Write function takes a non-const pointer to the buffer to be written to the file being restored. Will it actually modify the buffer? Assuming it doesn't, why wasn't it declared const? It would be much more convenient if it took a const pointer to the buffer, so that people with const buffers didn't have to const_cast every time they called the function. Would changing the parameter from non-const to const create any compatibility problems?

    Okay, let's take the questions in order.

    Will it actually modify the buffer? No.

    Why wasn't it declared const? My colleague Aaron Margosis explained that the function dates back to Windows NT 3.1, when const-correctness was rarely considered. A lot of functions from that area (particularly in the kernel) suffer from the same problem. For example, the computer name passed to the Reg­Connect­Registry function is a non-const pointer even though the function never reads from it.

    Last question: Can the parameter be changed from non-const to const without breaking compatibility?

    It would not cause problems from a binary compatibility standpoint, because a const pointer and a non-const pointer take the same physical form in Win32. However, it breaks source code compatiblity. Consider the following code fragment:

    BOOL WINAPI TestModeBackupWrite(
      HANDLE hFile,
      LPBYTE lpBuffer,
      DWORD nNumberOfBytesToWrite,
      LPDWORD lpNumberOfBytesWritten,
      BOOL bAbort,
      BOOL bProcessSecurity,
      LPVOID *lpContext)
     ... simulate a BackupWrite ...
     return TRUE;
                     LPDWORD, BOOL, BOOL, LPVOID *);
    BACKUPWRITEPROC TestableBackupWrite;
    void SetTestMode(bool testing)
     if (testing) {
      TestableBackupWrite = TestModeBackupWrite;
     } else {
      TestableBackupWrite = BackupWrite;

    The idea here is that the program can be run in test mode, say to do a simulated restore. (You see this sort of thing a lot with DVD-burning software.) The program uses Testable­Backup­Write whenever it wants to write to a file being restored from backup. In test mode, Testable­Backup­Write points to the Test­Mode­Backup­Write function; in normal mode, it points to the Backup­Write function.

    If the second parameter were changed from LPBYTE to const BYTE *, then the above code would hit a compiler error.

    Mind you, maybe it's worth breaking some source code in order to get better const-correctness, but for now, the cost/benefit tradeoff biases toward leaving things alone.

  • The Old New Thing

    Is a SID with zero subauthorities a valid SID? It depends whom you ask


    Here's an interesting table.

    Function Is Sub­Authority­Count=0 valid?
    IsValidSid Yes
    Convert­Sid­To­String­Sid Yes
    ConvertString­­Sid­To­Sid No

    That last entry creates the unfortunate situation where a SID with no subauthorities can be converted to a string, but cannot be converted back.

    If it's any consolation, SIDs with no subauthorities aren't encountered in normal usage, so if you ever accidentally reject one of these, it's not going to inconvenience anyone.

    Oh, and the answer to the question at the top: Yes, a SID with zero subauthorities is technically valid. It's a degenerate case that's not very interesting, but it is technically valid.

  • The Old New Thing

    How secure are GUIDs? Can I use it to generate passwords?


    GUIDs are globally unique identifiers, and given their rather incomprehensible presentation, some people may be tempted to use them (or parts of them) as passwords. For example, one person wondered whether it was okay to use the first eight characters of a GUID as a temporary account password.

    This is a really bad idea. GUIDs are designed for uniqueness, not for security.

    For example, we saw that substrings of GUIDs are not unique. For example, in the classic v1 algorithm, the first part of the GUID is a timestamp. Timestamps are a great technique for helping to build uniqueness, but they are horrifically insecure because, well, duh, the current time is hardly a secret!

    Using the first eight characters of a v1 GUID as a password is equivalent to using the current time as a password. And given that the current time isn't a secret, you made things an awful lot easier for the bad guys. All they need to know is what time you generated the GUID and they can get the first eight characters. Even if they don't know the exact time, if they can narrow it down to a range of a few minutes, they can brute-force the remaining options.

    Even if you use one of the other GUID generation algorithms, that doesn't solve the problem, because the purpose of a GUID is to be globally unique, not to be unguessable.

    Besides, the first eight characters of a GUID carry only 32 bits of entropy, because they come from a very limited alphabet: They can be a digit, or a letter from A to F. That's it.

    If you want something cryptographically secure, then use a cryptographically-secure random number. Don't use something that "looks random to me." And use more entropy than 32 bits.

    (Yes, this is basically a rehash of an earlier article, but I wanted to address directly the matter of using GUIDs as passwords.)

  • The Old New Thing

    2015 mid-year link clearance


    The semi-annual link clearance is here!

  • The Old New Thing

    Microspeak: Stretch goal


    Recall that Microspeak is not merely for jargon exclusive to Microsoft, but also for jargon that you need to know.

    A project will set some goals, which are things it intends to accomplish. It may also set stretch goals, which are things it hopes to acomplish. If you fail to achieve a stretch goal, your project is still a success, but if you make it, your project is even more awesome. A stretch goal could be unrelated to an existing goal.

    Goals for this release

    • Support dynamic widget recolorization.

    Stretch goals for this release

    • Improve throughput by 10% compared to previous version.

    Non-goals for this release

    • Support offline mode.

    But more often, it takes the form of a higher level of achievement for an existing goal:

    Goal: 40% of the programs will land actual spend within 10% of estimate. Stretch goal: 60%.

    Setting a stretch goal is tricky. You want to set it just at the edge of achievability. If you make it unrealistic, then nobody will take it seriously.

    You can think of a stretch goal as an "extra credit" assignment. You won't be penalized for missing it, but making it will earn you kudos.

  • The Old New Thing

    Parsing a string as a 64-bit integer, somehow


    Today's Little Program takes a string and tries to parse it as a 64-bit integer in formats that a programmer would likely encounter.

    Here's a first stab:

    using System;
    using System.Globalization;
    class Program
     static long ParseLongSomehow(string s)
      if (s.StartsWith("0x", StringComparison.OrdinalIgnoreCase)) {
       return long.Parse(s.Substring(2), NumberStyles.HexNumber);
      } else {
       return long.Parse(s);
     public static void Main(string[] args)
      long value = ParseLongSomehow(args[0]);
      Console.WriteLine("0x{0:X}", value);

    If the string begins with 0x, then we treat the rest of the argument as a hex value; otherwise, we treat it as a decimal value.

    Unfortunately, this doesn't work if the input is 9223372036854775808, which is the value of 1 << 63, a value that is representable as a 64-bit unsigned value but not a 64-bit signed value.

    Our problem statement was pretty vague, so let's write a functional specification. It helps to know what problem you're solving before you start to solve it. Otherwise, you're just flailing around writing code before you have a plan. When I tried to solve this problem, I flailed around a bit until I realized that I didn't have a spec.

    What formats would a programmer be likely to encounter as the string representation of a 64-bit integer?

    • 0x1234: 64-bit number in hex format, case-insensitive. The value can range from 0 to UInt64.MaxValue.
    • 12345: 64-bit unsigned number in decimal format. The value can range from 0 to UInt64.MaxValue.
    • -12345: 64-bit signed number in decimal format. The value can range from Int64.MinValue to Int64.MaxValue.
    • Other formats may be permitted, but you need to support at least the above.

    Writing down exactly what I was doing and what I wasn't doing was the part that solved my flailing. I had been worrying about things like -0x12345 and -9223372036854775809 and 9999999999999999999, even though those numbers would not be something a programmer would be likely to encounter.

    From the specification we can develop our algorithm.

    • If the string begins with 0x, then parse what's left as an unsigned 64-bit hexadecimal number.
    • If the string begins with a minus sign, then parse it as a 64-bit signed number in decimal format.
    • If the string does not begin with a minus sign, then parse it as a 64-bit unsigned number in decimal format.

    And that is pretty easy to implement.

     static long ParseLongSomehow(string s)
      if (s.StartsWith("0x", StringComparison.OrdinalIgnoreCase)) {
       return long.Parse(s.Substring(2), NumberStyles.HexNumber);
      } else if (s[0] == '-') {
       return long.Parse(s);
      } else {
       return (long)ulong.Parse(s);

    Note that we are a little sloppy with our treatment of whitespace. We accept leading and trailing spaces on decimal values, and allow trailing spaces on hex values (and even allow spaces between the 0x and the first hex digit). That's okay, because the spec allows us to accept formats beyond the ones listed.

    Now, for bonus points, let's revise the functional specification a little bit, specifically by adding another case:

    • 0x12`3456789A: 64-bit number in hex format, case-insensitive, with backtick separating the upper 32 bits from the lower 32 bits.

    This is the format used by the Windows debugger engine.

     static long ParseLongSomehow(string s)
      if (s.StartsWith("0x", StringComparison.OrdinalIgnoreCase)) {
       return long.Parse(s.Substring(2).Replace("`", ""), NumberStyles.HexNumber);
      } else if (s[0] == '-') {
       return long.Parse(s);
      } else {
       return (long)ulong.Parse(s);

    We'll leave it here for now. Next time, we'll start putting some blocks together.

  • The Old New Thing

    Windows 95 and Windows NT manage the TEB on x86 differently


    Today, a historical note of no consequence. Just a story.

    The Win32 x86 ABI specifies that the FS register holds a selector which is based at the current thread's TEB. In other words, fs:[n] is the nth byte of the TEB.

    It so happens that the two operating systems chose to manage the FS register differently.

    Windows 95 gave each TEB in the system its own selector.

    Windows NT allocated a single selector to represent the TEB, and each time the processor changed threads, the selector base was updated to match the TEB for the new thread. With this model, every thread has the same value for FS, but the selector's descriptor kept changing.

    It's as if you had a car-rental service, and one of the features of the service is that the radio remembers your presets. The instructions for setting the radio are as follows:

    • Enter the four-digit customer preferences ID printed on your receipt.
    • Your radio is now set to your preferences.

    There are two ways you could set up this system.

    Windows 95 assigns each customer a unique customer preferences ID and prints it on the receipt. When the customer enters the ID, the radio looks up the ID in a database and applies the user's radio preferences.

    Windows NT prints the same preferences ID on every receipt: 1234. Before the customer picks up the car from the rental service, the service agent sets the radio to the customer's preferences. When the customer enters the ID, the radio does nothing (aside from printing an error message if you enter anything other than 1234).

    Even though the Windows NT way creates more work for the service agent, it does solve an important problem: It lets your service scale to more than 10,000 customers, for once you have 10,001 customers, you cannot assign each of them a unique four-digit ID any more.

    Car Windows
    car processor
    customer thread
    radio preferences TEB
    customer ID selector

    By assigning a unique selector to each TEB, Windows 95 limited itself to at most 8192 threads. (In practice, the limit was much lower because selectors were used for other things, too.) This was not an issue in practice because Windows 95 would run into other problems long before you ran into the 8192-thread limit.

    But Windows NT was designed to be scalable to very large workloads, and they couldn't artificially limit themselves to a maximum of 8192 threads.

    The consequences of changing the meaning of the FS register at every thread switch means that some tricks that happened to work in Windows 95 didn't work in Windows NT. For example, in Windows 95, if you captured the value of the FS register in one thread, you could use it (in violation of the ABI) on another thread in the same process and still access the originating thread's TEB. If you tried this trick on Windows NT, you would just see your own TEB.

    In the analogy, it's as if you copied the customer preferences ID from another customer's receipt and tried to use it in your car. In a Windows NT car, you would just get your own preferences again, because every receipt has the same customer preferences ID printed on it.

    According to the Win32 ABI for x86, the only thing you can do with the FS register is dereference it to access your TEB. If you try to fiddle with its value or try to copy its value somewhere, you are off in unsupported territory, and the resulting behavior is undefined.

  • The Old New Thing

    Where is the full version of the music that plays when you start Windows 98 for the first time?


    A customer was presumably exercising the unlimited support part of their support contract when they asked, "Where is the full song for the music that plays when you start Windows 98 for the first time? The Welcome application plays only the first 30 seconds. Can you send us the rest of the song?"

    I guess the IT department really likes that music and wants the extended dance remix for their late-night raves. (If you stick it out, there's a special appearance in the linked video of the screen I wrote.)

    The music was commissioned to be 30 seconds long. There is no extended version of the song, or at least not one that Microsoft acquired the rights to. The original composer could have written a five-minute version and sent Microsoft only the first 30 seconds, or maybe the piece is only 30 seconds long to begin with. All we know is that we have a 30-second music clip.

  • The Old New Thing

    A trick for finding the correct internal mailing list


    Suppose you have a problem and you want to report it to an internal mailing list, but you don't know the correct mailing list to use. Here's a trick that apparently is not that well-known, though perhaps that's a reflection on people's lack of creativity than any inherent difficulty in execution.

    For example, suppose you have a problem with the file system. One thing you can try is to go to the company Address Book and type "file system" and see what turns up.

    Of course, you have to be judicious about this. Some people use this trick but aren't very careful about their search terms, and they end up sending questions to mailing lists like "Windows Team All" which are intended for announcements, not Q&A.

    You want to look for a feature-specific discussion group or Q&A group or bug reporting group, not an entire product group.

    Use your common sense.

    In many cases, you have an idea of a person who might be able to help you. You may know the name of somebody from the file system team because you see their name in status report mail, or you find their name in the change logs, or because they helped you out once before. But, of course, you don't want to go to that person directly, because that would be presumptuous, and they might be on vacation, or they might be busy, or they might not be the right person after all.

    But what you can do is look up their entry in the company Address Book, then click on Membership to see what mailing lists they belong to. Browse the list looking for something named "File system bugs" or "File system talk" or "File system automated testing failures" or "File system code reviews", depending on what sort of help you need. If you know more than one person who might help, you can do some set theory to narrow down on a good candidate mailing list to contact.

    (Within Microsoft, a common convention is for mailing lists that deal with general Q&A to have "talk" in their name, whereas those that deal with urgent issues have "hot" in their name. And then there is "casual" which is where team members hang out and discuss the latest episode of Breaking Bad Mad Men Game of Thrones.)

    Bonus chatter: There is a Microsoft internal Web site called Polyarchy which visualizes organizational data. One of the many things you can do is give it a list of people, and it will show you the mailing lists that they all belong to, thereby doing the set theory magic I alluded to earlier.

  • The Old New Thing

    Sorry for the interruption, but it doesn't happen often


    Many years ago, my feature manager and I were called into the project leader's office for some reason or other, I forget exactly what.

    We were about five minutes into the meeting when the project leader's mobile phone rang. This was back in the days when mobile phones were not commonplace, and having one was a way of showing off your status.

    The project leader answered the call, but instead of saying, "I'm in a meeting, can I call you back?" he proceeded to carry on a conversation with the caller as if we weren't there, while we sat there and waited.

    I got up and left his office.

    I went downstairs to the lobby and read whatever newspaper happened to be sitting there for visitors to read while they waited. I think it was The Wall Street Journal, but it could have been The New York Times.

    After finishing a section of the paper, I came back upstairs to the project leader's office. By that time, the project leader had finished his call, whatever it was.

    As I sat down, the project leader said, "Sorry, I don't get calls on this phone often."

    I immediately replied, "That's okay, I don't read The Wall Street Journal often either."

Page 1 of 453 (4,526 items) 12345»