• The Old New Thing

    After I encrypt memory with CryptProtectMemory, can I move it around?


    A customer had a question about the the Crypt­Protect­Memory function. After using it to encrypt a memory block, can the memory block be moved to another location and decrypted there? Or does the memory block have to be decrypted at the same location it was encrypted?

    The answer is that the memory does not need to be decrypted at the same memory address at which it was encrypted. The address of the memory block is not used as part of the encryption key. You can copy or move the memory around, and as long as you don't tamper with the bytes, and you perform the decryption within the scope you specified, then it will decrypt.

    That the buffer can be moved around in memory is obvious if the scope was specified as CRYPT­PROTECT­MEMORY_CROSS_PROCESS or CRYPT­PROTECT­MEMORY_SAME_LOGON, because those scopes encompass more than one process, so the memory will naturally have a different address in each process. The non-obvious part is that it also holds true for CRYPT­PROTECT­MEMORY_SAME_PROCESS.

    You can also decrypt the buffer multiple times. This is handy if you need to use the decrypted contents more than once, or if you want to hand out the encrypted contents to multiple clients, and leave each client to delay decrypting the data until immediately before they need it. (And then either re-encrypting or simply destroying the data after it is no longer needed in plaintext form.)

    Today's Little Program demonstrates the ability to move encrypted data and to decrypt it more than once.

    #include <windows.h> #include <wincrypt.h> #include <stdio.h> // horrors! mixing C and C++! union MessageBuffer { DWORD secret; char buffer[CRYPTPROTECTMEMORY_BLOCK_SIZE]; }; static_assert(sizeof(DWORD) <= CRYPTPROTECTMEMORY_BLOCK_SIZE, "Need a bigger buffer"); int __cdecl main(int, char **) { MessageBuffer message; // Generate a secret message into the buffer. message.secret = GetTickCount(); printf("Shhh... the secret message is %u\n", message.secret); // Now encrypt the buffer. CryptProtectMemory(message.buffer, sizeof(message.buffer), CRYPTPROTECTMEMORY_SAME_PROCESS); printf("You can't see it now: %u\n", message.secret); // Copy the buffer to a new location in memory. MessageBuffer copiedMessage; CopyMemory(copiedMessage.buffer, message.buffer, sizeof(copiedMessage.buffer)); // Decrypt the copy (at a different address). CryptUnprotectMemory(copiedMessage.buffer, sizeof(copiedMessage.buffer), CRYPTPROTECTMEMORY_SAME_PROCESS); printf("Was the secret message %u?\n", copiedMessage.secret); SecureZeroMemory(copiedMessage.buffer, sizeof(copiedMessage.buffer)); // Do it again! CopyMemory(copiedMessage.buffer, message.buffer, sizeof(copiedMessage.buffer)); // Just to show that the original buffer is not needed, // let's destroy it. SecureZeroMemory(message.buffer, sizeof(message.buffer)); // Decrypt the copy a second time. CryptUnprotectMemory(copiedMessage.buffer, sizeof(copiedMessage.buffer), CRYPTPROTECTMEMORY_SAME_PROCESS); printf("Was the secret message %u?\n", copiedMessage.secret); SecureZeroMemory(copiedMessage.buffer, sizeof(copiedMessage.buffer)); return 0; }

    Bonus chatter: The enumeration values for the encryption scope are rather confusingly named and numbered. I would have called them

    Old name Old value New name New value

    I would have changed the name of the last flag to CRYPT­PROTECT­MEMORY_SAME_MACHINE for two reasons. First, the old name CRYPT­PROTECT­MEMORY_CROSS_PROCESS implies that the memory must travel to another process; i.e., that if you encrypt with cross-process, then it must be decrypted by another process. Second, the flag name creates confusion when placed next to CRYPT­PROTECT­MEMORY_SAME_LOGON, because CRYPT­PROTECT­MEMORY_SAME_LOGON is also a cross-process scenario.

    And I would have renumbered the values so that the entries are in a logical order: Higher numbers have larger scope than lower values.

    Exercise: Propose a theory as to why the old names and values are the way they are.

  • The Old New Thing

    On the ways of creating a GUID that looks pretty


    A customer had what at first appeared to be a question born of curiousity.

    Is it possible that a GUID can be generated with all ASCII characters for bytes? In other words, is it possible that a GUID can be generated, and then if you interpret each of the 16 bytes as an ASCII character, the result is a printable string? Let's say for the sake of argument that the printable ASCII characters are U+0020 through U+007E.

    Now, one might start studying the various GUID specifications to see whether such as GUID is legal. For example, types 1 and 2 are based on a timestamp and MAC address. An all-ASCII MAC address is legal. The locally-administered bit has value 0x02, and one you set that bit, all the other bits can be freely assigned by the network administrator. But then you might notice the Type Variant field, and the requirement that all new GUIDs must set the top bit, so that takes you out of the printable ASCII region, so bzzzzt, no all-ASCII GUID for you.

    But we've fallen into the trap of answering the question instead of solving the problem.

    What is the problem that you're trying to solve, where you are wondering about all-ASCII GUIDs?

    We want to create some sentinel values in our database, and we figured we could use some all-ASCII GUIDs for convenience.

    If you want a sentinel value that is guaranteed to be unique, why not create a GUID?

    C:\> uuidgen
    GUID_SpecialSentinel = {that GUID}

    Now you are guaranteed that the value is unique and will never collide with any other valid GUID.

    We could do that, but we figured it'd be handy if those sentinel values spelled out something so they'd be easier to spot in a dump file. If we know that all-ASCII GUIDs are not valid, then we can use all-ASCII GUIDs for our sentinel values.

    Now, while uuidgen does produce valid GUIDs, it's also the case that those valid GUIDs aren't particularly easy to remember, nor do they exactly trip off the tongue. After all, the space of values that are easy to pronounce and remember is much, much smaller than 2¹²⁸. It's probably more on the order of 2²⁰, which is not enough bits to ensure global uniqueness. Heck, it's not even enough bits to describe all the pixels on your screen!

    So w00t! Since all-ASCII GUIDs are not generatable under the current specification for GUIDs, I can go ahead and name my GUID {6d796152-6e6f-4364-6865-6e526f636b73} which shows up in a memory dump as

    52 61 79 6d 6f 6e 64 43-68 65 6e 52 6f 63 6b 73  RaymondChenRocks

    I am so awesome!

    But even if you convince yourself that no current GUID generation algorithm could create a GUID that collides with your special easy-to-remember and quick-to-pronounce sentinel GUIDs, there is no guarantee that you will make a particularly unique choice of sentinel value.

    This is also known as What if two people did this?

    There are many people named Raymond Chen in the world. Heck, there are many people named Raymond Chen at Microsoft. (We get each other's mail sometimes.) What if somebody else named Raymond Chen gets this same clever idea and creates their own sentinel value called RaymondChenRocks? Everything works great until my database starts interoperating with the other Raymond Chen's database, and now we have a GUID collision.

    Now, the most common way to create a duplicate GUID is to duplicate it. But here, we created a duplicate GUID because the thing we created was not generated via a duplicate-avoidance algorithm. If the algorithm wasn't designed to avoid duplicates, then it's not too surprising that there may be duplicates. I just pulled this GUID out of my butt. (Mind you, my butt rocks.)

    Okay, so let's go back to the original problem so we can solve it.

    The most straightforward solution is simply to create a standard GUID each time you need a new sentinel value. "Oh, I need a GUID to represent an item which has been discontinued. Let me run uuidgen and hey look, there's a new GUID. I will call it GUID_Discontinued." This solves the uniqueness problem, and it is very simple to explain and prove correct. This is what people end up doing the vast majority of the time, and it's what I recommend.

    Okay, you want to have the property that these special GUIDs can be easily spotted in crash dumps. One way to do this is to extract the MAC address from a network card, then destroy the card. You can now use the 60 bits of the timestamp fields to encode your ASCII message.

    A related problem is that you want to generate a GUID based on some other identifying information, with the properties that

    • Two items with the same identifying information should have the same GUID.
    • Two items with different identifying information should have different GUIDs.
    • None of these GUIDs should collide with GUIDs generated by any other means.

    For that, you can use a name-based GUID generation algorithm.

  • The Old New Thing

    If more than one object causes a WaitForMultipleObjects to return, how do I find out about the other ones?


    There is often confusion about the case where you call Wait­For­Multiple­Objects (or its variants), passing bWaitAll = FALSE, and more than one object satisfies the wait.

    When bWaitAll is FALSE, this function checks the handles in the array in order starting with index 0, until one of the objects is signaled. If multiple objects become signaled, the function returns the index of the first handle in the array whose object was signaled.

    The function modifies the state of some types of synchronization objects. Modification occurs only for the object or objects whose signaled state caused the function to return.

    The worry here is that you pass bWaitAll = FALSE, and two objects satisfy the wait, and they are both (say) auto-reset events.

    The first paragraph says that the return value tells you about the event with the lowest index. The second paragraph says that the function modifies the state of the "object or objects" whose signaled state caused the function to return.

    The "or objects" part is what scares people. If two objects caused the function to return, and I am told only about one of them, how do I learn about the other one?

    The answer is, "Don't worry; it can't happen."

    If you pass bWaitAll = FALSE, then only one object can cause the function to return. If two objects become signaled, then the function declares that the lowest-index one is the one that caused the function to return; the higher-index ones are considered not have have caused the function to return.

    In the case of the two auto-reset events: If both events are set, one of them will be chosen as the one that satisfies the wait (the lower-index one), and it will be reset. The higher-index one remains unchanged.

    The confusion stems from the phrase "object or objects", causing people to worry about the case where bWaitAll = FALSE and there are multiple objects which cause the function to return.

    The missing detail is that when you pass bWaitAll = FALSE, then at most one object can cause the function to return. ("At most", because if the operation times out, then no object caused the function to return.)

    The presence of the phrase "or objects" is to cover the case where you pass bWaitAll = TRUE.

  • The Old New Thing

    Why is the System.DateCreated property off by a few seconds?


    If you ask for the System.Date­Created property of a shell item, the timestamp that comes back is up to two seconds different from the file's actual timestamp. (Similarly for System.Date­Modified and System.Date­Accessed.) Why is that?

    This is an artifact of a decision taken in 1993.

    In general, shell namespace providers cache information in the ID list at the time the ID list is created so that querying basic properties from an item can be done without accessing the underlying medium.

    In 1993, saving 4KB of memory had a measurable impact on system performance. Therefore, bytes were scrimped and saved, and one place where four whole bytes were squeezed out was in the encoding of file timestamps in ID lists. Instead of using the 8-byte FILE­TIME structure, the shell used the 4-byte DOS date-time format. Since the shell created thousands of ID lists, a four-byte savings multiplied over thousands of items comes out to several kilobytes of data.

    But one of the limitations of the DOS date-time format is that it records time in two-second increments, so any timestamp recorded in DOS date-time format can be up to two seconds away from its actual value. (The value is always truncated rather than rounded in order to avoid problems with timestamps from the future.) Since Windows 95 used FAT as its native file system, and FAT uses the DOS date-time format, this rounding never created any problems in practice, since all the file timestamps were already pre-truncated to 2-second intervals.

    Of course, Windows NT uses NTFS as the native file system, and NTFS records file times to 100-nanosecond precision. (Though the accuracy is significantly less.) But too late. The ID list format has already been decided, and since ID lists can be saved to a file and transported to another computer (e.g. in the form of a shortcut file), the binary format cannot be tampered with. Hooray for compatibility.

    Bonus chatter: In theory, the ID list format could be extended in a backward-compatible way, so that every ID list contained two timestamps, a compatible version (2-second precision) and a new version (100-nanosecond precision). So far, there has not been significant demand for more accurate timestamps inside of ID lists.

  • The Old New Thing

    The details of the major incident were not clearly articulated, but whatever it is, it's already over

    When a server is taken offline, be it a planned unplanned outage or an unplanned unplanned outage or something else, the operations team send out a series of messages alerting customers to the issue.

    Some time ago, I received a notification that went like this:

    From: Adam Smith
    Subject: Nosebleed Service : Major Incident Notification - Initial
    Date: mm/dd/yyyy 1:16AM

    Major Incident Notification


    Affected Users


    Start: mm/dd/yyyy 12:00AM Pacific Standard Time
    mm/dd/yyyy 8:00AM UTC
    End: No ETA at this time.

    Incident Duration: 1 hour 15 minutes



    Continued Notifications


    Information & Support

    • Other Support: Please send questions or feedback to

    Thank you,

    Adam Smith
    IT Major Incident Management

    Well that clears things up.

    Curiously, the message includes an incident duration but doesn't have an ETA. Thankfully, the message was sent one minute after the incident was over, so by the time I got it, everything was back to normal.

  • The Old New Thing

    Opening the classic folder browser dialog with a specific folder preselected


    Today's Little Program shows how to set the initial selection in the SHBrowse­For­Folder dialog.

    The design of the SHBrowse­For­Folder function had a defect: The BROWSEINFO structure doesn't have a cbSize member at the start. This means that the structure cannot ever change because the function would have no way of knowing whether you are calling with the old structure or the new one. If it weren't for this defect, setting the initial selection would have been easy: Add a pidlInitialSelection member to the structure and have people fill it in.

    Alas, any new functionality in the SHBrowse­For­Folder function therefore requires that the new functionality be expressed within the constraints of the existing structure.

    Fortunately, there's a callback that takes a message number.

    The workaround, therefore, is to express any new functionalty in the form of new callback messages.

    And that's how the ability to set the initial selection in the folder browser dialog came about. A new message BFFM_INITIALIZED was created, and in handling that message, the callback can specify what it wants the selection to be.

    #define UNICODE
    #define _UNICODE
    #include <windows.h>
    #include <ole2.h>
    #include <oleauto.h>
    #include <shlobj.h>
    #include <stdio.h> // horrors! Mixing C and C++!
    int CALLBACK Callback(
        HWND hwnd, UINT uMsg, LPARAM lParam, LPARAM lpData)
     switch (uMsg) {
      SendMessage(hwnd, BFFM_SETSELECTION, TRUE,
     return 0;
    int __cdecl wmain(int, wchar_t **)
     CCoInitialize init;
     TCHAR szDisplayName[MAX_PATH];
     BROWSEINFO info = { };
     info.pszDisplayName = szDisplayName;
     info.lpszTitle = TEXT("Pick a folder");
     info.ulFlags = BIF_RETURNONLYFSDIRS;
     info.lpfn = Callback;
     PIDLIST_ABSOLUTE pidl = SHBrowseForFolder(&info);
     if (pidl) {
      SHGetPathFromIDList(pidl, szDisplayName);
      wprintf(L"You chose %ls\n", szDisplayName);
     return 0;

    We initialize COM and then call the SHBrowse­For­Folder function with information that includes a callback. The callback responds to the BFFM_INITIALIZED message by changing the selection.

    It's not hard, but it's a bit cumbersome.


    Bonus chatter: The presence of the callback means that the function cannot shunt the work to a new thread when called from an MTA thread because you are now stuck with the problem of which thread the callback should run on.

    • The callback may want to access resources that belong to the original thread, so that argues for the callback being run on the original thraed.
    • The callback may also want to access resources inside the dialog box, say in order to customize it. That argues for the callback being run on the new thread.

    You can't have it both ways, so you're stuck.

    But it doesn't really matter, because you shouldn't be performing UI from a multi-threaded apartment anyway. There's not much point in making it easier for people to do the wrong thing.

  • The Old New Thing

    Why is CreateToolhelp32Snapshot returning incorrect parent process IDs all of a sudden?


    A customer reported a problem with the Create­Toolhelp32­Snapshot function.

    From a 32-bit process, the code uses Create­Toolhelp32­Snapshot and Process32­First/Process32­Next to identify parent processes on a 64-bit version of Windows. Sporadically, we find that the th32Parent­Process­ID is invalid on Windows Server 2008. This code works fine on Windows Server 2003. Here's the relevant fragment:

    std::vector<int> getAllChildProcesses(int pidParent)
     std::vector<int> children;
     HANDLE snapshot = CreateToolhelp32Snapshot(
        TH32CS_SNAPPROCESS, 0);
     if (snapshot != INVALID_HANDLE_VALUE) {
      PROCESSENTRY32 entry;
      entry.dwSize = sizeof(entry); // weird that this is necessary
      if (Process32First(snapshot, &entry)) {
       do {
        if (entry.th32ParentProcessID == pidParent) {
        } while (Process32Next(snapshot, &entry));
     return children;

    (The customer snuck another pseudo-question in a comment. Here's why it is necessary.)

    One of my colleagues asked what exactly was "invalid" about the process IDs. (This is like the StackOverflow problem where somebody posts some code and says simply "It doesn't work".)

    My colleague also pointed out that the thParent­Process­ID is simply a snapshot of the parent process ID at the time the child process was created. Since process IDs can be recycled, once the parent process exits, the process ID is left orphaned, and it may get reassigned to another unrelated process. For example, consider this sequence of events:

    • Process A creates Process B.
    • Process A terminates, thereby releasing its ID for reuse.
    • Process C is created.
    • Process C reuses Process A's process ID.

    At this point, Process B will have a th32Parent­Process­ID equal to Process A, but since the ID for Process A has been reused for Process C, it will also be equal to Process C, even though there is no meaningful relationship between processes B and C.

    If Process B needs to rely on its parent process ID remaining assigned to that process (and not getting reassigned), it needs to maintain an open handle to the parent process. (To avoid race conditions, this should be provided by the parent itself.) An open process handle prevents the process object from being destroyed, and in turn that keeps the process ID valid.

    There is another trick of checking the reported parent process's creation time and seeing if it is more recent than the child process's creation time. If so, then you are a victim of process ID reuse, and the true parent process is long gone. (This trick has its own issues. For example, you may not have sufficient access to obtain the parent process's creation time.)

    After a few days, the customer liaison returned with information from the customer. It looks like all of the guidance and explanation provided by my colleague either never made it to the customer, or the customer simply ignored it.

    The customer wants to detect what child processes are spawned by a particular application, let's call it P. We built a special version with extra logging, and it shows that the PROCESS­ENTRY32.th32Parent­Process­ID for wininit.exe and csrss.exe were both 0x15C, which is P's process ID. This erroneous reporting occurs while P is still running and continues after P exits. Do you think it's possible that process 0x15C was used by some other process earlier?

    Yes, that possible. That is, in fact, what my colleague was trying to explain.

    It isn't clear why the customer is trying to track down all child processes of process P, but the way to do this is to create a job object and put process P in it. You can then call Query­Information­Job­Object with Job­Object­Basic­Process­Id­List to get the list of child processes.

  • The Old New Thing

    It appears that some programmers think that Windows already ships with that time machine the Research division is working on


    There are some compatibility bugs we investigate where the root cause is that the application relied on time travel. Wait, let me explain.

    An application might issue an asynchronous request for a file to be renamed before they create the file itself. The program happened to work because it took time for the request to get scheduled and reach the file system, and that delay gave the application time to put the file on the disk just in time for the rename operation to see it.

    Another example is an application which installs a shortcut onto the Start menu that points to a file that they haven't installed yet. The installer happened to work because it took time for the Start menu to notice that a new shortcut was created, and by the time it went looking at the shortcut, the installer had copied the target into place.

    Okay, so maybe it's not so much a time machine as a race condition, but the inherent problem is that the application wanted to do some operation that was dependent on a prerequisite, but issued the operations in the wrong order, and they were relying on the fact that they could get the prerequisite done before the operation even noticed the problem.

    It's like writing a check with insufficient funds,¹ hoping that you can deposit money into the account before the check is cashed. If the check-cashing process ever gets optimized (say, by using electronic check presentation), your sneaky trick will stop working and your check will bounce.

    Now, the developer of the application probably wasn't consciously relying on this race condition, but they never noticed the problem during internal testing because they managed always to win the race. (Or maybe they did notice the problem during internal testing, but since it was sporadic, they chalked it up to "unreproducible failures".)

    In the case of the file renaming operation, losing the race condition means that the original file hangs around on the disk without being renamed. In the case of the shortcut, it means that your shortcut appears on the Start menu with a blank icon.

    If you have one operation that relies upon the successful completion of a previous operation, it would be in your best interest to wait for the previous operation to complete before issuing the dependent operation.

    ¹ As technologically advanced as the United States purports to be, it is still quite common that payments between parties are made by sending little pieces of paper back and forth. For those who live in genuinely technologically advanced countries to whom the idea of sending pieces of paper is rather quaint, here's how it works.

    The original model for checks is simple.

    • Alice has an account at Alligator Bank and wishes to send $10 to Bob.
    • Alice writes a check, which is a piece of paper that says roughly "I authorize Alligator Bank to pay $10 from my account to Bob."
    • Alice sends the check to Bob.
    • Bob goes to Alligator Bank and presents the check, along with proof that he is Bob.
    • Alligator Bank confirms the check's validity, deducts $10 from Alice's account, and gives Bob $10. (If this step fails, the check is said to have bounced.)
    • Alligator Bank stamps paid on the check and gives it back to Alice as confirmation that the payment occurred.

    It's inconvenient for Bob to have to go to Alligator Bank to get his money, but he can ask his bank to do it for him.

    • Bob has an account at Bunny Bank.
    • Bob goes to Bunny Bank and presents the check, along with proof that he is Bob.
    • Bunny Bank sends the check to Alligator Bank demanding payment.
    • Alligator Bank confirms the check's validity, deducts $10 from Alice's account, and sends $10 to Bunny Bank.
    • Bunny Bank credits $10 to Bob's account.

    Over the decades, there have been tweaks to the above process, but the basic system remains in place.

    • Instead of an O() algorithm (where each bank contacts each other bank), the system uses an O(n) algorithm (where each bank contacts a central clearinghouse, which then redistributes the checks).
    • Bunny Bank credits Bob's account before receiving confirmation from Alligator Bank that the check is valid.
    • Check images are sent between banks instead of physical checks.

    There is a category of scams that take advantage of the second detail. I'll leave you to read about them yourself.

    Electronic presentation is an alternative process wherein the information on the check is used to create an electronic payment, which is processed almost immediately, and the original check is never processed as a check.

  • The Old New Thing

    Under what circumstances will a dialog box not use the caption specified in the resource file?


    Could it be space aliens?

    Under what circumstances will a dialog box not use the caption specified in the resource file? In particular, we have a modal dialog box that is not using the caption from the resource file. Even if we explicitly call Set­Window­Text from within the WM_INIT­DIALOG handler, the call succeeds but the caption remains unchanged.

    The dialog box's initial title is the value specified in the resource template. And if you set it again in the WM_INIT­DIALOG handler, then that new title overwrites the title from the resource template. Perhaps the problem is that some other code that runs after your WM_INIT­DIALOG handler is changing the title yet again.

    The customer sheepishly wrote back,

    [banging head against the wall]

    Being skeptical that there could ever be anything else overwriting the code I went to debug with Spy++. After some considerable effort I found out that yes, further down ~30 lines there's a call to Set­Window­Text that changes the title to something else.

    Thanks for making me look again.

    Sometimes the fault is not in our stars but in ourselves.

  • The Old New Thing

    What was the relationship between Outlook and Outlook Express?


    Brian wonders whether project Stimpy became Outlook Express.

    As noted in the article, projects Ren and Stimpy were merged into a single project, which became Outlook. You could say that Stimpy became Outlook, too.

    Outlook Express (code name Athena) was originally known as Internet Mail and News. This was back in the day when the cool, hip thing for Web browsers to do was to incorporate as many Internet client features as possible. In the case of Internet Mail and News, this was POP (mail) and NNTP (news).

    After Outlook became a breakout hit, the Internet Mail and News project was renamed to Outlook Express in an attempt to ride Outlook's coattails. It was a blatant grab at Outlook's brand awareness. (See also: SharePoint Workspaces was renamed OneDrive for Business; Lync was renamed Skype for Business.)

    The decision to give two unrelated projects the same marketing name created all sorts of false expectations, because it implied that Outlook Express was a "light" version of Outlook. People expected that Outlook Express could be upgraded to Outlook, or that Outlook Express and Outlook data files were compatible with each other.

    Code name reuse is common at Microsoft, and for a time, the code names Ren and Stimpy were popular, especially for projects that were closely-related. (As I vaguely recall, there was a networking client/server project that called the server Ren and the client Stimpy. But I may be misremembering, and Ren and Stimpy may just have been the names of the two source code servers.) You may have heard the names Ren and/or Stimpy in reference to some other projects. Doesn't mean that your projects are related to any others with the same name.

Page 9 of 455 (4,543 items) «7891011»