• The Old New Thing

    Why did the original code for FIND.COM use lop as a label instead of loop?

    • 4 Comments

    A few years ago, I left you with an exercise: Given the code

            mov     dx,st_length            ;length of the string arg.
            dec     dx                      ;adjust for later use
            mov     di, line_buffer
    lop:
            inc     dx
            mov     si,offset st_buffer     ;pointer to beg. of string argument
    
    comp_next_char:
            lodsb
            cmp     al,byte ptr [di]
            jnz     no_match
    
            dec     dx
            jz      a_matchk                ; no chars left: a match!
            call    next_char               ; updates di
            jc      no_match                ; end of line reached
            jmp     comp_next_char          ; loop if chars left in arg.
    

    why is the loop label called lop instead of loop?

    The answer is that calling it loop would create ambiguity with the 8086 instruction loop.

    Now, you might say (if your name is Worf), that there is no ambiguity. "Every line consists of up to four things (all optional). A label, an instruction/pseudo-instruction, operands, and comments. The label is optionally followed by a colon. If there is no label, then the line must start with whitespace."

    If those were the rules, then there would indeed be no ambiguity.

    But those aren't the rules. Leading whitespace is not mandatory. If you are so inclined, you can choose to begin your instructions all in column zero.

    mov dx,st_length
    dec dx
    mov di, line_buffer
    lop:
    inc dx
    mov si,offset st_buffer
    comp_next_char:
    lodsb
    cmp al,byte ptr [di]
    jnz no_match
    dec dx
    jz a_matchk
    call next_char
    jc no_match
    jmp comp_next_char
    

    It's not recommended, but it's legal. (I have been known to do this when hard-coding breakpoints for debugging purposes. That way, a search for /^int 3/ will find all of my breakpoints.)

    Since you can put the opcode in column zero, a line like this would be ambiguous:

    loop ret
    

    This could be parsed as "Label this line loop and execute a ret instruction." Or it could be parsed as "This is an unlabeled line, consisting of a loop instruction that jumps to the label ret."

    Label Opcode Operand
    loop ret
    – or –
    loop ret

    Disallowing instruction names as labels or macros or equates is the simplest way out of this predicament. Besides, you probably shouldn't be doing it anyway. Imagine the havoc if you did

    or equ and
    
  • The Old New Thing

    When I set the "force X off" policy to Disabled, why doesn't it force X on?

    • 18 Comments

    A customer was using one of the many "force X off" policies, but instead of using it to force X off, they were trying to use it to force X on by setting the policy to Disabled. For example, there is a "Hide and disable all items on the desktop". The customer was setting this policy to Disabled, expecting it to force all icons visible on the desktop, removing the option on the desktop View menu to hide the icons.

    As we discussed some time ago, group policy is for modifying default behavior, and interpreting them requires you to have a degree in philosophy.

    In particular, a policy which forces X off has three states:

    • Enabled: X is forced off.
    • Disabled: X is not forced off.
    • Not configured: No opinion. Let another group policy object decide.

    Disabling a policy means "Return to default behavior", and the default behavior in many cases is that the user can decide whether they want X or not by selecting the appropriate option. In philosophical terms, "Not forced off" is not the same as "Forced on."

    If you want to force X on, then you have to look for a policy that says "Force X on." (And if there isn't one, then forcing X on is not something currently supported by group policy.)

  • The Old New Thing

    In the Seattle area, people treat you nicer once they learn your last name is Gates

    • 11 Comments

    One of the team members during the days of Windows 95 happens to have the last name Gates, no relation.

    She says that it's surprising how nice salespeople are to you once they learn what your last name is.

    Go figure.

  • The Old New Thing

    After I encrypt memory with CryptProtectMemory, can I move it around?

    • 23 Comments

    A customer had a question about the the Crypt­Protect­Memory function. After using it to encrypt a memory block, can the memory block be moved to another location and decrypted there? Or does the memory block have to be decrypted at the same location it was encrypted?

    The answer is that the memory does not need to be decrypted at the same memory address at which it was encrypted. The address of the memory block is not used as part of the encryption key. You can copy or move the memory around, and as long as you don't tamper with the bytes, and you perform the decryption within the scope you specified, then it will decrypt.

    That the buffer can be moved around in memory is obvious if the scope was specified as CRYPT­PROTECT­MEMORY_CROSS_PROCESS or CRYPT­PROTECT­MEMORY_SAME_LOGON, because those scopes encompass more than one process, so the memory will naturally have a different address in each process. The non-obvious part is that it also holds true for CRYPT­PROTECT­MEMORY_SAME_PROCESS.

    You can also decrypt the buffer multiple times. This is handy if you need to use the decrypted contents more than once, or if you want to hand out the encrypted contents to multiple clients, and leave each client to delay decrypting the data until immediately before they need it. (And then either re-encrypting or simply destroying the data after it is no longer needed in plaintext form.)

    Today's Little Program demonstrates the ability to move encrypted data and to decrypt it more than once.

    #include <windows.h> #include <wincrypt.h> #include <stdio.h> // horrors! mixing C and C++! union MessageBuffer { DWORD secret; char buffer[CRYPTPROTECTMEMORY_BLOCK_SIZE]; }; static_assert(sizeof(DWORD) <= CRYPTPROTECTMEMORY_BLOCK_SIZE, "Need a bigger buffer"); int __cdecl main(int, char **) { MessageBuffer message; // Generate a secret message into the buffer. message.secret = GetTickCount(); printf("Shhh... the secret message is %u\n", message.secret); // Now encrypt the buffer. CryptProtectMemory(message.buffer, sizeof(message.buffer), CRYPTPROTECTMEMORY_SAME_PROCESS); printf("You can't see it now: %u\n", message.secret); // Copy the buffer to a new location in memory. MessageBuffer copiedMessage; CopyMemory(copiedMessage.buffer, message.buffer, sizeof(copiedMessage.buffer)); // Decrypt the copy (at a different address). CryptUnprotectMemory(copiedMessage.buffer, sizeof(copiedMessage.buffer), CRYPTPROTECTMEMORY_SAME_PROCESS); printf("Was the secret message %u?\n", copiedMessage.secret); SecureZeroMemory(copiedMessage.buffer, sizeof(copiedMessage.buffer)); // Do it again! CopyMemory(copiedMessage.buffer, message.buffer, sizeof(copiedMessage.buffer)); // Just to show that the original buffer is not needed, // let's destroy it. SecureZeroMemory(message.buffer, sizeof(message.buffer)); // Decrypt the copy a second time. CryptUnprotectMemory(copiedMessage.buffer, sizeof(copiedMessage.buffer), CRYPTPROTECTMEMORY_SAME_PROCESS); printf("Was the secret message %u?\n", copiedMessage.secret); SecureZeroMemory(copiedMessage.buffer, sizeof(copiedMessage.buffer)); return 0; }

    Bonus chatter: The enumeration values for the encryption scope are rather confusingly named and numbered. I would have called them

    Old name Old value New name New value
    CRYPT­PROTECT­MEMORY_SAME_PROCESS 0 CRYPT­PROTECT­MEMORY_SAME_PROCESS 0
    CRYPT­PROTECT­MEMORY_SAME_LOGON 2 CRYPT­PROTECT­MEMORY_SAME_LOGON 1
    CRYPT­PROTECT­MEMORY_CROSS_PROCESS 1 CRYPT­PROTECT­MEMORY_SAME_MACHINE 2

    I would have changed the name of the last flag to CRYPT­PROTECT­MEMORY_SAME_MACHINE for two reasons. First, the old name CRYPT­PROTECT­MEMORY_CROSS_PROCESS implies that the memory must travel to another process; i.e., that if you encrypt with cross-process, then it must be decrypted by another process. Second, the flag name creates confusion when placed next to CRYPT­PROTECT­MEMORY_SAME_LOGON, because CRYPT­PROTECT­MEMORY_SAME_LOGON is also a cross-process scenario.

    And I would have renumbered the values so that the entries are in a logical order: Higher numbers have larger scope than lower values.

    Exercise: Propose a theory as to why the old names and values are the way they are.

  • The Old New Thing

    On the ways of creating a GUID that looks pretty

    • 39 Comments

    A customer had what at first appeared to be a question born of curiousity.

    Is it possible that a GUID can be generated with all ASCII characters for bytes? In other words, is it possible that a GUID can be generated, and then if you interpret each of the 16 bytes as an ASCII character, the result is a printable string? Let's say for the sake of argument that the printable ASCII characters are U+0020 through U+007E.

    Now, one might start studying the various GUID specifications to see whether such as GUID is legal. For example, types 1 and 2 are based on a timestamp and MAC address. An all-ASCII MAC address is legal. The locally-administered bit has value 0x02, and one you set that bit, all the other bits can be freely assigned by the network administrator. But then you might notice the Type Variant field, and the requirement that all new GUIDs must set the top bit, so that takes you out of the printable ASCII region, so bzzzzt, no all-ASCII GUID for you.

    But we've fallen into the trap of answering the question instead of solving the problem.

    What is the problem that you're trying to solve, where you are wondering about all-ASCII GUIDs?

    We want to create some sentinel values in our database, and we figured we could use some all-ASCII GUIDs for convenience.

    If you want a sentinel value that is guaranteed to be unique, why not create a GUID?

    C:\> uuidgen
    GUID_SpecialSentinel = {that GUID}
    

    Now you are guaranteed that the value is unique and will never collide with any other valid GUID.

    We could do that, but we figured it'd be handy if those sentinel values spelled out something so they'd be easier to spot in a dump file. If we know that all-ASCII GUIDs are not valid, then we can use all-ASCII GUIDs for our sentinel values.

    Now, while uuidgen does produce valid GUIDs, it's also the case that those valid GUIDs aren't particularly easy to remember, nor do they exactly trip off the tongue. After all, the space of values that are easy to pronounce and remember is much, much smaller than 2¹²⁸. It's probably more on the order of 2²⁰, which is not enough bits to ensure global uniqueness. Heck, it's not even enough bits to describe all the pixels on your screen!

    So w00t! Since all-ASCII GUIDs are not generatable under the current specification for GUIDs, I can go ahead and name my GUID {6d796152-6e6f-4364-6865-6e526f636b73} which shows up in a memory dump as

    52 61 79 6d 6f 6e 64 43-68 65 6e 52 6f 63 6b 73  RaymondChenRocks
    

    I am so awesome!

    But even if you convince yourself that no current GUID generation algorithm could create a GUID that collides with your special easy-to-remember and quick-to-pronounce sentinel GUIDs, there is no guarantee that you will make a particularly unique choice of sentinel value.

    This is also known as What if two people did this?

    There are many people named Raymond Chen in the world. Heck, there are many people named Raymond Chen at Microsoft. (We get each other's mail sometimes.) What if somebody else named Raymond Chen gets this same clever idea and creates their own sentinel value called RaymondChenRocks? Everything works great until my database starts interoperating with the other Raymond Chen's database, and now we have a GUID collision.

    Now, the most common way to create a duplicate GUID is to duplicate it. But here, we created a duplicate GUID because the thing we created was not generated via a duplicate-avoidance algorithm. If the algorithm wasn't designed to avoid duplicates, then it's not too surprising that there may be duplicates. I just pulled this GUID out of my butt. (Mind you, my butt rocks.)

    Okay, so let's go back to the original problem so we can solve it.

    The most straightforward solution is simply to create a standard GUID each time you need a new sentinel value. "Oh, I need a GUID to represent an item which has been discontinued. Let me run uuidgen and hey look, there's a new GUID. I will call it GUID_Discontinued." This solves the uniqueness problem, and it is very simple to explain and prove correct. This is what people end up doing the vast majority of the time, and it's what I recommend.

    Okay, you want to have the property that these special GUIDs can be easily spotted in crash dumps. One way to do this is to extract the MAC address from a network card, then destroy the card. You can now use the 60 bits of the timestamp fields to encode your ASCII message.

    A related problem is that you want to generate a GUID based on some other identifying information, with the properties that

    • Two items with the same identifying information should have the same GUID.
    • Two items with different identifying information should have different GUIDs.
    • None of these GUIDs should collide with GUIDs generated by any other means.

    For that, you can use a name-based GUID generation algorithm.

  • The Old New Thing

    If more than one object causes a WaitForMultipleObjects to return, how do I find out about the other ones?

    • 23 Comments

    There is often confusion about the case where you call Wait­For­Multiple­Objects (or its variants), passing bWaitAll = FALSE, and more than one object satisfies the wait.

    When bWaitAll is FALSE, this function checks the handles in the array in order starting with index 0, until one of the objects is signaled. If multiple objects become signaled, the function returns the index of the first handle in the array whose object was signaled.

    The function modifies the state of some types of synchronization objects. Modification occurs only for the object or objects whose signaled state caused the function to return.

    The worry here is that you pass bWaitAll = FALSE, and two objects satisfy the wait, and they are both (say) auto-reset events.

    The first paragraph says that the return value tells you about the event with the lowest index. The second paragraph says that the function modifies the state of the "object or objects" whose signaled state caused the function to return.

    The "or objects" part is what scares people. If two objects caused the function to return, and I am told only about one of them, how do I learn about the other one?

    The answer is, "Don't worry; it can't happen."

    If you pass bWaitAll = FALSE, then only one object can cause the function to return. If two objects become signaled, then the function declares that the lowest-index one is the one that caused the function to return; the higher-index ones are considered not have have caused the function to return.

    In the case of the two auto-reset events: If both events are set, one of them will be chosen as the one that satisfies the wait (the lower-index one), and it will be reset. The higher-index one remains unchanged.

    The confusion stems from the phrase "object or objects", causing people to worry about the case where bWaitAll = FALSE and there are multiple objects which cause the function to return.

    The missing detail is that when you pass bWaitAll = FALSE, then at most one object can cause the function to return. ("At most", because if the operation times out, then no object caused the function to return.)

    The presence of the phrase "or objects" is to cover the case where you pass bWaitAll = TRUE.

  • The Old New Thing

    Why is the System.DateCreated property off by a few seconds?

    • 38 Comments

    If you ask for the System.Date­Created property of a shell item, the timestamp that comes back is up to two seconds different from the file's actual timestamp. (Similarly for System.Date­Modified and System.Date­Accessed.) Why is that?

    This is an artifact of a decision taken in 1993.

    In general, shell namespace providers cache information in the ID list at the time the ID list is created so that querying basic properties from an item can be done without accessing the underlying medium.

    In 1993, saving 4KB of memory had a measurable impact on system performance. Therefore, bytes were scrimped and saved, and one place where four whole bytes were squeezed out was in the encoding of file timestamps in ID lists. Instead of using the 8-byte FILE­TIME structure, the shell used the 4-byte DOS date-time format. Since the shell created thousands of ID lists, a four-byte savings multiplied over thousands of items comes out to several kilobytes of data.

    But one of the limitations of the DOS date-time format is that it records time in two-second increments, so any timestamp recorded in DOS date-time format can be up to two seconds away from its actual value. (The value is always truncated rather than rounded in order to avoid problems with timestamps from the future.) Since Windows 95 used FAT as its native file system, and FAT uses the DOS date-time format, this rounding never created any problems in practice, since all the file timestamps were already pre-truncated to 2-second intervals.

    Of course, Windows NT uses NTFS as the native file system, and NTFS records file times to 100-nanosecond precision. (Though the accuracy is significantly less.) But too late. The ID list format has already been decided, and since ID lists can be saved to a file and transported to another computer (e.g. in the form of a shortcut file), the binary format cannot be tampered with. Hooray for compatibility.

    Bonus chatter: In theory, the ID list format could be extended in a backward-compatible way, so that every ID list contained two timestamps, a compatible version (2-second precision) and a new version (100-nanosecond precision). So far, there has not been significant demand for more accurate timestamps inside of ID lists.

  • The Old New Thing

    The details of the major incident were not clearly articulated, but whatever it is, it's already over

    • 15 Comments
    When a server is taken offline, be it a planned unplanned outage or an unplanned unplanned outage or something else, the operations team send out a series of messages alerting customers to the issue.

    Some time ago, I received a notification that went like this:

    From: Adam Smith
    Subject: Nosebleed Service : Major Incident Notification - Initial
    Date: mm/dd/yyyy 1:16AM

    Major Incident Notification

    dfdsfsd

    Affected Users

    fdfsdfsdf

    Start: mm/dd/yyyy 12:00AM Pacific Standard Time
    mm/dd/yyyy 8:00AM UTC
    End: No ETA at this time.

    Incident Duration: 1 hour 15 minutes

    Impact

    fsdfdsfsdf

    Continued Notifications

    fdsfsdf

    Information & Support

    • Other Support: Please send questions or feedback to

    Thank you,

    Adam Smith
    IT Major Incident Management

    Well that clears things up.

    Curiously, the message includes an incident duration but doesn't have an ETA. Thankfully, the message was sent one minute after the incident was over, so by the time I got it, everything was back to normal.

  • The Old New Thing

    Opening the classic folder browser dialog with a specific folder preselected

    • 24 Comments

    Today's Little Program shows how to set the initial selection in the SHBrowse­For­Folder dialog.

    The design of the SHBrowse­For­Folder function had a defect: The BROWSEINFO structure doesn't have a cbSize member at the start. This means that the structure cannot ever change because the function would have no way of knowing whether you are calling with the old structure or the new one. If it weren't for this defect, setting the initial selection would have been easy: Add a pidlInitialSelection member to the structure and have people fill it in.

    Alas, any new functionality in the SHBrowse­For­Folder function therefore requires that the new functionality be expressed within the constraints of the existing structure.

    Fortunately, there's a callback that takes a message number.

    The workaround, therefore, is to express any new functionalty in the form of new callback messages.

    And that's how the ability to set the initial selection in the folder browser dialog came about. A new message BFFM_INITIALIZED was created, and in handling that message, the callback can specify what it wants the selection to be.

    #define UNICODE
    #define _UNICODE
    #define STRICT_TYPED_ITEMIDS
    #include <windows.h>
    #include <ole2.h>
    #include <oleauto.h>
    #include <shlobj.h>
    #include <stdio.h> // horrors! Mixing C and C++!
    
    int CALLBACK Callback(
        HWND hwnd, UINT uMsg, LPARAM lParam, LPARAM lpData)
    {
     switch (uMsg) {
     case BFFM_INITIALIZED:
      SendMessage(hwnd, BFFM_SETSELECTION, TRUE,
                  reinterpret_cast<LPARAM>(L"C:\\Windows"));
      break;
     }
     return 0;
    }
    
    int __cdecl wmain(int, wchar_t **)
    {
     CCoInitialize init;
     TCHAR szDisplayName[MAX_PATH];
     BROWSEINFO info = { };
     info.pszDisplayName = szDisplayName;
     info.lpszTitle = TEXT("Pick a folder");
     info.ulFlags = BIF_RETURNONLYFSDIRS;
     info.lpfn = Callback;
     PIDLIST_ABSOLUTE pidl = SHBrowseForFolder(&info);
     if (pidl) {
      SHGetPathFromIDList(pidl, szDisplayName);
      wprintf(L"You chose %ls\n", szDisplayName);
      CoTaskMemFree(pidl);
     }
     return 0;
    }
    

    We initialize COM and then call the SHBrowse­For­Folder function with information that includes a callback. The callback responds to the BFFM_INITIALIZED message by changing the selection.

    It's not hard, but it's a bit cumbersome.

    Sorry.

    Bonus chatter: The presence of the callback means that the function cannot shunt the work to a new thread when called from an MTA thread because you are now stuck with the problem of which thread the callback should run on.

    • The callback may want to access resources that belong to the original thread, so that argues for the callback being run on the original thraed.
    • The callback may also want to access resources inside the dialog box, say in order to customize it. That argues for the callback being run on the new thread.

    You can't have it both ways, so you're stuck.

    But it doesn't really matter, because you shouldn't be performing UI from a multi-threaded apartment anyway. There's not much point in making it easier for people to do the wrong thing.

  • The Old New Thing

    Why is CreateToolhelp32Snapshot returning incorrect parent process IDs all of a sudden?

    • 28 Comments

    A customer reported a problem with the Create­Toolhelp32­Snapshot function.

    From a 32-bit process, the code uses Create­Toolhelp32­Snapshot and Process32­First/Process32­Next to identify parent processes on a 64-bit version of Windows. Sporadically, we find that the th32Parent­Process­ID is invalid on Windows Server 2008. This code works fine on Windows Server 2003. Here's the relevant fragment:

    std::vector<int> getAllChildProcesses(int pidParent)
    {
     std::vector<int> children;
    
     HANDLE snapshot = CreateToolhelp32Snapshot(
        TH32CS_SNAPPROCESS, 0);
     if (snapshot != INVALID_HANDLE_VALUE) {
      PROCESSENTRY32 entry;
      entry.dwSize = sizeof(entry); // weird that this is necessary
      if (Process32First(snapshot, &entry)) {
       do {
        if (entry.th32ParentProcessID == pidParent) {
         children.push_back(processEntry.th32ProcessID);
        } while (Process32Next(snapshot, &entry));
      }
      CloseHandle(snapshot);
     }
     return children;
    }
    

    (The customer snuck another pseudo-question in a comment. Here's why it is necessary.)

    One of my colleagues asked what exactly was "invalid" about the process IDs. (This is like the StackOverflow problem where somebody posts some code and says simply "It doesn't work".)

    My colleague also pointed out that the thParent­Process­ID is simply a snapshot of the parent process ID at the time the child process was created. Since process IDs can be recycled, once the parent process exits, the process ID is left orphaned, and it may get reassigned to another unrelated process. For example, consider this sequence of events:

    • Process A creates Process B.
    • Process A terminates, thereby releasing its ID for reuse.
    • Process C is created.
    • Process C reuses Process A's process ID.

    At this point, Process B will have a th32Parent­Process­ID equal to Process A, but since the ID for Process A has been reused for Process C, it will also be equal to Process C, even though there is no meaningful relationship between processes B and C.

    If Process B needs to rely on its parent process ID remaining assigned to that process (and not getting reassigned), it needs to maintain an open handle to the parent process. (To avoid race conditions, this should be provided by the parent itself.) An open process handle prevents the process object from being destroyed, and in turn that keeps the process ID valid.

    There is another trick of checking the reported parent process's creation time and seeing if it is more recent than the child process's creation time. If so, then you are a victim of process ID reuse, and the true parent process is long gone. (This trick has its own issues. For example, you may not have sufficient access to obtain the parent process's creation time.)

    After a few days, the customer liaison returned with information from the customer. It looks like all of the guidance and explanation provided by my colleague either never made it to the customer, or the customer simply ignored it.

    The customer wants to detect what child processes are spawned by a particular application, let's call it P. We built a special version with extra logging, and it shows that the PROCESS­ENTRY32.th32Parent­Process­ID for wininit.exe and csrss.exe were both 0x15C, which is P's process ID. This erroneous reporting occurs while P is still running and continues after P exits. Do you think it's possible that process 0x15C was used by some other process earlier?

    Yes, that possible. That is, in fact, what my colleague was trying to explain.

    It isn't clear why the customer is trying to track down all child processes of process P, but the way to do this is to create a job object and put process P in it. You can then call Query­Information­Job­Object with Job­Object­Basic­Process­Id­List to get the list of child processes.

Page 4 of 450 (4,496 items) «23456»