• The Old New Thing

    What does the FOF_NOCOPYSECURITYATTRIBS flag really do (or not do)?


    In the old days, the shell copy engine didn't pay attention to ACLs. It just let the file system do whatever the default file system behavior was. The result was something like this:

    • If you copied a file, it opened the destination, wrote to it, and that was it. Result: The copied file has the security attributes of destination (specifically, picking up the inheritable attributes from the container).
    • If you moved a file within the same drive, it moved the file with MoveFile, and that was it. Result: The file retained its security attributes.
    • If you moved a file between drives, then it was treated as a copy/delete. Result: The moved file has the security attributes of the destination (specifically, picking up the inheritable attributes from the container).

    Perfectly logical, right? If a new file is created, then the security attributes are inherited from the container. If an existing file is moved, then its security attributes move with it. And since moving a file across drives was handled as a copy/delete, moving a file across drives behaved like a copy.

    Users, however, found this behavior confusing. For example, they would take a file from a private folder like their My Documents folder, and move it to a public location like Common Documents, and... the file would still be private.

    The FOF_NO­COPY­SECURITY­ATTRIBS flag was introduced in Windows 2000 to address this confusion. If you pass this flag, then when you move a file, even within a drive, the security attributes of the moved file will match the destination directory. (The way the shell implements this flag, by the way, is to move the file like normal, and then reset the security attributes to match the destination. So even though it sounds like a flag that says "don't do X" would be less work than doing X, it's actually more work, because we actually do X+Y and then undo the X part. But it's still far cheaper than copying the file and deleting the original.)

    Note that omitting the FOF_NO­COPY­SECURITY­ATTRIBS flag does not mean "Always copy security attributes." If you don't pass the flag, then the security attributes follow the default file system behavior, which sometimes transfers the security attributes and sometimes doesn't. In retrospect, the flag might have been better-named something like FOF_SET­SECURITY­ATTRIBS­TO­MATCH­DESTINATION.

    Now the question is how to summarize all this information in the MSDN documentation for the FOF_NO­COPY­SECURITY­ATTRIBS flag? After receiving this explanation of how the flag works, one customer suggested that the text be changed to read "Do not copy the security attributes of the moved file. The destination file receives the security attributes of its new folder. Note that this flag has no effect on copied files, which will always receive the security attributes of the new folder."

    But this proposed version actually can be misinterpreted. Everything starting with "Note that" is intended to be guidance. It isn't actually part of the specification; rather, it's sort of "thinking out loud", taking the actual specification and calling out some of its consequences. But how many people reading the above proposed text would fail to realize that the first two sentences are normative but the third sentence is interpretive? In particular, the interpretation says that the copied file will "always" receive the security attributes of the new folder. Is that really true? Maybe in the future there will be a new flag like COPY_FILE_INCLUDE_SECURITY_ATTRIBUTES, and now the "always" isn't so "always" any more.

    Anyway, now that you know what the FOF_NO­COPY­SECURITY­ATTRIBS flag does (and doesn't do), maybe you can answer this customer's question:

    Download a file via Internet Explorer and put it on the desktop. The file will be marked as having come from the Internet Zone.

    Now copy the file with the FOF_NO­COPY­SECURITY­ATTRIBS to some other location.

    The resulting file is still marked as Internet Zone. I expected that FOF_NO­COPY­SECURITY­ATTRIBS would remove the Internet Zone security information. Is this a bug in SHFileOperation?

    (This article provides enough information for you to explain why the Internet Zone marker is not removed. The answer to the other half of the customer's question—actually removing the marker—lies in this COM method.)

  • The Old New Thing

    The memcmp function reports the result of the comparison at the point of the first difference, but it can still read past that point


    This story originally involved a more complex data structure, but that would have required too much explaining (with relatively little benefit since the data structure was not related to the moral of the story), so I'm going to retell it with double null-terminated strings as the data structure instead.

    Consider the following code to compare two double-null-terminated strings for equality:

    size_t SizeOfDoubleNullTerminatedString(const char *s)
      const char *start = s;
      for (; *s; s += strlen(s) + 1) { }
      return s - start + 1;
    BOOL AreDoubleNullTerminatedStringsEqual(
        const char *s, const char *t)
     size_t slen = SizeOfDoubleNullTerminatedString(s);
     size_t tlen = SizeOfDoubleNullTerminatedString(t);
     return slen == tlen && memcmp(s, t, slen) == 0;

    "Aha, this code is inefficient. Since the memcmp function stops comparing as soon as it finds a difference, I can skip the call to SizeOfDoubleNullTerminatedString(t) and simply write

    BOOL AreDoubleNullTerminatedStringsEqual(
        const char *s, const char *t)
     return memcmp(s, t, SizeOfDoubleNullTerminatedString(s)) == 0;

    because we can never read past the end of t: If the strings are equal, then tlen will be equal to slen anyway, so the buffer size is correct. And if the strings are different, the difference will be found at or before the end of t, since it is not possible for a double-null-terminated string to be a prefix of another double-null-terminated string. In both cases, we never read past the end of t."

    This analysis is based on a flawed assumption, namely, that memcmp compares byte-by-byte and does not look at bytes beyond the first point of difference. The memcmp function makes no such guarantee. It is permitted to read all the bytes from both buffers before reporting the result of the comparison.

    In fact, most implementations of memcmp do read past the point of first difference. Your typical library will try to compare the two buffers in register-sized chunks rather than byte-by-byte. (This is particularly convenient on x86 thanks to the block comparison instruction rep cmpsd which compares two memory blocks in DWORD-sized chunks, and x64 doubles your fun with rep cmpsq.) Once it finds two chunks which differ, it then studies the bytes within the chunks to determine what the return value should be.

    (Indeed, people with free time on their hands or simply enjoy a challenge will try to outdo the runtime library with fancy-pants memcmp algorithms which compare the buffers in larger-than-normal chunks by doing things like comparing via SIMD registers.)

    To illustrate, consider an implementation of memcmp which uses 4-byte chunks. Typically, memory comparison functions do some preliminary work to get the buffers aligned, but let's ignore that part since it isn't interesting. The inner loop goes like this:

    while (length >= 4)
     int32 schunk = *(int32*)s;
     int32 tchunk = *(int32*)t;
     if (schunk != tchunk) {
       -- difference found - calculate and return result
     length -= 4;
     s += 4;
     t += 4;
    Let's compare the strings s = "a\0b\0\0" and t = "a\0\0". The size of the double-null-terminated string s is 4, so the memory comparison goes like this: First we read four bytes from s into schunk, resulting in (on a little-endian machine) 0x00620061. Next, we read four bytes from t into tchunk, resulting in 0x??000061. Oops, we read one byte past the end of the buffer.

    If t happened to sit right at the end of a page, and the next page was uncommitted memory, then you take an access violation while trying to read tchunk. Your optimization turned into a crash.

    Remember, when you say that a buffer is a particular size, the basic ground rules of programming say that it really has to be that size.

  • The Old New Thing

    How do I get the color depth of the screen?


    How do I get the color depth of the screen? This question already makes an assumption that isn't always true, but we'll answer the question first, then discuss why the answer is wrong.

    If you have a device context for the screen, you can query the color depth with a simple arithmetic calculation:

    colorDepth = GetDeviceCaps(hdc, BITSPIXEL) *
                 GetDeviceCaps(hdc, PLANES);

    Now that you have the answer, I'll explain why it's wrong, but you can probably guess the reason already.

    Two words: Multiple monitors.

    If you have multiple monitors connected to your system, each one can be running at a different color depth. For example, your primary monitor might be running at 32 bits per pixel, while the secondary is stuck at 16 bits per pixel. When there was only one monitor, there was such a thing as the color depth of the screen, but when there's more than one, you first have to answer the question, "Which screen?"

    To get the color depth of each monitor, you can take your device context and ask the window manager to chop the device context into pieces, each corresponding to a different monitor.

    EnumDisplayMonitors(hdc, NULL, MonitorEnumProc, 0);
    // this function is called once for each "piece"
    BOOL CALLBACK MonitorEnumProc(HMONITOR hmon, HDC hdc,
                                  LPRECT prc, LPARAM lParam)
       // compute the color depth of monitor "hmon"
       int colorDepth = GetDeviceCaps(hdc, BITSPIXEL) *
                        GetDeviceCaps(hdc, PLANES);
       return TRUE;

    If you decide to forego splitting the DC into pieces and just ask for "the" color depth, you'll get the color depth information for the primary monitor.

    As a bonus (and possible optimization), there is a system metric GetSystemMetrics(SM_SAMEDISPLAYFORMAT) which has a nonzero value if all the monitors in the system have the same color format.

  • The Old New Thing

    Why are the keyboard scan codes for digits off by one?


    In Off by one what, exactly?, my colleague Michael Kaplan wrote

    And this decision long ago that caused the scan codes to not line up for these digits when they could have...

    The word that struck me there was "decision".

    Because it wasn't a "decision" to make the scan codes almost-but-not-quite line up with digits. It was just a coincidence.

    If you look at the scan code table from Michael's article

    you can see stretches of consecutive scan codes, broken up by weird places where the consecutive pattern is violated. The weirdness makes more sense when you look at the original IBM PC XT keyboard:


    With this presentation, it becomes clearer how scan codes were assigned: They simply started at 01 and continued through the keyboard in English reading order. (Scan code 00 is an error code indicating keyboard buffer overflow.) The reason for the keyboard scan code being off-by-one from the digits is merely due to the fact that there was one key to the left of the digits. If there were two keys to the left of the digits, they would have been off by two.

    Of course, if the original keyboard designers had started counting from the lower left corner, like all right-thinking mathematically-inclined people, then this sort-of-coincidence would never have happened. The scan codes for the digits would have been 2E through 37, and nobody would have thought anything of it.

    It's a testament to the human brain's desire to find patterns and determine a reason for them that what is really just a coincidence gets interpreted as some sort of conspiracy.

  • The Old New Thing

    Why does each drive have its own current directory?


    Commenter Dean Earley asks, "Why is there a 'current directory' AND an current drive? Why not merge them?"

    Pithy answer: Originally, each drive had its own current directory, but now they don't, but it looks like they do.

    Okay, let's unwrap that sentence. You actually know enough to answer the question yourself; you just have to put the pieces together.

    Set the wayback machine to DOS 1.0. Each volume was represented by a drive letter. There were no subdirectories. This behavior was carried forward from CP/M.

    Programs from the DOS 1.0 era didn't understand subdirectories; they referred to files by just drive letter and file name, for example, B:PROGRAM.LST. Let's fire up the assembler (compilers were for rich people) and assemble a program whose source code is on the A drive, but sending the output to the B drive.

    A>asm foo       the ".asm" extension on "foo" is implied
    Assembler version blah blah blah
    Source File: FOO.ASM
    Listing file [FOO.LST]: NUL throw away the listing file
    Object file [FOO.OBJ]: B: send the object file to drive B

    Since we gave only a drive letter in response to the Object file prompt, the assembler defaults to a file name of FOO.OBJ, resulting in the object file being generated as B:FOO.OBJ.

    Okay, now let's introduce subdirectories into DOS 2.0. Suppose you have want to assemble A:\SRC\FOO.ASM and put the result into B:\OBJ\FOO.OBJ. Here's how you do it:

    A> B:
    B> CD \OBJ
    B> A:
    A> CD \SRC
    A> asm foo
    Assembler version blah blah blah
    Source File: FOO.ASM
    Listing file [FOO.LST]: NUL
    Object file [FOO.OBJ]: B:

    The assembler reads from A:FOO.ASM and writes to B:FOO.OBJ, but since the current directory is tracked on a per-drive basis, the results are A:\SRC\FOO.ASM and B:\OBJ\FOO.OBJ as desired. If the current directory were not tracked on a per-drive basis, then there would be no way to tell the assembler to put its output into a subdirectory. As a result, DOS 1.0 programs were effectively limited to operating on files in the root directory, which means that nobody would put files in subdirectories (because their programs couldn't access them).

    From a DOS 1.0 standpoint, changing the current directory on a drive performs the logical equivalent of changing media. "Oh look, a completely different set of files!"

    Short attention span.

    Remembering the current directory for each drive has been preserved ever since, at least for batch files, although there isn't actually such a concept as a per-drive current directory in Win32. In Win32, all you have is a current directory. The appearance that each drive has its own current directory is a fake-out by cmd.exe, which uses strange environment variables to create the illusion to batch files that each drive has its own current directory.

    Dean continues, "Why not merge them? I have to set both the dir and drive if i want a specific working dir."

    The answer to the second question is, "They already are merged. It's cmd.exe that tries to pretend that they aren't." And if you want to set the directory and the drive from the command prompt or a batch file, just use the /D option to the CHDIR command:

    D:\> CD /D C:\Program Files\Windows NT
    C:\Program Files\Windows NT> _

    (Notice that the CHDIR command lets you omit quotation marks around paths which contain spaces: Since the command takes only one path argument, the lack of quotation marks does not introduce ambiguity.

  • The Old New Thing

    Why does TaskDialog return immediately without showing a dialog? - Answer


    Last time, I left an exercise to determine why the Task­Dialog function was not actually displaying anything. The problem had nothing to do with an invalid window handle parameter and had all to do with original window being destroyed.

    My psychic powers told me that the window's WM_DESTROY handler called Post­Quit­Message. As we learned some time ago, quit messages cause modal loops to exit. Since the code was calling Task­Dialog after the window was destroyed, there was a WM_QUIT message still sitting in the queue, and that quit message caused the modal loop in Task­Dialog to exit before it got a chance to display anything.

    Switching to Message­Box wouldn't have changed anything, since Message­Box responds to quit messages the same way as Task­Dialog.

    (Worf was the first person to post the correct answer.)

  • The Old New Thing

    Why does my asynchronous I/O request return TRUE instead of failing with ERROR_IO_PENDING?


    A customer reported that their program was not respecting the FILE_FLAG_OVERLAPPED flag consistently:

    My program opens a file handle in FILE_FLAG_OVERLAPPED mode, binds it to an I/O completion callback function with Bind­Io­Completion­Callback, and then issues a Write­File against it. I would expect that the Write­File returns FALSE and Get­Last­Error() returns ERROR_IO_PENDING, indicating that the I/O operation is being performed asynchronously, and that the completion function will be called when the operation completes. However, I find that some percentage of the time, the call to Write­File returns TRUE, indicating that the operation was performed synchronously. What am I doing wrong? I don't want my thread to block on I/O; that's why I'm issuing asynchronous I/O.

    When you specify FILE_FLAG_OVERLAPPED, you're promising that your program knows how to handle I/O which completes asynchronously, but it does not require the I/O stack to behave asynchronously. A driver can choose to perform your I/O synchronously anyway. For example, if the write operation can be performed by writing to cache without blocking, the driver will just copy the data to the cache and indicate synchronous completion. Don't worry, be happy: Your I/O completed even faster than you expected!

    Even though the I/O completed synchronously, all the asynchronous completion notification machinery is still active. It's just that they all accomplished their job before the Write­File call returned. This means that the event handle will still be signaled, the completion routine will still run (once you wait alertably), and if the handle is bound to an I/O completion port, the I/O completion port will receive a completion notification.

    You can use the Set­File­Completion­Notification­Modes function to change some aspects of this behavior, giving some control of the behavior of the I/O subsystem when a potentially-asynchronous I/O request completes synchronously.

  • The Old New Thing

    The overlooked computer room at school that became my "office" for a while


    VIMH's comment on an unmarked restroom that is largely unknown reminds me of a story from my college days.

    Since my final project involved computational geometry, I was granted a key to the rooms in our department which had the computers with fancy graphical displays. (The term "fancy graphical display" is a relative one, mind you. By today's standards they would be pretty lame.) Use of the computers in these rooms was normally reserved for faculty and graduate students. During my wanderings through the department building, I discovered that there was a small storage room in an unused corner of the basement that contained not only the boxes piled high, like you might expect, but also one graphics display terminal.

    I was pleased at my discovery and even more pleased to discover over time that nobody ever came to visit. I had stumbled across the forgotten computer room.

    After a few weeks, I moved in a small tape cassette recorder (that being the fanciest audio technology I could afford at the time) so I could listen to music while I worked. Rachmaninoff's Third Piano Concerto became the mental soundtrack to my final project. Initially, I stowed the tape recorder in the corner when I left the room, but I gradually became lazy and just left it on the table next to the computer.

    This is normally the part of the story where our hero's casual mistake leads to his downfall: A custodian discovers the tape recorder, reports it to the administrator, and our hero is kicked out of the department for misuse of school facilities.

    But that's not what happened. As far as I remember, there was only one time another person paid a visit to the overlooked computer room while I was working in there. He jiggled the door handle, found it locked, and waved apologetically. (He was probably not even somebody authorized to be in the room, because if he were, he would have had a key.)

  • The Old New Thing

    Why does TaskDialog return immediately without showing a dialog?


    A customer reported a problem with the Task­Dialog function.

    We've encountered a strange behavior in the Task­Dialog function. A user reported that when exiting our application, our program played an error beep that didn't appear to be associated with an error. After investigating, we found that the sound is coming from our application trying to display an error dialog before closing by calling Task­Dialog. The error beep is played but no dialog appears.

    Some background on the error condition that we're trying to report: We're calling Create­Window­Ex, and the window procedure creates another window in its WM_CREATE handler. It looks like the original window is destroyed before WM_CREATE is finished, so Create­Window­Ex returns ERROR_INVALID_WINDOW_HANDLE. It's not clear why this is causing Task­Dialog to fail, but this is the only scenario where we see this behavior. All other calls to Task­Dialog work as expected. We know how to fix the original problem with Create­Window­Ex, but we would like to understand what's going on with Task­Dialog in case there's another bug here.

    With a little bit of psychic powers, you can solve this customer's problem too.

    (If you have a good memory, you may have noticed that it's a variation on a previous puzzle. But I get to keep recycling it because every year, a new batch of developers stumbles across the problem.)

  • The Old New Thing

    Wildly popular computer game? The Windows product team has you covered


    In Windows 95, the most heavily-tested computer game was DOOM. Not because the testers spent a lot of time developing test plans and test harnesses and automated run-throughs. Nope, it was because it was by far the most popular game the Windows 95 team members played to unwind.

    It was a huge breakthrough when DOOM finally ran inside a MS-DOS box and didn't require booting into MS-DOS mode any more. Now you could fire up DOOM without having to exit all your other programs first.

    I've learned that in Windows Vista, the most heavily tested game was World of Warcraft. Most members of the DirectX application compatibility team are WoW players, in addition to a large number in the Windows division overall.

    So if you have a wildly popular computer game for the PC, you can be pretty sure that the Windows team will be all over it. "For quality control purposes, I assure you."

    Related story: How to make sure your network card works with Windows.

Page 121 of 426 (4,251 items) «119120121122123»