• The Old New Thing

    It rather involved being on the other side of this airtight hatchway: Surreptitious file access by administrator


    A security report was received that went something like this:

    A user can bypass file sharing locks by opening a read-only handle to the physical volume containing the file in question. This allows the user to extract the contents of protected files by reading the corresponding sectors directly from the disk. Since this operation requires administrator access, any user with administrator access can extract data from files that are normally inaccessible due to file locks, such as the SAM database.

    Yes, that's right. An attacker who gains administrator privileges can extract data from any file on the computer.

    But so what? The attacker is already on the other side of the airtight hatchway. They already pwn your machine. That a pwned machine can be pwned is not really all that surprising.

    That some files are not accessible due to file locks is not a security measure. It is a consequence of, um, file access.

    Besides, once you gain administrator access, a much easier way to steal the SAM is to merely grab a backup copy.

    What, you can't find a backup copy?

    No problem.

    After all, you're the administrator. One of your job responsibilities is to maintain regular system backups.

    So create a backup of the SAM file. Of course the system will let you do this. It is your job after all.

    For example, you can use the Volume Shadow Service to create a volume snapshot, then mount the snapshot and extract the SAM file.

    Bingo, instant copy of the SAM database.

    Just doing your job.

  • The Old New Thing

    Redirecting the Favorites folder is technically legal, but you wouldn't like it


    A customer liaison asked for assistance in debugging why Internet Explorer frequently stops responding at their customer's site. "We have tried a number of things like clearning the temporary Internet files, disabling add-ons, and resetting Internet Explorer settings. Please let me know if you can provide guidance based on the dump files provided below to indicate what could be causing Internet Explorer to hang here."

    The dump file showed 23 threads, and all of them seemed to be normal, except for one which looked like this:


    (Remember, you can get symbols for operating system binaries.)

    General debugging tip: If you see a really huge stack, that's a good sign that something interesting is going on. Boring stacks tend to be small.

    Furthermore, frames near the bottom of the stack tend to describe what the purpose of the thread is, whereas frames near the top of the stack tend to describe what the thread is actually doing right now. (Exercise: Why?)

    In this case, we see a stack that was probably created to manage a tab window (CTab­Window::_Tab­Window­Thread­Proc) and it's currently stuck in an I/O operation. You can then look at the file name to see what file is stuck.

    0:001> du 04cd6aac
    04cd6aac "\\server\share\abcdefg\Favorites\Mail.url"

    It looks like this user stored their Favorites on a network share that is not responding.

    The customer liaison replied,

    Thanks a lot for this information. Can you help me understand how do we tell that the dump indicates this file I/O is causing IE to hang? Having this information would help me better explain this to the customer.

    I wasn't sure how to respond to this. If you see a function with the words File and one of Open, Read, or Write in its name, there's a good chance that it opens, reads, or writes a file. You probably want to look to see what file is being opened, read from, or written to, because that may give a clue why the I/O operation is stuck.

    It turns out that this customer redirected the user's Favorites to a network location. The Internet Explorer folks tell me that this is not an explicitly supported scenario in the sense that they did not do any tuning to make this scenario work well, and frequent hangs are consequently not unexpected. If you redirect the Favorites to a network location, then you get what you get. And if that server frequently becomes unavailable, then what you get often sucks.

  • The Old New Thing

    What is the default size of the Recycle Bin, and how can an administrator control the size of the Recycle Bin?


    A customer was setting up a file server to which they intended to redirect all their employees' documents. They were concerned about the amount of disk space used by all the Recycle Bins on that server.

    Is there a fixed default size for the Recycle Bin, or is it based on the size of the disk? Is there a way we can change the default size for the Recycle Bin?

    The customer is concerned that a user with a small quota on a large drive may end up filling their quota with Recycle Bin content and have no space left for their documents. For example, suppose you have a 1TB drive and each user has a 15 GB quota. If the Recycle Bin were based on disk size, and the Recycle Bin were set to use five percent of the disk, then that would give each user 5% × 1 TB = 51.2 GB of Recycle Bin, which is larger than their quota. Users can fill their Recycle Bin and have no room for documents!

    Fortunately, it doesn't work that way. The Recycle Bin calculations are always based on the disk quota, not the disk size. In the above example, each user's Recycle Bin would be 5% × 15 GB = 768 MB.

    Now as to what the default is, that's a trickier question.

    Up through Windows XP, the default Recycle Bin was ten percent of the user's quota on the underlying volume. Starting in Windows Vista, the algorithm was tweaked, and the default size is ten percent of the first 40 GB of quota, and five percent of any quota above 40 GB. (Note that future versions of Windows may tweak the defaults. This is provided for informational purposes, not contractual.)

    If you don't like the default, you can set an explicit maximum value by policy. If you are willing to live a little dangerously, you can dig under the covers and tweak values on a per-folder basis. Note that once you dig under the covers, you are in unsupported territory, so if it stops working (or starts misbehaving), then you're on your own.

  • The Old New Thing

    2014 mid-year link clearance


    Another round of the semi-annual link clearance.

    James Mickens section

    January 2014: This World of Ours

    I have seen a video called “Gigantic Martian Insect Party,” and I have seen another video called “Gigantic Martian Insect Party 2: Don’t Tell Mom,” and I hated both videos, but this did not stop me from directing the sequel “Gigantic Martian Insect Party Into Darkness.”

    March 2014: To Wash It All Away

    Clearing the cache to fix a Web browser is like when your dad was driving you to kindergarten, and the car started to smoke, and he tried to fix the car by banging on the hood three times and then asking you if you could still smell the carbon monoxide, and you said, “Yeah, its better,” because you didn’t want to expose your dad as a fraud, and then both of you rode to school in silence as you struggled to remain conscious.

    And a recorded presentation: Computers are a Sadness, I am the Cure, which is hilarious because it's true, especially his final security recommendation.

  • The Old New Thing

    Getting the location of the Close button in the title bar, from Windows 2000 or Windows XP


    Today's Little Program locates the × button in the corner of the window and displays a balloon tip pointing at it. We did this some time ago with the help of the WM_GET­TITLE­BAR­INFO­EX message, which is new for Windows Vista. But what if you don't have that message available, say, because you're running on Windows 2000 or Windows XP or (gasp) Windows 98?

    You can use the classic Accessibility interface IAccessible to enumerate the buttons in the title bar and see which one the window reports as the Close button.

    Let's take the program from last time and change the Get­Close­Button­Center function:

    #include <oleacc.h>
    #include <atlbase>
    BOOL GetCloseButtonCenter(HWND hwnd, POINT *ppt)
     CComPtr<IAccessible> spacc;
     if (FAILED(AccessibleObjectFromWindow(hwnd, OBJID_TITLEBAR,
                       IID_PPV_ARGS(&spacc)))) return FALSE;
     CComQIPtr<IEnumVARIANT> spenum(spacc);
     if (!spenum) return FALSE;
     for (CComVariant vtChild; spenum->Next(1, &vtChild, nullptr) == S_OK;
          vtChild.Clear()) {
      CComVariant vtState;
      if (FAILED(spacc->get_accState(vtChild, &vtState))) continue;
      if (vtState.vt != VT_I4) continue;
      if (vtState.lVal & (STATE_SYSTEM_INVISIBLE |
                          STATE_SYSTEM_OFFSCREEN |
                          STATE_SYSTEM_UNAVAILABLE)) continue;
      long left, top, width, height;
      if (FAILED(spacc->accLocation(&left, &top, &width, &height,
                                    vtChild))) continue;
      POINT pt = { left + width / 2, top + height / 2 };
      if (SendMessage(hwnd, WM_NCHITTEST, 0,
                      MAKELPARAM(pt.x, pt.y)) == HTCLOSE) {
       *ppt = pt;
       return TRUE;
     return FALSE;

    We obtain the IAccessible interface for the title bar and proceed to enumerate its children. For each child, we get its location, and then use the WM_NC­HIT­TEST message to determine programmatically what that location corresponds to. If the answer is "This is the Close button," then we found the button and report its center.

    Note that this once again highlights the distinction between WM_NC­MOUSE­MOVE and WM_NC­HIT­TEST. Hit-testing can occur for reasons other than mouse movement.

    Exercise: Why couldn't we use the IAccessible::get_accName method to figure out which button each child represents?

  • The Old New Thing

    Undefined behavior can result in time travel (among other things, but time travel is the funkiest)


    The C and C++ languages are notorious for the very large section of the map labeled here be dragons, or more formally, undefined behavior.

    When undefined behavior is invoked, anything is possible. For example, a variable can be both true and false. John Regehr has a list of interesting examples, as well as some winners of the ensuing contest.

    Consider the following function:

    int table[4];
    bool exists_in_table(int v)
        for (int i = 0; i <= 4; i++) {
            if (table[i] == v) return true;
        return false;

    What does this have to do with time travel, you ask? Hang on, impatient one.

    First of all, you might notice the off-by-one error in the loop control. The result is that the function reads one past the end of the table array before giving up. A classical compiler wouldn't particularly care. It would just generate the code to read the out-of-bounds array element (despite the fact that doing so is a violation of the language rules), and it would return true if the memory one past the end of the array happened to match.

    A post-classical compiler, on the other hand, might perform the following analysis:

    • The first four times through the loop, the function might return true.
    • When i is 4, the code performs undefined behavior. Since undefined behavior lets me do anything I want, I can totally ignore that case and proceed on the assumption that i is never 4. (If the assumption is violated, then something unpredictable happens, but that's okay, because undefined behavior grants me permission to be unpredictable.)
    • The case where i is 5 never occurs, because in order to get there, I first have to get through the case where i is 4, which I have already assumed cannot happen.
    • Therefore, all legal code paths return true.

    As a result, a post-classical compiler can optimize the function to

    bool exists_in_table(int v)
        return true;

    Okay, so that's already kind of weird. A function got optimized to basically nothing due to undefined behavior. Note that even if the value isn't in the table (not even in the illegal-to-access fifth element), the function will still return true.

    Now we can take this post-classical behavior one step further: Since the compiler can assume that undefined behavior never occurs (because if it did, then the compiler is allowed to do anything it wants), the compiler can use undefined behavior to guide optimizations.

    int value_or_fallback(int *p)
     return p ? *p : 42;

    The above function accepts a pointer to an integer and either returns the pointed-to value or (if the pointer is null) returns the fallback value 42. So far so good.

    Let's add a line of debugging to the function.

    int value_or_fallback(int *p)
     printf("The value of *p is %d\n", *p);
     return p ? *p : 42;

    This new line introduces a bug: It dereferences the pointer p without checking if it is null. This tiny bug actually has wide-ranging consequences. A post-classical compiler will optimize the function to

    int value_or_fallback(int *p)
     printf("The value of *p is %d\n", *p);
     return *p;

    because it observes that the null pointer check is no longer needed: If the pointer were null, then the printf already engaged in undefined behavior, so the compiler is allowed to do anything in the case the pointer is null (including acting as if it weren't).

    Okay, so that's not too surprising. That may even be an optimization you expect from a compiler. (For example, if the ternary operator was hidden inside a macro, you would have expected the compiler to remove the test that is provably false.)

    But a post-classical compiler can now use this buggy function to start doing time travel.

    void unwitting(bool door_is_open)
     if (door_is_open) {
     } else {
      // wait for the door to open using the fallback value
      fallback = value_or_fallback(nullptr);

    A post-classical compiler can optimize this entire function to

    void unwitting(bool door_is_open)


    The compiler observed that the call value_or_fallback(nullptr) invokes undefined behavior on all code paths. Propagating this analysis backward, the compiler then observes that if door_is_open is false, then the else branch invokes undefined behavior on all code paths. Therefore, the entire else branch can be treated as unreachable

    Okay, now here comes the time travel:

    void keep_checking_door()
     for (;;) {
      printf("Is the door open? ");
      char response;
      if (scanf("%c", &response) != 1) return;
      bool door_is_open = response == 'Y';

    A post-modern compiler may propagate the analysis that "if door_is_open is false, then the behavior is undefined" and rewrite this function to

    void keep_checking_door()
     for (;;) {
      printf("Is the door open? ");
      char response;
      if (scanf("%c", &response) != 1) return;
      bool door_is_open = response == 'Y';
      if (!door_is_open) abort();

    Observe that even though the original code rang the bell before crashing, the rewritten function skips over ringing the bell and just crashes immediately. You might say that the compiler went back in time and unrung the bell.

    This "going back in time" is possible even for objects with external visibility like files, because the standard allows for anything at all to happen when undefined behavior is encountered. And that includes hopping in a time machine and pretending you never called fwrite.

    Even if you claim that the compiler is not allowed to perform time travel,¹ it's still possible to see earlier operations become undone. For example, it's possible that the undefined operation resulted in the file buffers being corrupted, so the data never actually got written. Even if the buffers were flushed, the undefined operation may have resulted in a call to ftruncate to logically remove the data you just wrote. Or it may have resulted in a Delete­File to delete the file you thought you had created.

    All of these behaviors have the same observable effect, namely that the earlier action appears not to have occurred. Whether they actually occurred and were reversed or never occurred at all is moot from a compiler-theoretic point of view.

    The compiler may as well have propagated the effect of the undefined operation backward in time.

    ¹ For the record, the standard explicitly permits time travel in the face of undefined behavior:

    However, if any such execution contains an undefined operation, this International Standard places no requirement on the implementation executing that program with that input (not even with regard to operations preceding the first undefined operation).

    (Emphasis mine.)

    ² Another way of looking at this transformation is that the compiler saw that the else branch invokes undefined behavior on all code paths, so it rewrote the code as

    void unwitting(bool door_is_open)
     if (door_is_open) {
     } else {

    taking advantage of the rule that undefined behavior allows anything to happen, so in this case, it decided that "anything" was "calling walk_on_in by mistake."

    Bonus chatter: Note that there are some categories of undefined behavior which may not be obvious. For example, dereferencing a null pointer is undefined behavior even if you try to counteract the dereference before it does anything dangerous.

    int *p = nullptr;
    int& i = *p;
    foo(&i); // undefined

    You might think that the & and the * cancel out and the result is as if you had written foo(p), but the fact that you created a reference to a nonexistent object, even if you never carried through on it, invokes undefined behavior (§8.5.3(1)).

    Related reading: What Every C Programmer Should Know About Undefined Behavior, Part 1, Part 2, Part 3.

    Update: Broke the &* into two lines because it is the lone * that is the problem.

  • The Old New Thing

    For Honor, For Excellence, For Pizza


    Hacker News member citizenlow recalls the time I went over after hours to help out the Money team debug a nasty kernel issue. They were running into mysterious crashes during their stress testing and asked for my help in debugging it.

    I helped out other teams quite a bit, like writing a new version of Dr. Watson for the Windows 98 team or writing a new version of the MSConfig tool based on a sketch on a bar napkin. And for a time, I followed the official policy for moonlighting to make sure everybody understood that I was doing work outside the boundaries of my official job duties.

    When the Money folks asked me for help, I told them that before I could help them, they would have to help me fill out some paperwork.

    • Who will you be working for? Microsoft Corporation.
    • Where will you be doing the work? Office XX/YYYY on Microsoft Redmond Campus.
    • When will the work begin and end? Begin on YYYY/MM/DD at 5pm, ending YYYY/MM/DD at 11pm.
    • How much will you be paid for this work?

    The Money folks were not sure how to answer that last question, since they didn't have any formal budget or procedures for hiring an outside consultant, much less any procedures for hiring one from inside the company.

    I told them, "Just write One slice of pizza."

    Nobody from the Personnel department seemed to notice the odd circumstances of this moonlighting request; they simply rubber-stamped it and put it in my file.

    The crash, it turns out, was in Windows itself. There was a bug in the special compiler the Languages team produced to help build certain components of Windows 95 which resulted in an incorrect address computation under a particularly convoluted boundary condition. The Money folks had merely stumbled across this bug as part of their regular testing. I notified the appropriate people, and the Windows team applied a workaround in their code to tickle the compiler into generating the correct code.

    As I recall, the pizza was just fine. It was just your average delivery pizza, nothing gourmet or anything. Not that it had to be, because I wasn't there for the pizza.

  • The Old New Thing

    Getting past the question to solving the problem, episode 2014.06.25


    Today is another example where the right thing to do is not to answer the customer's question but rather to solve the customer's problem.

    A customer liaison asked, "What do the registry keys X and Y do? We noticed that they are both read from and written to when you open a common file dialog. Just curious."

    I replied, "I'm curious as to your curiousity. I'm afraid that you are curious because your customer is curious, and then the customer will start relying on the keys in a strange and unsupported way." The format of those keys has varied from one version of Windows to another, so there is nothing there applications can rely on. But maybe we can figure out why the customer is snooping there so we can solve the customer's problem.

    The customer liaison was kind enough to explain. "The customer wants to know how to set the default folder shown in the Open and Save As dialogs."

    The algorithm for choosing the default folder shown in the Open and Save As dialogs is spelled out in the documentation of the OPEN­FILE­NAME structure. There is no registry key that can force all dialogs to use a particular folder. But what is the customer's actual problem?

    The customer liaison explained, "The customer wants to change the default save location for Paint and Notepad."

    Okay, now we're getting closer to a solvable problem. Paint defaults to saving in the user's Pictures folder, and Notepad defaults to saving in the user's Documents folder, so you can use folder redirection to point the Pictures and Documents folders to locations of your choosing, noting of course that this will affect all applications which look for those folders.

    It turns out that this was what the customer actually needed. They redirected the user's Pictures and Documents folders to their preferred location via GPO, and everybody was happy.

  • The Old New Thing

    The social interactions in two preschool classes, in graphic form


    Each preschooler at my daughter's school was asked a few simple questions, and the answers were printed in the yearbook. Among other things, the preschoolers were asked to complete the sentence, "I like to play with (person)."

    This is the type of question that leads to tears and hurt feelings.

    Whatever. Their parents are going to be stuck with the therapy bills. (My daughter is not a preschooler at the school, so I avoided a therapy bill. At least not over this.)

    From this data, I created a graph. Each arrow points from a student to the person they said they like to play with.

    1 7 16
    2 8 17
    3 9 14 18
    10 19
    4 11 15 20
    5 12
    6 13

    This class breaks up into four cliques. Two of the cliques consists of a pair of playmates, and one hanger-on. The large clique consists of two focal points (students 9 and 10) who play with each other. The medium-sized clique has a single focal point (student 18) who plays with a best friend (14).

    I think that student 14 is in the best spot. He (or she) is not himself popular, but the popular kid plays with him (or her).

    The second preschool class has a more complex structure.

    23 30
    24 31 35
    25 32 36
    26 33 37
    27 34 38

    The upper left group consists of a core of four students (23, 30, 31, 24) who play with each other, plus some hangers-on.

    The lower left group consists of a pair of friends (27 and 28) and their hangers-on.

    The right-hand group consists of one very popular student (37) who plays with a best friend (36), and their hangers-on.

    The most interesting student is number 26.

    All of the other students gave only one name in response to the prompt. But student 26 gave three names. As a result, that student links together the three cliques in the class.

    Student 26 is bringing people together. I admire that.

  • The Old New Thing

    Finding the "Run as administrator" command, a game of hide-and-seek


    Back in the old days, the "Run as administrator" menu option was placed on the extended menu. To get the extended menu, you hold the shift key when you right-click on the shortcut.

    In Windows 7, the "Run as administrator" option was moved to the primary menu, so you no longer need to hold the shift key to get it.

    Well, except that sometimes you still need to hold the shift key.

    The deal is that there are two "Run as administrator" commands. One of them is for running shortcuts to regular applications as administrator. The other is for running shortcuts to MSI applications as administrator. Shortcuts to MSI applications are weird because they aren't shortcuts to EXEs but rather are shortcuts to an entry in the MSI database.

    When the decision was made to move the "Run as administrator" command from the extended menu to the primary menu, the person who made the change moved only one of the "Run as administrator" commands (the one for regular application) and forgot to move the other one (the one for shortcuts to MSI applications). As a result, you still need to hold the shift key to run MSI applications elevated.

    The problem is fixed on the Windows 8 Start menu. It correctly shows the "Run as administrator" option for both shortcuts to regular applications and shortcuts to MSI applications.

Page 3 of 427 (4,265 items) 12345»