August, 2006

  • The Old New Thing

    Environment variable expansion occurs when the command is read


    On the command line (and in batch files), environment variable expansion occurs when the command is read. This sounds obvious at first, but it has its own consequences.

    In the online documentation for SET, one such consequence is spelled out:

        set VAR=before
        if "%VAR%" == "before" (
            set VAR=after
            if "%VAR%" == "after" @echo If you see this, it worked

    would never display the message, since the %VAR% in both "if" statements is substituted when the first "if" statement is read, since it logically includes the body of the "if", which is a compound statement.

    In other words, the "if" command is not complete until the closing parenthesis is read. You can see this if you type the commands interactively:

    C:\>set VAR=before
    C:\>if "%VAR%" == "before" (
    More? set VAR=after
    More? if "%VAR%" == "after" @echo If you see this, it worked
    More? )

    Notice that the "if" command didn't execute until you closed the parenthesis; the command interpreter kept prompting "More?" to collect the body of the "if". This means that everything you type as the body of the "if" is evaluated before the "if" condition or any of the lines in the body are evaluated. It's as if you had typed

    C:\>if "before" == "before" (
    More? set VAR=after
    More? if "before" == "after" @echo If you see this, it worked
    More? )

    Note that this is different from most UNIX shells, which do not expand environment variables until the enclosing command is executed. For example,

    $ var=before
    $ var=after; echo $var

    Notice that the $x is not expanded until the echo command's arguments are being computed. The analogous commands in the Windows command interpreter result in something quite different:

    C:\>set VAR=before
    C:\>set VAR=after & echo %VAR%

    That's because the command interpreter expanded the environment variables at the time the line was read (not at the time the line is executed), yielding

    set VAR=after & echo before

    As a result, the old value of VAR is echoed. Some people treat this as a feature, allowing them to "restore" a variable without having to save it anywhere:

    set VAR=newvalue & call helper.cmd & set VAR=%VAR%

    This command sets the VAR variable to a new value, calls helper.cmd (which presumably uses the value of the %VAR% variable to control its behavior), then magically restores the variable to its original value since the %VAR% is expanded early, producing the old value.

    But what if you want the variable to be expanded at execution time rather than at parse time? For that, you use "delayed expansion", which is enabled by the /V command line option or by using the SETLOCAL ENABLEDELAYEDEXPANSION command in a batch file.

    C:\> copy con "%TEMP%\helper.cmd"
    set VAR=before
    set VAR=after & echo immediate:%VAR%, delayed:!VAR!
            1 file(s) copied.
    C:\> "%TEMP%\helper.cmd"
    C:\>set VAR=before
    C:\>set VAR=after   & echo immediate:before, delayed:!VAR!
    immediate:before, delayed:after

    Immediate expansion is performed with percent signs, whereas delayed expansion is performed with exclamation points.

    Why is immediate expansion the default? Because prior to Windows NT, that was the only type of expansion supported by the command interpreter. Retaining immediate expansion as the default preserved backwards compatibility with existing batch files. (The original command interpreter was written in assembly language. You really didn't want to be too clever or it would make your brain hurt trying to maintain the code. An interpreter loop of the form "Read a line, expand environment variables, evaluate" was therefore simple and effective.)

    Armed with this understanding of immediate versus delayed expansion, perhaps you can explain what is really going on here. (Hint: It has nothing to do with ERRORLEVEL.)

  • The Old New Thing

    Raymond's excursions into East Asian pop music, episode 2: China Dolls (中國娃娃)


    The wife of one of my colleagues took a trip through the Far East as part of her work. One of the things she did was buy a bunch of music CDs from the various countries she visited. But not just any CDs. To decide which ones to get, she used a very scientific method that didn't require knowing how to read or speak the local language: She would go into a music store and just observe the teenage girls as they did their shopping. Based on this sampling, she would buy the CDs that appeared to be the most popular.

    One of the CDs she loaned me is of a Thai pop duo called China Dolls (中國娃娃). (Here's a Japanese fan site.)

    "Wait, if they are Thai, why is the group name China Dolls?" Well, because one of the members is Thai-Taiwanese and the other is Thai-Chinese, and they play up the Chinese angle.

    The songs on the CD are translations of their top-selling first album from Thai into Mandarin so that they would appeal to a Taiwanese audience. (My untrained ears detect what I believe to be a Cantonese accent in some of their songs.) The plan was a success: The CD made it to #3 on the charts. YouTube has the video of what may be their signature song, 單眼皮女生, variously translated as Single Eyelid Girl or Girl with Slanty Eyes, performed in Mandarin Chinese with some Thai mixed in. And of course English, because in my experience, all East Asian pop music contains some English. It's just a rule.

    What particularly impressed me about the performers was not so much their artistic ability (for their intonation is occasionally lacking, though they certainly dance with a lot of energy and the tunes are catchy enough) but that Hwa-Hwa speaks four languages [mpg] [wmv]. (And check out the sneaky backwards peace sign from Bell. Warning: Do not make this hand gesture in England!)

    Episode 1.

  • The Old New Thing

    Who says there's only one? There can be more than one logon session


    An extension of the "What if two programs did this?" thought experiment is the "Who says there's only one?" question.

    A common question I see is, "From a service, how do I do X with the currently logged-on user?" (Where "X" can be a variety of things such as interact with them or impersonate them.) But who says that there's only one?

    With the introduction of Fast User Switching in Windows XP, the possibility for multiple logged-on users exists even in consumer scenarios. You might say, "Well, I mean that among all the users that are logged on, I want the one that's using the computer right now." Except that with Media Center Extender, there can still be two users, one sitting at the console and another in the TV room with the extender, and both of them are using the computer right now. And on the server side of things, Terminal Services has been around since NT4 (a limited version of which is available in Windows XP Professional under the name "Remote Desktop").

    There are many variations on this question. "How can I check whether the 'Press Ctrl+Alt+Del to begin' dialog is being displayed?" There is not just one such dialog; there is one for every session. If there are three users logged onto the machine, one of them logged in and active, another one disconnected, and a third sitting at the 'Press Ctrl+Alt+Del to begin' dialog, what should the answer be? That depends on what you're planning to use this information for.

    Sometimes it takes a few iterations before people get the message, and sometimes they never do.

    "How do I impersonate the currently logged-on user?"

    "Which user? There can be more than one. <explanation>"

    "I want the one on the desktop."

    "They all have their own desktop."

    "I want the one on the current desktop."

    "What do you mean by 'the' current desktop? There is one current desktop for each user."

    "I want the desktop I should display my UI on."

    "Your question has become circular. You want to display your UI on the desktop that your UI should be displayed on."

    "Why won't you answer my question?!"

  • The Old New Thing

    [6] days since last monorail breakdown


    It's soon going to come to the point where this is no longer news. The Seattle monorail broke down again, just six days since the previous breakdown, which was in turn just two days after operations resumed.

    I think they need to put up a big sign at the Monorail station at Seattle Center that reads

    6 days since last breakdown.

    Well, except that today it would read "2".

  • The Old New Thing

    Sucking the exception pointers out of a stack trace


    Often, you find yourself staring at a stack trace for a caught exception and want to see the original exception.

    ChildEBP RetAddr  Args to Child
    030c21d0 76df3448 00000000 030c6138 76db6b0d ntdll!DbgBreakPoint
    030c21dc 76db6b0d 030c2204 77b8d585 030c220c ole32!PeekMessageExceptionFilter+0x42
    030c21e4 77b8d585 030c220c 00000000 030c220c ole32!CCliModalLoop::MyPeekMessage+0x41
    030c220c 77f36992 030c25d0 030c6128 030c22e8 msvcrt!_except_handler3+0x61
    030c2230 77f36964 030c25d0 030c6128 030c22e8 ntdll!ExecuteHandler2+0x26
    030c22d8 77f36884 030c1000 030c22e8 00010007 ntdll!ExecuteHandler+0x24
    030c25b8 77f6e0dd 030c25d0 00000000 00000000 ntdll!RtlRaiseException+0x3d
    030c262c 77d3c239 77d4a4b6 77d3e2c5 030c3767 ntdll!RtlDeactivateActivationContextUnsafeFast+0x233
    030c2630 77d4a4b6 77d3e2c5 030c3767 030c26a0 USER32!UserCallWinProcCheckWow+0x167
    030c2634 77d3e2c5 030c3767 030c26a0 77d4a46f USER32!_NLG_Return2
    030c265c 77d3e288 030c57b4 ffffffff 030c2688 USER32!__local_unwind2+0x70
    030c2688 77f36992 030c26f8 030c57b4 030c27a4 USER32!_except_handler3+0xd5
    030c26ac 77f36964 030c26f8 030c57b4 030c27a4 ntdll!ExecuteHandler2+0x26
    030c2a74 77b8d36d 030c6128 77b8d36d 00000000 ntdll!ExecuteHandler+0x24
    030c2a9c 77b8d59d 030c6128 030c2ac0 00000000 msvcrt!__global_unwind2+0x18
    030c2ac0 77f36992 030c2ba4 030c6128 030c2bc0 msvcrt!_except_handler3+0x75
    030c2ae4 77f36964 030c2ba4 030c6128 030c2bc0 ntdll!ExecuteHandler2+0x26
    030c2b8c 77f36796 030c1000 030c2bc0 030c2ba4 ntdll!ExecuteHandler+0x24
    030c2b8c 77b7aa54 030c1000 030c2bc0 030c2ba4 ntdll!KiUserExceptionDispatcher+0xe
    030c3300 77b7b4dc 030c3324 7715b1b4 00000000 msvcrt!_woutput_l+0x18

    (You too can get symbols for operating system binaries, either by using the symbol server to get the symbols on-demand or, if you have a gigabyte of disk space, you can download symbol packages to get them all at one go. Even if you go for the symbol package, you still need the symbol server, since it gets updated with symbols for binaries that have been updated since the most recent service pack.)

    Here, we caught an exception in the PeekMessageExceptionFilter. What was the exception? Well, an exception filter receives a pointer to an EXCEPTION_POINTERS structure as its argument.

    typedef struct _EXCEPTION_POINTERS {
        PEXCEPTION_RECORD ExceptionRecord;
        PCONTEXT ContextRecord;

    Here, we see that the parameter to PeekMessageExceptionFilter is

    030c21dc 76db6b0d 030c2204 77b8d585 030c220c ole32!PeekMessageExceptionFilter+0x42
    0:0000> dd 030c2204 l2
    030c2204 030c25d0 030c22e8
             -------- --------
             .exr     .cxr

    The first value points to the exception record and the second points to the context record. You can view the exception by typing .exr 030c25d0 and view the context for the exception (i.e., what code was executing when the exception occurred) by typing .cxr 030c22e8. Those two values also appear as the first and (go figure) third parameters to ExecuteHandler2.

    It so happens that doing the .exr on this particular exception record reports that the exception was c015000f which happens to be STATUS_SXS_EARLY_DEACTIVATION, and after setting the context to the exception context record, the stack goes

    ChildEBP RetAddr
    030c262c 77d3c239 77d4a4b6 77d3e2c5 030c3767 ntdll!RtlDeactivateActivationContextUnsafeFast+0x233
    030c2630 77d4a4b6 77d3e2c5 030c3767 030c26a0 USER32!UserCallWinProcCheckWow+0x167
    030c2634 77d3e2c5 030c3767 030c26a0 77d4a46f USER32!_NLG_Return2
    030c265c 77d3e288 030c57b4 ffffffff 030c2688 USER32!__local_unwind2+0x70
    030c2688 77f36992 030c26f8 030c57b4 030c27a4 USER32!_except_handler3+0xd5
    030c26ac 77f36964 030c26f8 030c57b4 030c27a4 ntdll!ExecuteHandler2+0x26
    030c2a74 77b8d36d 030c6128 77b8d36d 00000000 ntdll!ExecuteHandler+0x24
    030c2a9c 77b8d59d 030c6128 030c2ac0 00000000 msvcrt!__global_unwind2+0x18
    030c2ac0 77f36992 030c2ba4 030c6128 030c2bc0 msvcrt!_except_handler3+0x75
    030c2ae4 77f36964 030c2ba4 030c6128 030c2bc0 ntdll!ExecuteHandler2+0x26
    030c2b8c 77f36796 030c1000 030c2bc0 030c2ba4 ntdll!ExecuteHandler+0x24
    030c2b8c 77b7aa54 030c1000 030c2bc0 030c2ba4 ntdll!KiUserExceptionDispatcher+0xe
    030c3300 77b7b4dc 030c3324 7715b1b4 00000000 msvcrt!_woutput_l+0x18

    Wow, we took an exception while trying to handle another exception! (It so happens this was easy to spot in the original stack trace, but in the general case, the next outer exception may require digging.)

    Repeat the exercise with this next exception:

    0:000> .exr 030c2ba4
    ExceptionAddress: 77b7aa54 (msvcrt!_woutput_l+0x00000018)
       ExceptionCode: c00000fd (Stack overflow)
      ExceptionFlags: 00000000
    NumberParameters: 2
       Parameter[0]: 00000001
       Parameter[1]: 030c2e88
    0:000> .cxr 030c2bc0
    eax=030c33b0 ebx=00000000 ecx=0000005c edx=00000000 esi=030c33c4 edi=030c33c4
    eip=77b7aa54 esp=030c2e8c ebp=030c3300 iopl=0         nv up ei pl nz na pe nc
    cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00010202
    001b:77b7aa54 53          push    ebx

    Aha, so the SXS exception was triggered by a stack overflow. At this new context, you can use the "k" command to see how we got into this state.

    It so happens that this particular bug was, as predicted, a stack overflow caused by unintended recursion when a call to an off-thread COM object forced the calling thread to wait for the reply, during which time a new request came in. The precise details of the problem aren't important; the goal today was to show how you can suck the exception pointers out of the stack to see what Win32 exception was the source of the problem.

  • The Old New Thing

    We encourage everyone to pack gel-filled bras in their checked baggage


    My thanks to the Annals of Improbable Research for pointing out this recommendation from the TSA.

    I don't own a gel-filled bra, so I'll have to do some extra shopping before my next plane trip.

  • The Old New Thing

    Applications and DLLs don't have privileges; users do


    I can't believe you people are actually asking for backdoors. If an end user can do it, then so can a bad guy.

    In response to the requirement that all drivers on 64-bit Windows be signed, one commenter suggested adding a backdoor that permits unsigned drivers, using some "obscure registry key". Before somebody can jump up and shouts "security through obscurity!", the commenter adds this parenthetical: "(that no application has privileges to do by default)".

    What does that parenthetical mean? How do you protect a registry key from an application? And if applications don't have privileges to modify a key, then who does?

    The Windows security model is based on identity. Applications don't have privileges. Users have privileges. If an application is running in your user context, then it can do anything you can, and that includes setting that "obscure registry key". (This is a variation on "Your debugging code can be a security hole".) Same goes for DLLs. There's no such thing as something only an individual program/library can read/write to or do. You can't check the "identity of the calling library" because you can't trust the return address. Coming up with some other "magic encryption key" like the full path to the DLL won't help either, because a key that anybody can guess with 100% accuracy isn't much of a key.

    Yes, UNIX has setuid, but that still doesn't make applications security principals. Even in UNIX, permissions are assigned to users, not to applications.

    That's one of the reasons I get so puzzled when I hear people say, "Windows should let me do whatever I want with my system", while simultaneously saying, "Windows should have used ACLs to prevent applications from doing whatever they want with my system." But when you are running an application, the application is you. If you can do it, then an application can do it because the application is you.

    Some people want to extend the concept of security principal to a chunk of code. "This registry key can be written to only by this function." But how could you enforce this? Once you let untrusted code enter a process, you can't trust any return addresses any more. How else could you identify the caller, then?

    "Well, the DLL when it is created is given a magic cookie that it can use to prove its identity by passing that cookie to these 'super-secure functions'. For example,

    // SECRET.DLL - a DLL that protects a secret registry key
    HANDLE g_hMagicCookie;
    // this function is called by means to be determined;
    // it tells us the magic cookie to use to prove our identity.
    void SetMagicCookie(HANDLE hMagicCookie)
     g_hMagicCookie = hMagicCookie;

    and then the program can use the magic cookie to prove that it is the caller. For example, you could have RegSetValueWithCookie(g_hMagicCookie, hkey, ...), where passing the cookie means 'It's me calling, please give me access to that thing that only I have access to."

    That won't stop the bad guys for long. They just have to figure out where the DLL saves that cookie and read it, and bingo, they're now you.

    // bad-guy program
    int CALLBACK WinMain(...)
     // call some random function from SECRET.DLL
     // so it gets loaded and the magic cookie gets
     // initialized.
     // experimentation tells us that SECRET.DLL
     // keeps its magic cookie at address 0x70131970
     HANDLE hMagicCookie = *(HANDLE*)0x70131970;
     RegSetValueWithCookie(hMagicCookie, hkey, ...);
     return 0;

    Ta-da, we now have a program that writes to that registry key that SECRET.DLL was trying to protect. It does it by merely waiting for SECRET.DLL to receive its magic cookie, then stealing that cookie.

    "Well, sure, but if I combine that with the check-the-return-address technique, then that'll stop them."

    No, that doesn't stop anybody. All the bad guy has to do is change the RegSetValueWithCookie(hMagicCookie, hkey, ...) to code that hunts for a trusted address inside SECRET.DLL and cooks up a fake stack so that when control reaches RegSetValueWithCookie, everything in memory looks just like a legitimate call to the function, except that the attacker got to pass different parameters.

    You can come up with whatever technique you want, it won't do any good. Once untrusted code has been granted access to a process, the entire process is compromised and you cannot trust it. Worst case, the attacker just sets a breakpoint on RegSetValueWithCookie, waits for the breakpoint to hit, then edits the stack to modify the parameters and resumes execution.

    That's why code is not a security principal.

    Corollary: Any security policy that says "Applications cannot do X without permission from the user" is flawed from conception. The application running as the user is the user. It's one thing to have this rule as a recommendation, even a logo requirement, but it's another thing to enforce this rule in the security subsystem.

  • The Old New Thing

    A modest proposal: On allowing mobile phones on airplanes


    Ever since the FAA decided to reconsider its ban on the use of cellular phones on airplanes during flight, there has been quite a reaction over whether this is a good thing.

    To resolve this issue, I present this modest proposal.

    Remember back in the days when smoking was permitted on airplanes? When you bought a ticket, you were asked whether you wanted to be in the smoking or non-smoking section. We can do the same thing with mobile phones. You can ask to be seated in the "yapping" or "non-yapping" section. If you're in the yapping section, then yap all you want. Call your grandmother and talk with her the entire flight about her colonoscopy, see if I care. On the other hand, in the non-yapping section, the phone stays off.

    This principle of yap-or-no-yap can be extended to other in-flight disturbances. The no-yapping section would be renamed the "quiet section", where electronic devices must be muted (or used with headphones) and conversations must take place at low volume.

    (Hey, it worked for Amtrak, it can work on airplanes.)

    Side note: I wonder if the temporary ban on carry-on luggage in the UK improved airplane boarding times. Perhaps we could ban carry-on luggage across the board, not as a security measure, but as an efficiency. I don't know about you, but I'm really anonyed by those people whose "one carry-on bag and one personal item" consist of a carry-on bag the size of a small refrigerator and a personal item the size of a microwave oven. They clog up the aisle for ages trying to heave both of their bags into the overhead bin (even though the personal item is supposed to go under the seat in front of you). Okay, enough ranting.

    Next time (if there is a next time), I'll solve the problem of steroids in baseball.

  • The Old New Thing

    Don't trust the return address, no really


    In the discussion of how to prevent non-"trusted" DLLs from using private OS resources, more than one person suggested having the LoadLibrary or FindResource function behave differently depending on who the caller is. But we already saw that you can't trust the return address and that you definitely shouldn't use the return address to make a security decision (which is what these people are proposing).

    All attackers have to do is find some other "trusted" code to do their dirty work. For example, the LoadString function internally calls FindResource to locate the appropriate string bundle. Therefore, if attackers want to get a string resource from a "trusted" DLL, they could use LoadString to do it, since LoadString will call FindResource, and FindResource will say, "Oh, my caller is USER32.DLL, which is trusted." Bingo, they just stole a string resource.

    "Well, I could add that same check to LoadString."

    I was just giving LoadString as an example of a "middle man" function that you can exploit. Sure, extra code could be added to LoadString to check its return address and reject attempts to load strings from "protected" libraries if the caller is "untrusted", but attackers would just look for some other middle man they could exploit. And even if you were diligent enough to protect all such potential middle-men, you still are vulnerable to the sort of stack-manipulation games that don't require anything from a "trusted" DLL aside from a return instruction. (And there are plenty of those.)

    No, you cannot impose security boundaries within a process. Once you let code run unchecked in your process, you have to treat the entire process as compromised. Even the parts that you thought were trustworthy.

    Now, you might say, "Oh, we're not really making a security decision here. We just want to make circumventing the system so much hard work that somebody who goes to that much effort knows that they're doing something unsupported." But as commenter Duncan Bayne points out, that applies only to the first person to do it. They then make a library out of their technique, or publish it in a magazine article, and now anybody can use it without a struggle, and consequently without it crossing their mind that "Gosh, maybe this isn't such a great idea to use in production software."

  • The Old New Thing

    Computer monitors float, screen upwards


    Oceanographer Curtis Ebbesmeyer made another appearance on my local public radio station. Among the ocean garbage trivia is the fact that computer monitors float screen upwards (timecode 4:00).

    Other fascinating facts:

    • Garbage from Japan collects near the United States; garbage from the United States collects near Japan.
    • Researchers studying dead albatross chicks have found 60-year-old plastic in the birds' stomachs.
    • Some beaches "specialize" in certain types of garbage.
Page 2 of 4 (39 items) 1234