• The Old New Thing

    That doesn't sound like South Frisian to me


    I dreamed that I was back in college taking a course in South Frisian, but I suspected something was up because the words didn't sound Germanic at all, and we were taught the words to a Christmas carol as Nom Yom Hear What I Hear?

    Also, because the course was taught by known prevaricator/exaggerator Robert Irvine.

  • The Old New Thing

    Is it wrong to call SHFileOperation from a service?


    A customer had a simple question: "Is it wrong to call SHFile­Operation from a service?"

    I don't know if I'd call it wrong, but I'd call it highly inadvisable.

    • SHFile­Operation was designed for interactive operations, so you're using it outside its original design parameters.
    • Many shell extensions ignore "no UI" flags and put up UI anyway. As a result, your call to SHFile­Operation may end up getting stuck on unexpected UI. Now you have a service displaying UI, and that's just asking for trouble.
    • The shell for the most part does not expect to be called while impersonating. There are a few functions specifically designed for use while impersonating; those exceptions are called out explicitly in their respective documentation. SHFile­Operation is not one of those functions.
    • Since SHFile­Operation uses the shell namespace, you are at risk of loading shell extensions into a service. Shell extensions typically are not written with the strict security requirements of a service in mind, and you may end up creating a security hole. Somebody could plant a desktop.ini into a directory your service operates on, and now your service has been tricked into loading a shell namespace extension. The bad guys are constantly searching for buggy shell extensions that they can use as an attack point. And if they can get into a service, well, then they just hit the jackpot!

    Update: See Is it wrong to call SHFileOperation from a service? Revised.

  • The Old New Thing

    I marked my parameter as [optional], so why do I get an RPC error when I pass NULL?


    Consider the following interface declaration in an IDL file:

    // Code in italics is wrong
    interface IFoo : IUnknown
        HRESULT Cancel([in, optional, string] LPCWSTR pszReason);

    The idea here is that you want to be able to call the Cancel method as pFoo->Cancel(NULL) if you don't want to provide a reason.

    If you try this, you'll find that the call sometimes fails with error 0x800706F4, which decodes to HRESULT_FROM_WIN32(RPC_X_NULL_REF_POINTER). What's going on here?

    The optional attribute does not mean what you think it means. To a C or C++ programmer, an "optional" pointer parameter typically means that it is valid to pass NULL/nullptr as the parameter value. But that's not what it means to the IDL compiler.

    To the IDL compiler, optional parameters are hints to the scripting engine that the parameter should be passed as VT_ERROR/DISP_E_PARAM­NOT­FOUND. The attribute is meaningful only when applied to parameters of type VARIANT or VARIANT*.

    What you actually want is the unique attribute. This somewhat confusingly-named attribute means "The parameter is allowed to be a null pointer." Therefore, the interface should have been written as

    interface IFoo : IUnknown
        HRESULT Cancel([in, unique, string] LPCWSTR pszReason);

    At the lowest level in the marshaler, pointer parameters are marked as ref, unique, or ptr. ref parameters may not be null, whereas unique and ptr parameters are allowed to be null. Larry Osterman explained to me that the default for interface pointers (anything derived from IUnknown) is unique and the default for all other pointer types is ref. Therefore, if you want to say that NULL is a valid value for a non-interface pointer parameter, you must say so explicitly by annotating the parameter as [unique].

    It's probably too late to change the behavior of MIDL to reject the [optional] tag on non-VARIANT parameters because in the decades since the attribute was introduced, it's probably being used incorrectly approximately twenty-five bazillion times, and making it an error would break a lot of code. (Even if you just made it a warning, that wouldn't help because a lot of people treat warnings as errors.)

    Exercise: Why is the RPC_X_NULL_REF_POINTER error raised only sometimes?

  • The Old New Thing

    How can I make sure my program is launched only from my helper program and no other parent?


    Say you have a collection of programs which work together. One of them is the "master" program that runs the show, and it has a bunch of "assistant" programs that it launches to accomplish various subtasks. These assistants are not meant to be run by themselves; they are meant to be run only by the master program. How do you design the assistant so that it can only be run by the master?

    There's nothing you can do to force the assistant to be run only by the master, since anything you do to detect the case can be faked out by an attacker. (Worst case is that they just run your program under the debugger and patch out the code that looks for the master.) So the purpose of this test is not so much to create an airtight hatchway as it is to prevent users from randomly wandering into the Program Files directory and double-clicking stuff to see what happens.

    The simplest way of doing this is to require a command-line parameter that the master passes to say, "Hey, it's me, the master. It's okay to do that thing you do." The command line parameter could be anything. assistant.exe /run say. If the command line parameter is not present, then the assistant says, "Um, please don't run this program directly. Use the master."

    You might decide to get really fancy and make the secret handshake super-complicated, but remember that there is no added security benefit here. The user can compromise assistant.exe by simply attaching a debugger to it, at which point any defensive mechanism you create can simply be disabled by a sufficiently-resourceful attacker. (And there's a class of people who will see that you put a lot of work into protecting your assistant, and that will just convince them to work harder to circumvent the protection. Because something with this much protection must certainly be very valuable!)

    There's also a benefit to keeping the secret handshake simple: It makes it a lot easier for you to debug the assistant program. Instead of having to set up the master and then get the master to do all the things it needs to generate the secret handshake for the assistant, you can just run your assistant directly with the magic flag, and boom, you're off and debugging.

    To make it even harder to run your program by accident, you can give it an extension that is not normally executable, like .MOD. That way, it cannot be double-clicked, but you can still pass it to Create­Process or (with some cajoling) Shell­Execute­Ex.

  • The Old New Thing

    Can an x64 function repurpose parameter home space as general scratch space?


    We saw some time ago that the x64 calling convention in Windows reserves space for the register parameters on the stack, in case the called function wants to spill them. But can the called function use the memory for other purposes, too?

    You sort of already know the answer to this question. Consider this function:

    void testfunction(int a)
     a = 42;

    How would a naïve compiler generate code for this function?

        sub rsp, 8 ;; realign the stack
        ;; spill all register parameters into home locations
        mov [rsp+0x10], rcx
        mov [rsp+0x18], rdx
        mov [rsp+0x20], r8
        mov [rsp+0x28], r9
        ;; a = 42
        mov [rsp+0x10], 42
        ;; return
        add rsp, 8 ;; clean up local frame

    Observe that after spilling the register parameters into their home locations onto the stack, the function modified the local variable, which updated the value in the home location.

    Since a function can arbitrarily modify a parameter, you can see that a function is therefore allowed to arbitrarily modify a parameter's home location. At which point you can see that an optimizing compiler might choose an arbitrary value completely unrelated to the parameter.

    Our test function has only one parameter. What about the other three home registers?

    The caller is responsible for allocating space for parameters to the callee, and must always allocate sufficient space for the 4 register parameters, even if the callee doesn't have that many parameters.

    A function can therefore treat those 32 bytes as bonus free play. The rationale behind those 32 bytes is that it gives you a place to spill your inbound register parameters so that they will be adjacent to the stack-based parameters. (We saw how the naïve compiler took advantage of this by not trying to be clever in its function prologue and simply spilling all register parameters whether it needs them or not.)

    Nevertheless, you are free to use them for whatever purpose you like, and if you're looking at heavily-optimized code, you'll probably find that the compiler found all sorts of clever things it can do with them. For example, a common trick is to use them to save the nonvolatile registers that the function locally uses to hold the corresponding parameter!

    (Did this article look familiar? Turns out I covered this article a few years ago, but I'm senile and accidentally repeated a topic. And since I put so much effort into writing it, I'm going to make you suffer through it, even though it's a repeat. Hey, television programs repeat during the summer.)

  • The Old New Thing

    How do you prevent the linker from discarding a function you want to make available for debugging?


    We saw some time ago that you can ask the Windows symbolic debugger engine to call a function directly from the debugger. To do this, of course, the function needs to exist.

    But what if you want a function for the sole purpose of debugging? It never gets called from the main program, so the linker will declare the code dead and remove it.

    One sledgehammer solution is to disable discarding of unused functions. This the global solution to a local problem, since you are now preventing the discard of any unused function, even though all you care about is one specific function.

    If you are comfortable hard-coding function decorations for specific architectures, you can use the /INCLUDE directive.

    #if defined(_X86_)
    #define DecorateCdeclFunctionName(fn) "_" #fn
    #elif defined(_AMD64_)
    #define DecorateCdeclFunctionName(fn) #fn
    #elif defined(_IA64_)
    #define DecorateCdeclFunctionName(fn) "." #fn
    #elif defined(_ALPHA_)
    #define DecorateCdeclFunctionName(fn) #fn
    #elif defined(_MIPS_)
    #define DecorateCdeclFunctionName(fn) #fn
    #elif defined(_PPC_)
    #define DecorateCdeclFunctionName(fn) ".." #fn
    #error Unknown architecture - don't know how it decorates cdecl.
    #pragma comment(linker, "/include:" DecoratedCdeclFunctionName(TestMe))
    EXTERN_C void __cdecl TestMe(int x, int y)

    If you are not comfortable with that (and I don't blame you), you can create a false reference to the debugging function that cannot be optimized out. You do this by passing a pointer to the debugging function to a helper function outside your module that doesn't do anything interesting. Since the helper function is not in your module, the compiler doesn't know that the helper function doesn't do anything, so it cannot optimize out the debugging function.

    struct ForceFunctionToBeLinked
      ForceFunctionToBeLinked(const void *p) { SetLastError(PtrToInt(p)); }
    ForceFunctionToBeLinked forceTestMe(TestMe);

    The call to Set­Last­Error merely updates the thread's last-error code, but since this is not called at a time where anybody cares about the last-error code, it is has no meaningful effect. The compiler doesn't know that, though, so it has to generate the code, and that forces the function to be linked.

    The nice thing about this technique is that the optimizer sees that this class has no data members, so no data gets generated into the module's data segment. The not-nice thing about this technique is that it is kind of opaque.

  • The Old New Thing

    The MARGINS parameter to the DwmExtendFrameIntoClientArea function controls how far the frame extends into the client area


    A customer wrote a program that calls Dwm­Extend­Frame­Into­Client­Area to extend the frame over the entire client area, but then discovered that this made programming difficult:

    I have a window which I want to have a glassy border but an opaque body. I made my entire window transparent by calling Dwm­Extend­Frame­Into­Client­Area, and I understand that this means that I am now responsible for managing the alpha channel when drawing so that the body of my window remains opaque while the glassy border is transparent. Since most GDI functions are not alpha-aware, this management is frustrating. Is there a better way? In pictures, I only want the red portion of the diagram below to be on glass; the inside yellow part should be opaque like normal. Is there an API that can do this?

    This customer's excitement about the glass frame is like somebody who buys a pallet of tangerine juice even though he only wanted two glasses. And now he has questions about how to store the rest of the tangerine juice he didn't want.

    This customer, it appears, passed −1 as the MARGINS to Dwm­Extend­Frame­Into­Client­Area which means "Bring it on, baby! Give me all tangerine all the time everywhere!" If you only want the glass to extend into part of your client area, then say so. Set the MARGINS to the thickness of the glass border (the thickness of the red portion of the above diagram).

  • The Old New Thing

    Why does my window style change when I call SetWindowText?


    A customer observed some strange behavior with window styles:

    We ran some weird behavior: Calling the Set­Window­Text function causes a change in window styles. Specifically, calling Set­Window­Text results in the WM_STYLE­CHANGING and WM_STYLE­CHANGED messages, and sometimes the result is that the WS_TAB­STOP style is removed. Is this a bug? What would cause this?

    The Set­Window­Text message sends the WM_SET­TEXT message to the control, at which point anything that happens is the window's own responsibility. If it wants to change styles based on the text you sent, then that's what happens. The window manager doesn't do anything special to force it to happen or to prevent it.

    That's weird, because I'm not even listening for WM_SET­TEXT messages. I also verified that there is no call into my code during the call to the the Set­Window­Text function.

    I'm assuming that the window belongs to the same process as the caller. If the window belongs to another process, then the rules are different.

    I'm changing the text of a window created by the same thread.

    Okay, so let's see what we have so far. The customer is calling the Set­Window­Text function to change the text of a window created on the same thread. There is no handler for the WM_SET­TEXT message, and yet the window style is changing. At this point, you might start looking for more obscure sources for the problem, like say a global hook of some sort. While I considered the possibilities, the customer added,

    It may be worth noting that I'm using the Sys­Link.

    Okay, now things are starting to make sense, and it didn't help that the customer provided misleading information in the description of the problem. For example, when the customer wrote, "There is no handler for the WM_SET­TEXT message," the customer was not referring to the window whose window text is changing but to some other unrelated window.

    It's like responding to the statement "A confirmation letter should have been sent to the account holder" with "I never got the confirmation letter," and then the person spends another day trying to figure out why the confirmation letter was never sent before you casually mention, "Oh, I'm not the account holder."

    The WM_SET­TEXT message is sent to the window you passed to Set­Window­Text; in this case, it's the Sys­Link window. It is therefore the window procedure of the Sys­Link window that is relevant here.

    The Sys­Link control remembers whether it was originally created with the WS_TAB­STOP, and if the markup it is given has no tab stops, then it removes the style; if the markup has tab stops, then it re-adds the style.

    How do I add a tab stop to a string? I couldn't find any reference to it and all my guesses failed.

    The tab stops in question are the hyperlinks you added when you used the <A>...</A> notation. If the text has no hyperlinks, then the control removes the WS_TAB­STOP style because it is no longer something you can tab to.

  • The Old New Thing

    Foiled by my withered hand


    A few years ago, some email was sent out to the product team asking for a volunteer hand model to demonstrate how to open the Windows Vista box. Alas, I withdrew myself from consideration due to my withered hand.

  • The Old New Thing

    The peculiar appeal of the Baseball Uncyclopedia


    NPR's Only a Game interviewed the authors of The Baseball Uncyclopedia, an irreverent guide to our national pastime, and it was a blast to listen to. The two authors clearly are huge baseball fans, but they bring to it a fascination not with mind-numbing statistics but with the deep history of the sport. (Plus the fact that they disagree on many things, which leads to really funny arguments.) And who can't appreciate the theory that Kevin Costner is responsible for Shoeless Joe Jackson not being in the Baseball Hall of Fame?

Page 370 of 450 (4,494 items) «368369370371372»