April, 2012

  • The Old New Thing

    When you don't speak a language, don't sound like you speak the language

    • 35 Comments

    I appreciate the help from Christoph and Voo in refining my German. But that reminds me of a story about a friend of a friend.

    She was in Japan to visit some friends. Although she speaks English and Mandarin fluently, she doesn't know any Japanese, so her friends taught her how to say "Sorry, I don't speak Japanese." She managed to say this sentence quite well despite learning it purely phonetically.

    One day they were walking down the street as a group, and a gentleman approached and asked her for directions. She responded with the only sentence she knew: "Sorry, I don't speak Japanese."

    The gentleman was offended by this response and began scolding her for her rudeness. "Look, if you don't want to talk to me, just say so. Don't pretend like you don't speak Japanese." Since he was scolding her in Japanese, all she could do was stand there bewildered while this guy yelled at her.

    Fortunately, her friends intervened and explained to the gentleman, "No really, she doesn't speak any Japanese. We just taught her that one sentence."

    The lesson I took from this story was that when you don't speak a language, it's important to sound like you don't speak the language. In a way, it's a good thing that my German is a little bit off. That way, the person I'm talking with knows that my German is not all that great.

    Examples: During a trip to Germany, I discovered that when I asked a simple question, people would answer in rapid-fire German, overflowing my internal parsing buffer. During my trip to Sweden, I applied the lesson from this article, and found that people switched to simpler Swedish and spoke more slowly. As a result, I had little difficulty understanding what people were saying to me. (Of course, it didn't help me understand what they were saying to each other.)

  • The Old New Thing

    What were the tests that WinG did to evaluate video cards?

    • 46 Comments

    Georg Rottensteiner was curious about the weird things that WinG performed on installation to evaluate video cards. "What did it do actually and what for?"

    I don't actually know, since I was not involved in the WinG project, but I remember chatting with one of the developers who was working on video card benchmarks.

    He says that video card benchmarks are really hard to develop, not just because video cards are complicated, but also because video drivers cheat like a Mississippi riverboat card sharp on a boat full of blind tourists.

    He discovered all sorts of crazy shenanigans. Like a video driver which compares the string you ask it to display with the text "The quick brown fox jumps over the lazy dog." If the string matches exactly, then it returns without drawing anything three quarters of the time. The reason: Benchmarks often use that sample string to evaluate text rendering performance. The driver vendors realized that the fastest code is code that doesn't run, so by ignoring three quarters of the "draw this string" requests, they could improve their text rendering performance numbers fourfold.

    That was the only one of the sneaky tricks I remember from that conversation. (I didn't realize there was going to be a quiz 17 years later or I'd have taken notes.) Another example of benchmark cheating was a driver which checked if the program name was TUNNEL.EXE and if so, enabled a collection of benchmark-specific optimizations.

    Anyway, I suspect that the weird things that the WinG installer did were specifically chosen to be things that no video card driver had figured out a way to cheat, at least at the time he wrote the test. I wouldn't be surprised if fifteen seconds after WinG was released, video driver vendors started studying it to see how they could cheat the WinG benchmark...

  • The Old New Thing

    A process shutdown puzzle, Episode 2

    • 25 Comments

    A customer reported that their program would very sporadically crash in the function Close­Thread­pool­Cleanup­Group­Members. The customer was kind enough to provide a stack trace at the point of the crash:

    ntdll!RtlUnhandledExceptionFilter2+0x31e
    KERNELBASE!UnhandledExceptionFilter+0x175
    ntdll!RtlUserThreadStart$filt$0+0x3f
    ntdll!__C_specific_handler+0x8f
    ntdll!RtlpExecuteHandlerForException+0xd
    ntdll!RtlDispatchException+0x3a6
    ntdll!RtlRaiseException+0x223
    ntdll!TppRaiseInvalidParameter+0x48
    ntdll!TpReleaseCleanupGroupMembers+0x246
    litware!CThreadPool::UnInitialize+0x22
    litware!_CRT_INIT+0xbf
    litware!__DllMainCRTStartup+0x18b
    ntdll!LdrpCallInitRoutine+0x3f
    ntdll!LdrShutdownProcess+0x205
    ntdll!RtlExitUserProcess+0x90
    kernel32!ExitProcessImplementation+0xa
    contoso!wmain+0x193
    contoso!__wmainCRTStartup+0x13d
    kernel32!BaseThreadInitThunk+0xd
    ntdll!RtlUserThreadStart+0x1d
    

    The customer wondered, "Could the problem be that my cleanup group does not have a callback? MSDN seems to suggest that this is okay."

    The exception being thrown is STATUS_INVALID_PARAMETER, but that doesn't really say much.

    But that's okay, because the smoking gun isn't the exception being raised. It's in the stack.

    Do you see it?

    The code is calling Close­Thread­pool­Cleanup­Group­Members from inside DllMain while handling the DLL_PROCESS_DETACH notification. Looking further up the stack, you can see this was triggered by a call to ExitProcess, and now all the stuff you know about how processes exit kicks in.

    For example, that the first thing that happens is that all threads are forcibly terminated.

    That's your next clue.

    Observe that the customer's DLL is trying to communicate with the thread pool during process termination. But wait, all the threads have already been terminated. It's trying to communicate with a nonexistent thread pool.

    The thread pool realizes, "Hey, like I've already been destroyed. I can't do what you ask because there is no thread pool any more. You want me to block until all currently executing callback functions finish, but those callback functions will never finish (if they even exist at all) because the threads hosting their thread pool got destroyed. Not that I can tell whether they are executing or not, because I am already destroyed. The only options are to hang or crash. I think I'll crash."

    The customer needs to restructure the program so that it either cleans up its thread pool work before the ExitProcess, or it can simply skip all thread pool operations when the reason for the DLL_PROCESS_DETACH is process termination.

  • The Old New Thing

    Shortcut properties are in the shortcut, so if they can read the shortcut, they can read the properties

    • 33 Comments

    A customer wanted to know if "there was a way to hide the properties of a shortcut."

    We asked for an explanation of the problem they were trying to solve, so we could understand what their question meant. The customer liaison explained:

    The customer is insisting on this, even though I think it's really the wrong approach. They want to put a password into the parameters of a shortcut, but they don't want their employees to see the password when they right-click the shortcut and select Properties. We're trying to convince them of better ways of doing this, but right now they want to see if they can solve it by marking the field as "hidden" somehow.

    If the password is anywhere in the shortcut file, the employees can dig it out. After all, the shell needs to dig it out, and since the shell runs with the user's privileges, in order for the shell to see it, the user must be able to see it. In other words, you can't hide anything in a shortcut because the user can just open the shortcut in Notepad and see all your "hidden" data. Or they can go to Task Manager and ask to see the command line. Or they can connect a debugger to Explorer and set a breakpoint on the Create­Process function.

    It's like saying, "I want my employees to be able to bake cakes, but I don't want them to have access to an oven. To block access to the oven, I put a combination lock on the oven controls. On the other hand, I want to write a cake recipe that lets the employees bake cakes in the oven. Therefore, the recipe says Step 5: Go to the oven and press 1234. But now the employee can just read the recipe and find out the combination to the oven! Is there a way I can write a cake recipe that lets them bake a cake without revealing the oven combination?"

    The recipe executes with the privileges of the employee. If you want the employee to be able to bake a cake by following the recipe, then they need to be able to perform the steps in the recipe, and that means being able to go to the oven and press 1234.

    The oven analogy does provide some ideas on how you can solve the problem. For example, if you simply don't want employees to be able to email the oven combination to their friends with the subject line Here's the combination to the oven!, then change the way access to the oven is managed. Instead of putting a combination lock on the oven, put an employee ID card scanner on the oven that enables the oven controls if the employee has oven privileges.

    For the original problem, this means changing your database so that instead of using a single password to control access and trusting each client to use it wisely, it uses the security identity of the client to control access. (I'm assuming that the password on the command line is a database password.)

    On the other hand, if your goal is to prevent employees from using the oven to do anything other than bake at 350°F for one hour, you can change the employee ID card scanner so that it checks the employee for cake-baking privileges, and if so, sets the oven to bake for 350°F for one hour and locks the controls (except perhaps for a cancel button). If you have multiple recipes—say, cakes and cookies—the ID card scanner checks which recipes the employees are allowed to use and lets them choose which preset they want to activate.

    For the original problem, this means changing your database so that the user identity selects which operations are permitted on the database. Some users have permission to see only records for active clients, whereas others have permission to see all records, and still others have modify permission.

    If your goal is to prevent employees from doing anything other than baking cakes according to this specific recipe, then you need to move even more actions "behind the counter", because you have no way of knowing that "that pan full of batter" was created according to your walnut cake recipe, or whether it's some unauthorized recipe with extra cinnamon. If you don't trust your employees to follow recipes, then you need to take the recipe out of their hands. The instructions are now Step 1: Order a walnut cake from the cafeteria.

    For the original problem, this means changing your database so that instead of letting the employee access records directly, the employee submits the action to the database ("change client 23 address to 123 Main Street"), and the database verifies that the employee has "change a client address" permission, and if so, performs the record update.

    Of course, if you want to ignore all the security guidance and "just hide the password in the shortcut file, why won't you answer my question?", you can just put the password somewhere other than the shortcut file. You could, say, have the shortcut run a batch file, and the batch file has the password.

  • The Old New Thing

    How do I prevent users from opening TIF files?

    • 30 Comments

    A customer had a question about their Windows XP installations. (This question was from several years ago, so the fine details aren't really relevant any more, but I'm actually telling this story for a commentary opportunity.)

    The customer wanted to disable all file associations for TIFF files. Their first attempt was by deleting HKEY_CLASSES_ROOT\.tif and HKEY_CLASSES_ROOT\.tiff. This successfully renders TIFF files with a generic document icon, but when the user double-clicks the file, the registration is re-established and Windows Picture and Fax Viewer opens the file.

    The company had some strange company security policy that says that TIFF files should not have any file association. I don't know the rationale behind it, but they did say that they only needed to block the default file association. If the user explicitly creates a new association via the Open With dialog, then that is not covered by the policy.

    Deleting the registrations doesn't work because Windows XP has an autorepair feature for certain commonly-corrupted file associations, and TIFF is one of them. If the TIFF registration is corrupted and the user is a member of the Administrators group, then Windows XP will restore the default association. (If the user is not a member of the Administrators group, then the usual "Windows cannot open this file" dialog box appears.)

    Therefore, the solution to the customer's odd problem is not to delete the TIFF registrations (which causes them to be detected as corrupted) but rather to simply set a new default handler for TIFF files that merely displays a message to the user. If you're willing to use the Windows Script Host, then it's a one-line program:

    WScript.Echo("TIFF file associations are disabled.")
    

    If you have been reading carefully, you have already noticed a serious problem with the customer's configuration: The fact that they are seeing the TIFF autorepair code kicking in means that they are letting their employees run with Administrator privileges, which means that their so-called "security requirement" is like worrying about employees being able to sneak into the building through a ventilation grating, when you give everybody a key to the front door.

  • The Old New Thing

    I thought I was so clever, salvaging an old floppy drive from a dead computer, but I didn't think *two* steps ahead...

    • 37 Comments

    When one of the oldest computers at Microsoft still doing useful work finally died, I had the presence of mind to salvage the 5¼″ floppy drive from the machine, so that I could (someday) extract the data off all the old 5¼″ floppy discs I have packed away in boxes meaning to convert someday. (Mind you, the data capacity of a giant box of 5¼″ floppy disks is approximately equal to half of a CD.)

    Oh, and by the way, if you know what a floppy drive is, then this question on superuser.com will make you feel old.

    I thought I was so clever, salvaging an old floppy drive from a dead computer so I could use it to rescue data from obsolescence, but that was only thinking one step ahead. I failed to think two steps ahead: Nobody makes motherboards with 5¼″ floppy drive connectors!

    Bonus coincidental posting date: The Geeks Who Saved Prince of Persia's Source Code From Digital Death.

  • The Old New Thing

    Why are programs allowed to use normal characters as hotkeys?

    • 36 Comments

    alv wonders why programs are allowed to create hotkeys for normal characters. The example given is the famous AltGr modifier, equivalent to Ctrl+Alt. Programs which ignore the guidance and use Ctrl+Alt as a shortcut modifier end up stealing keys from many non-English keyboard layouts, thereby preventing users from typing things like the Euro sign € (which is an AltGr combination on most keyboards that support it), or even worse, preventing Polish users with the Polish programmer keyboard layout from typing the letter Ż.

    Given that using Ctrl+Alt as a keyboard shortcut modifier can prevent people from typing perfectly legitimate letters, why does Windows allow it in the first place?

    Because there are many cases where it is acceptable to commandeer keyboard sequences that would otherwise result in normal typing. You do it all the time and probably don't realize it.

    For example, in an edit control, the Ctrl+A shortcut is frequently overridden to mean select all instead of entering a literal U+0001 into the edit control text. You probably would go crazy if you lost the Ctrl+C, Ctrl+X and Ctrl+V hotkeys. Ctrl+Z for undo is also pretty popular. And who can forget TAB for dialog box navigation. And imagine the riots if ESC couldn't be used as a hotkey any more.

    There are some programs which go so far as to use normal typing characters as hotkeys. For example, an email program might let you compose a new message by simply pressing N. That's just the letter N, unshifted, unmodified.

    Given that there are legitimate reasons for allowing normal typing to be treated as a hotkey, implementing a global ban on that behavior would break a lot of scenarios that you personally would almost certainly want to keep working. Of course, it also means that it's the programmer's responsibility to think twice before stealing a keyboard sequence that can be used for normal typing. If the user is actually typing, you may have stolen something you should have left alone.

    You can't even have the rule "all shortcut keys which correspond to normal typing are automatically disabled when focus is on an edit control" because you would lose hotkeys like Ctrl+C and TAB, as noted above, as well as scenario-specific hotkeys such as having the IPv4 control automatically jump to the next octet when you press ..

  • The Old New Thing

    What does INIT_ONCE_CTX_RESERVED_BITS mean?

    • 11 Comments

    Windows Vista adds the One-Time Initialization family of functions which address a common coding pattern: I want a specific chunk of code to run exactly once, even in the face of multiple calls from different threads. There are many implementations of this pattern, such as the infamous double-checked lock. The double-checked lock is very easy to get wrong, due to memory ordering and race conditions, so the kernel folks decided to write it for you.

    The straightforward way of using a one-time-initialization object is to have it protect the initialization of some other object. For example, you might have it protect a static object:

    INIT_ONCE GizmoInitOnce = INIT_ONCE_STATIC_INIT;
    Gizmo ProtectedGizmo;
    
    BOOL CALLBACK InitGizmoOnce(
        PINIT_ONCE InitOnce,
        PVOID Parameter,
        PVOID *Context)
    {
        Gizmo *pGizmo = reinterpret_cast<Gizmo*>(Parameter);
        pGizmo->Initialize();
        return TRUE;
    }
    
    SomeFunction(...)
    {
        // Initialize ProtectedGizmo if not already initialized
        InitOnceExecuteOnce(&GizmoInitOnce,
                            InitGizmoOnce,
                            &ProtectedGizmo,
                            NULL);
    
        // At this point, ProtectedGizmo has been initialized
        ProtectedGizmo.Something();
        ...
    }
    

    Or you might have it protect a dynamic object:

    class Widget
    {
        Widget()
        {
            InitOnceInitialize(&m_InitOnce);
        }
    
        void Initialize();
    
        ...
    
        static BOOL CALLBACK InitWidgetOnce(
            PINIT_ONCE InitOnce,
            PVOID Parameter,
            PVOID *Context)
        {
            Widget *pWidget = reinterpret_cast<Widget*>(Parameter);
            pWidget->Initialize();
            return TRUE;
        }
    
        SomeMethod(...)
        {
            // Initialize ourselves if not already initialized
            InitOnceExecuteOnce(&InitWidgetOnce,
                                this,
                                NULL);
    
            // At this point, we have been initialized
            ... some other stuff ...
        }
    }
    

    But it so happens that you can also have the INIT_ONCE object protect itself.

    You see, once the INIT_ONCE object has entered the "initialization complete" state, the one-time initialization code only needs a few bits of state. The other bits are unused, so the kernel folks figured, "Well, since we're not using them, maybe the application wants to use them."

    That's where INIT_ONCE_CTX_RESERVED_BITS comes in. The INIT_ONCE_CTX_RESERVED_BITS value is the number of bits that the one-time initialization code uses after initialization is complete; the other bits are free for you to use yourself. The value of INIT_ONCE_CTX_RESERVED_BITS is 2, which means that you can store any value that's a multiple of 4. If it's a pointer, then the pointer must be DWORD-aligned or better. This requirement is usually easy to meet because heap-allocated objects satisfy it, and the pointer you want to store is usually a pointer to a heap-allocated object. As noted some time ago, kernel object handles are also multiples of four, so those can also be safely stored inside the INIT_ONCE object. (On the other hand, USER and GDI handles are not guaranteed to be multiples of four, so you cannot use this trick to store those types of handles.)

    Here's an example. First, the code which uses the traditional method of having the INIT_ONCE structure protect another variable:

    // using the static object pattern for simplicity
    
    INIT_ONCE PathInitOnce = INIT_ONCE_STATIC_INIT;
    LPWSTR PathToDatabase = NULL;
    
    BOOL CALLBACK InitPathOnce(
        PINIT_ONCE InitOnce,
        PVOID Parameter,
        PVOID *Context)
    {
        LPWSTR Path = (LPWSTR)LocalAlloc(LMEM_FIXED, ...);
        if (Path == NULL) return FALSE;
        ... get the path in Path...
        PathToDatabase = Path;
        return TRUE;
    }
    
    SomeFunction(...)
    {
        // Get the database path (initializing if necessary)
        if (!InitOnceExecuteOnce(&PathInitOnce,
                                 InitPathOnce,
                                 NULL,
                                 NULL)) {
            return FALSE; // couldn't get the path for some reason
        }
    
        // The "PathToDatabase" variable now contains the path
        // computed by InitPathOnce.
    
        OtherFunction(PathToDatabase);
        ...
    }
    

    Since the object being protected is pointer-sized and satisfies the necessary alignment constraints, we can merge it into the INIT_ONCE structure.

    INIT_ONCE PathInitOnce = INIT_ONCE_STATIC_INIT;
    
    BOOL CALLBACK InitPathOnce(
        PINIT_ONCE InitOnce,
        PVOID Parameter,
        PVOID *Context)
    {
        LPWSTR Path = (LPWSTR)LocalAlloc(LMEM_FIXED, ...);
        if (Path == NULL) return FALSE;
        ... get the path in Path...
        *Context = Path;
        return TRUE;
    }
    
    SomeFunction(...)
    {
        LPWSTR PathToDatabase;
        // Get the database path (initializing if necessary)
        if (!InitOnceExecuteOnce(&PathInitOnce,
                                 InitPathOnce,
                                 NULL,
                                 &PathToDatabase)) {
            return FALSE; // couldn't get the path for some reason
        }
    
        // The "PathToDatabase" variable now contains the path
        // computed by InitPathOnce.
    
        OtherFunction(PathToDatabase);
        ...
    }
    

    This may seem like a bunch of extra work to save four bytes (or eight bytes on 64-bit Windows), but if you use the asynchronous initialization model, then you have no choice but to use context-based initialization, as we learned when we tried to write our own lock-free one-time initialization code.

  • The Old New Thing

    Why don't I get a Caps Lock warning balloon any more?

    • 32 Comments

    A customer asked for help diagnosing a problem they were experiencing on Windows XP:

    My customer reports that on their machines, they do not get the warning balloon that appears when Caps Lock is set while you are typing into a password field. I searched for relevant KB articles but couldn't find anything related to that. Can you help?

    Time for the psychic powers.

    My psychic powers tell me that the customer disabled all balloon tips.

    The customer liaison replied

    You are right. Thanks for the help.

    This is a not uncommon situation with some customers. They change a setting, and then later report that they're having some problem caused by that setting. They don't bother going to a freshly-installed machine to see whether the problem occurs there as well, in order to isolate whether the problem is related to their customizations or not. They just assume that their customizations couldn't possibly be the cause of the problem, and they are so convinced of this that they don't even mention "Oh, we customized this setting" when they ask for help.

    By the way, the setting to disable all balloon tips in the entire system is a rogue feature. At the time that balloons were originally being developed (and the rules surrounding them being refined based on research and feedback), one developer was impatient with the progress toward making the balloons less annoying, and he just went in and added a rogue feature to disable them with a sledgehammer. And now that people know about it, the rogue feature has become a support burden.

  • The Old New Thing

    Why doesn't the Version tab show up for very large files?

    • 40 Comments

    If you have a really large file and try to view its properties in Explorer, you may find that the Version tab doesn't appear. What's going on?

    The Version tab uses the GetFileVersionInfo family of functions to obtain version information from files. It so happens that the GetFileVersionInfo function works by calling LoadLibraryEx(LOAD_LIBRARY_AS_DATAFILE) and then using functions like FindResource to locate the version resource so it can allocate a buffer to hold the version resource plus additional data to assist in character set translation.

    If the file is larger than the available address space in the Explorer process, then the call to LoadLibraryEx will fail due to lack of address space into which to map the image. Library not loaded means no version resource to show you.

    When we explained this behavior to a customer, the customer wanted to know the exact file size at which the problem occurs.

    There is no single exact file size at which the problem occurs because the amount of available address space is determined by large numbers of factors which cannot be boiled down to a simple description. It depends on the pattern of memory allocation that took place inside Explorer, which shell extensions and DLLs got loaded at what point in time, where they got relocated to, which ones got unloaded relative to others loading, how much memory they allocated and where those allocations ended up.

    It's like asking the airline, "I know that the later I make my reservation, the less likely I'm going to get the exact seat I want. What is the cutover point before which I will get a window seat and after which I won't?" There is no specific point in time where "all reservations before this point will definitely get a window seat, and all reservations after this point will definitely not get one." It all depends on the pattern of requests before you make your reservations. Indeed, it's even possible that if you had made your reservation later, you would have gotten that window seat, because somebody who had a window seat called in a cancellation.

    One customer apparently didn't understand the unpredictability of the cutover point and asked, "Is there a way to change this limit?"

Page 1 of 3 (22 items) 123