May, 2007

  • The Old New Thing

    Revelations of My Friends: An ancestor of today's Mad Libs


    Literary detective Paul Collins stumbled upon a precursor to today's Mad Libs known as Revelations of My Friends. Listen to the NPR story to hear Scott Simon play, and go to Paul Collins' web site to see some of the drawing that Scott laughed at.

  • The Old New Thing

    Don't be helpless: At least look at the function you're using


    Sometimes I see people ask a question and get an answer, but the answer doesn't quite work. But instead of trying to understand the answer in order to see why it doesn't work and develop a better solution, they just play stupid. Here's an example. The names have been changed but the story's the same.

    How do I get a handle to a bright pink elephant? We have a dialog box that talks about elephants, and we'd like to put some information about the bright pink ones on it.

    An answer came back rather quickly.

    Use GetElephantHandle(EPHNT_BRIGHTPINK).

    Some time later, the customer came back with a follow-up question.

    I can't get it to work. I get a compiler error:
    error C2065: 'EPHNT_BRIGHTPINK' : undeclared identifier

    Am I missing a header file?

    (So much for people knowing what to do when they get this error message.)

    Don't be helpless. Your head isn't just for looks. At least pretend to try to understand what you're doing. In this case, the function is GetElephantHandle, and you're having trouble because the EPHNT_BRIGHTPINK value we want isn't defined. Why not check out the values that are defined?

    #define EPHNT_DUMBO         0
    #define EPHNT_BABAR         1
    #define EPHNT_WHITE         2
    #define EPHNT_BRIGHT_PINK   3

    Wow, I bet the person who wrote EPHNT_BRIGHTPINK really meant EPHNT_BRIGHT_PINK.

    Armed with this new skill, perhaps you can solve this person's problem:

    Somebody recommended that I add the DS_SHELLEXT style to my property sheet dialog template to fix a sizing problem, but when I try that, I get an error that tells me that DS_SHELLEXT is not defined. Is there a file I have to include?
  • The Old New Thing

    Session 0 isolation: Where backward compatibility loses to security


    One of the major changes to services in Windows Vista is session 0 isolation. After reading the summary, you can follow that first supplementary link, Impact of Session 0 Isolation on Services and Drivers in Windows Vista, to dig deeper and receive guidance on how you need to modify your service.

    Then again, some of the questions I see regarding session 0 reveal that they were relying on behavior that wasn't true even in Windows XP. Many people† assume that session 0 is the one connected to the physical keyboard, but there is no such guarantee. If you turn on Fast User Switching and have multiple users logged on, the second and subsequent users will be on sessions other than session 0, even though they are at the physical keyboard when they log in. Conversely, if you use Remote Desktop Connection to connect to a Windows XP machine, you can connect to session 0 remotely. So whatever they were doing, it was already broken.

    Nitpicker's corner

    †The phrase "many people" means "many people". Microsoft employees fall under the category of people. (I can't believe people are nitpicking the nitpicker's corner.)

  • The Old New Thing

    The administrator is an idiot


    Nearly all computer administrators are idiots.

    That's not because the personnel department is incompetent or because it's impossible to train competent administrators. It's because, for a consumer operating system, the computer administrator didn't ask to be one. In nearly all cases, the computer administrator is dad or grandma.† They didn't ask to be to be the computer administrator. They just want to surf the web and read email from Jimmy.‡

    All this means is that you can't say, "Well, if the user is an administrator, as opposed to a normal user, then it's okay to show them all these dangerous things (such as critical operating system files) because they know what they're doing." Grandma doesn't know what she's doing.

    For a consumer operating system, a friendly user interface means protecting the administrators from themselves.

    Nitpicker's corner

    Sigh. One article without a nitpicker's corner and look what happens.

    †The words "dad" and "grandma" refer to archetypes for non-technical home users and are not intended to be interpreted as literally dad and grandma.

    ‡Not all grandchildren are named Jimmy.

  • The Old New Thing

    Metromint: What were they thinking?


    Some friends gave me a bottle of spearmint Metromint as a gift. And as it turns out, it was a mean-spirited gift.

    Let's look at that bottle. It calls itself "pure, simple mintwater®". What the heck is pure mintwater? Do you go to the local mintwater stream and collect it? Oh, wait, sorry. That should be mintwater®.

    With some trepidation, I took a sip. Yuck-o. It's like drinking chewing gum. Why would anybody want this?

    But wait, there's more. On the side of the bottle, there's a picture of a thermometer, and the mercury reads −6°, or "extra cool." Whatever that means. The bottle never explains.

    On their benefits page, if you click through to "rehydrate", it says that "the good stuff in Metromint gets where it's going quicker, and goes to work faster." Huh? What's the scientific basis for this claim? Quicker than what? And the biggest mystery of them all: What part of Metromint is the "good stuff"? I haven't found it yet.

    Whatever. My point is that this stuff is awful. And I'm not the only person who thinks so.

  • The Old New Thing

    Drag and drop is a general purpose transfer model, not exclusive to Explorer directories


    If you've designed your program so that it assumes that the the only thing a user can use drag/drop for is dropping your object into a file system directory, then you've already lost.

    piers wants to be able to determine the destination of a drag/drop operation. From the description, it appears that what piers really wants is the destination path, but who says that the drop destination is a directory? The user might have dropped the item into an email message, onto an FTP site, onto a Web site (via Web Folders), or even a directory on an operating system running inside a virtual machine!

    The follow-up question makes things even more confusing. If the user drops the files into an FTP site or some other virtual folder, how is your program supposed to be able to restart the transfer? You don't know the user's password on that FTP site. You don't know how to restart that virtual machine and log the user on. And even if you did, you don't know how to write to a directory on a virtual machine; only the virtual machine manager knows how to do that. There are an infinite variety of potential virtual folders out there; I doubt you know how to (or even have the ability to) push your data into each one.

    Once the user drops the data object, the remainder of the transfer is a private matter between the data source and the drop target. It's not like a data source can tell all drop targets, "I want to take over the transfer," because even if the drop target agreed to it, that still leaves the data source the problem of figuring out how to carry out that take-over.

    What is my recommendation?‡

    The data object in the drag/drop loop should follow the standard shell data object transfer protocol so that users can drop the object into an email message, onto an FTP site, etc.°

    For the special bonus behavior, I would create a drag/drop hook. A user that wants to do a transfer mediated by your program can use the right mouse button† to drag. When the drop occurs, a context menu will appear, including the drag/drop hook you created. That hook would create a new item called something like "Copy with CoolProgram". (Of course, the hook adds this item only if the data object identifiers itself as coming from CoolProgram.) If the user selects "Copy with CoolProgram", then you can do your CoolProgram-mediated copy.

    Nitpicker's corner

    †More properly, the secondary mouse button, since you may have swapped buttons.

    Notice that I do not assert that all Microsoft products follow my recommendation. Note also that this is my personal recommendation, not the official position of Microsoft Corporation.

    °And you should already understand the standard shell data object transfer protocol before you go off and design a nonstandard one.

  • The Old New Thing

    But they're not (puts on sunglasses) going to get away with murder


    I don't even watch CSI: Miami, but this sequence of Endless Caruso One Liners cracks me up. The Wikipedia entry for Horatio Crane acknowledges the awesome power of the Sunglasses of Justice, and even Jim Carrey gets into the act.

  • The Old New Thing

    How my lack of understanding of how processes exit on Windows XP forced a security patch to be recalled


    Last year, a Windows security update got a lot of flack for causing some machines to hang, and it was my fault. (This makes messing up a demo at the Financial Analysts Meeting look like small potatoes.)

    The security fix addressed a category of attacks wherein people could construct shortcut files or other items which specified a CLSID that was never intended to be used as a shell extension. As we saw earlier, lots of people mess up IUnknown::QueryInterface, and if you pass the CLSID of one of these buggy implementations, Explorer would dutifully create it and try to use it, and then bad things would happen. The object might crash or hang or even corrupt memory and keep running (sort of).

    To protect against buggy shell extensions, Explorer was modified to use a helper program called verclsid.exe whose job was to be the "guinea pig" and host the shell extension and do some preliminary sniffing around to make sure the shell extension passed some basic functionality tests before letting it run loose in Explorer. That way, if the shell extension went crazy, the victim would be the verclsid.exe process and not the main Explorer process.

    The verclsid.exe program created a watchdog thread: If the preliminary sniffing took too long, the watchdog assumed that the shell extension was hung and the watchdog told Explorer, "Don't use this shell extension."

    I was one of the people brought in to study this new behavior, poke holes in its design, poke holes in its implementation, review every line of code that changed and make sure that it did exactly what it was supposed to do without introducing any new bugs along the way. We found some issues, testers found some other issues, and all the while, the clock was ticking since this was a security patch and people enjoy mocking Microsoft over how long it takes to put a security patch together.

    The patch went out, and reports started coming in that machines were hanging. How could that be? We created a watchdog thread specifically to catch the buggy shell extensions that hung; why isn't the watchdog thread doing its job?

    That was a long set-up for today's lesson.

    After running its sanity tests, the verclsid.exe program releases the shell extension, un-initializes COM, and then calls ExitProcess with a special exit code that means, "All tests passed." If you read yesterday's installment, you already know where I messed up.

    The DLL that implemented the shell extension created a worker thread, so it did an extra LoadLibrary on itself so that it wouldn't get unloaded when COM freed it as part of CoUninitialize tear-down. When the DLL got its DLL_PROCESS_DETACH, it shut down its worker thread by the common technique of setting a "clean up now" event that the worker thread listened for, and then waiting for the worker thread to respond with a "Okay, I'm all done" event.

    But recall that the first stage in process exit is the termination of all threads other than the one that called ExitProcess. That means that the DLL's worker thread no longer exists. After setting the event to tell the (nonexistent) thread to clean up, it then waited for the (nonexistent) thread to say that it was done. And since there was nobody around listening for the clean-up event, the "all done" event never got set. The DLL hung in its DLL_PROCESS_DETACH.

    Why didn't our watchdog thread save us? Because the watchdog thread got killed too!

    Now, the root cause for all this was a buggy shell extension that did bad things in its DLL_PROCESS_DETACH, but blaming the shell extension misses the point. After all, it was the fact that there existed buggy shell extensions that created the need for the verclsid.exe program in the first place.

    Welcome Slashdot readers. Since you won't read the existing comments before posting your own, I'll float some of the more significant ones here.

    The buggy shell extension was included with a printer driver for a printer that is no longer manufactured. Good luck finding one of those in your test suite.

    The security update was recalled and reissued in a single action, which most people would call an update or refresh, but the word recall works better in a title.

  • The Old New Thing

    Unexpected consequences of self-checkout


    I heard an interesting report on Marketplace on surprises in the self-checkout lane. Impulse buying is down, and stores have come up with other ways to entice you into buying something you hadn't planned. And it turns out that fears from retailers that customers would cheat at the self-checkout turned out to be misplaced:

    The reality of the situation is that most losses or theft come from the employees themselves. One of the things that we're realizing is actually that customers are more honest than the people that are working there.
  • The Old New Thing

    Quick overview of how processes exit on Windows XP


    Exiting is one of the scariest moments in the lifetime of a process. (Sort of how landing is one of the scariest moments of air travel.)

    Many of the details of how processes exit are left unspecified in Win32, so different Win32 implementations can follow different mechanisms. For example, Win32s, Windows 95, and Windows NT all shut down processes differently. (I wouldn't be surprised if Windows CE uses yet another different mechanism.) Therefore, bear in mind that what I write in this mini-series is implementation detail and can change at any time without warning. I'm writing about it because these details can highlight bugs lurking in your code. In particular, I'm going to discuss the way processes exit on Windows XP.

    I should say up front that I do not agree with many steps in the way processes exit on Windows XP. The purpose of this mini-series is not to justify the way processes exit but merely to fill you in on some of the behind-the-scenes activities so you are better-armed when you have to investigate into a mysterious crash or hang during exit. (Note that I just refer to it as the way processes exit on Windows XP rather than saying that it is how process exit is designed. As one of my colleagues put it, "Using the word design to describe this is like using the term swimming pool to refer to a puddle in your garden.")

    When your program calls ExitProcess a whole lot of machinery springs into action. First, all the threads in the process (except the one calling ExitProcess) are forcibly terminated. This dates back to the old-fashioned theory on how processes should exit: Under the old-fashioned theory, when your process decides that it's time to exit, it should already have cleaned up all its threads. The termination of threads, therefore, is just a safety net to catch the stuff you may have missed. It doesn't even wait two seconds first.

    Now, we're not talking happy termination like ExitThread; that's not possible since the thread could be in the middle of doing something. Injecting a call to ExitThread would result in DLL_THREAD_DETACH notifications being sent at times the thread was not prepared for. Nope, these threads are terminated in the style of TerminateThread: Just yank the rug out from under it. Buh-bye. This is an ex-thread.

    Well, that was a pretty drastic move, now, wasn't it. And all this after the scary warnings in MSDN that TerminateThread is a bad function that should be avoided!

    Wait, it gets worse.

    Some of those threads that got forcibly terminated may have owned critical sections, mutexes, home-grown synchronization primitives (such as spin-locks), all those things that the one remaining thread might need access to during its DLL_PROCESS_DETACH handling. Well, mutexes are sort of covered; if you try to enter that mutex, you'll get the mysterious WAIT_ABANDONED return code which tells you that "Uh-oh, things are kind of messed up."

    What about critical sections? There is no "Uh-oh" return value for critical sections; EnterCriticalSection doesn't have a return value. Instead, the kernel just says "Open season on critical sections!" I get the mental image of all the gates in a parking garage just opening up and letting anybody in and out. [See correction.]

    As for the home-grown stuff, well, you're on your own.

    This means that if your code happened to have owned a critical section at the time somebody called ExitProcess, the data structure the critical section is protecting has a good chance of being in an inconsistent state. (Afer all, if it were consistent, you probably would have exited the critical section! Well, assuming you entered the critical section because you were updating the structure as opposed to reading it.) Your DLL_PROCESS_DETACH code runs, enters the critical section, and it succeeds because "all the gates are up". Now your DLL_PROCESS_DETACH code starts behaving erratically because the values in that data structure are inconsistent.

    Oh dear, now you have a pretty ugly mess on your hands.

    And if your thread was terminated while it owned a spin-lock or some other home-grown synchronization object, your DLL_PROCESS_DETACH will most likely simply hang indefinitely waiting patiently for that terminated thread to release the spin-lock (which it never will do).

    But wait, it gets worse. That critical section might have been the one that protects the process heap! If one of the threads that got terminated happened to be in the middle of a heap function like HeapAllocate or LocalFree, then the process heap may very well be inconsistent. If your DLL_PROCESS_DETACH tries to allocate or free memory, it may crash due to a corrupted heap.

    Moral of the story: If you're getting a DLL_PROCESS_DETACH due to process termination,† don't try anything clever. Just return without doing anything and let the normal process clean-up happen. The kernel will close all your open handles to kernel objects. Any memory you allocated will be freed automatically when the process's address space is torn down. Just let the process die a quiet death.

    Note that if you were a good boy and cleaned up all the threads in the process before calling ExitThread, then you've escaped all this craziness, since there is nothing to clean up.

    Note also that if you're getting a DLL_PROCESS_DETACH due to dynamic unloading, then you do need to clean up your kernel objects and allocated memory because the process is going to continue running. But on the other hand, in the case of dynamic unloading, no other threads should be executing code in your DLL anyway (since you're about to be unloaded), so—assuming you coded up your DLL correctly—none of your critical sections should be held and your data structures should be consistent.

    Hang on, this disaster isn't over yet. Even though the kernel went around terminating all but one thread in the process, that doesn't mean that the creation of new threads is blocked. If somebody calls CreateThread in their DLL_PROCESS_DETACH (as crazy as it sounds), the thread will indeed be created and start running! But remember, "all the gates are up", so your critical sections are just window dressing to make you feel good.

    (The ability to create threads after process termination has begun is not a mistake; it's intentional and necessary. Thread injection is how the debugger breaks into a process. If thread injection were not permitted, you wouldn't be able to debug process termination!)

    Next time, we'll see how the way process termination takes place on Windows XP caused not one but two problems.


    †Everybody reading this article should already know how to determine whether this is the case. I'm assuming you're smart. Don't disappoint me.

Page 4 of 5 (44 items) 12345