October, 2012

  • The Old New Thing

    Why does ShellExecute return SE_ERR_ACCESSDENIED for nearly everything?


    We saw a while ago that the Shell­Execute function returns SE_ERR_ACCESS­DENIED at the slightest provocation. Why can't it return something more meaningful?

    The short-term answer is that the return value from Shell­Execute is both a success code and an error code, and you check whether the value is greater than 32 to see which half you're in. In particular, the error code case is if the value you got is less than or equal to 32. This already demonstrates that the error codes are limited to values less than or equal to 32. And all those error codes are already accounted for, so there's nowhere to stick "an error not on the original list of 32 possible error codes." Therefore, any error that wasn't on the original list of error codes gets turned into SE_ERR_ACCESS­DENIED, in the same way that MS-DOS turned any error that didn't map to one of its original errors into 5 (access denied).

    Okay, but why was 32 chosen as the cut-off?

    The Shell­Execute function didn't come up with that number. That number came from the kernel folks, who decided that Win­Exec function returned the handle to the application that was executed on success, or an error code less than 32 on failure. And back in the old days, Shell­Execute was just a function that called Find­Executable and then passed the result to Win­Exec, so following the Win­Exec pattern made sense.

    (You may have noticed a tiny discrepancy there. The shell folks decided to add a new error code SE_ERR_DLL­NOT­FOUND with a numeric value of 32, thereby making the return value from Shell­Execute behave subtly differently from that of Win­Exec. The people who made this decision probably regretted it once it became clear that lots of applications were checking the return value incorrectly, but it's too late to fix it now.)

    Okay, so let's peel back another layer: Why did the Win­Exec function overload the return value? Well, overloaded return values were all the rage back then. A lot of functions to create something return the created object on success, or null on failure. The kernel folks said, "Well, we can do even better than that. Not only can we tell you that we failed to create the application, we can even tell you why! You see, MS-DOS has a maximum of 31 error codes, so we can just return the error code directly if we can ensure that no values less than 32 are valid segments. And we can make that guarantee because the 8086 processor reserves the first 1024 bytes of memory (the first 64 segments) for its interrupt vector table, so no application could possibly be loaded there. Hooray! We're such over-achievers!"

    This weird way of reporting errors from Shell­Execute has been preserved for compatibility. New applications would probably better served to switch to the Shell­Execute­Ex function instead, since it reports errors by calling Set­Last­Error with the real error code before returning. (In other words, you can call Get­Last­Error to get the real error code.)

    Bonus chatter: Wait a second, if Get­Last­Error gets you the real error code, how come the original report was that the Shell­Execute­Ex function also returns SE_ERR_ACCESS­DENIED?

    Because it depends on what you mean by "returns". Technically speaking, the Shell­Execute­Ex function returns FALSE for all errors, since it is prototyped as returning a BOOL. When somebody says that it returns an error code, you first have to ask where they got that error code from.

    If they got it from Get­Last­Error, then they'll get a meaningful error code, or at least something more meaningful than SE_ERR_ACCESS­DENIED.

    But if instead they look at the hInstApp member of the SHELL­EXECUTE­INFO structure, then they'll get that useless SE_ERR_ACCESS­DENIED value again. Because the hInstApp is where the legacy return value is recorded. If you look there, you're going to see the old lame error code. So don't look there.

  • The Old New Thing

    If there's a problem with a wiki, then you can fix it; that's why it's a wiki


    On an internal mailing list, somebody asked a question about how to do X with Y, and I replied with a link to an internal wiki that described how to do X with Y (answer: use the Z tool).

    Somebody else replied, "Time to update that article because the link to the Z tool is broken."

    Apparently, this person forgot that one of the defining characteristics of a wiki is that it's easy to edit. (Another characteristic is that it is collaboratively-edited; there is no central authority.)

    In other words, if you see something wrong, fix it yourself. Don't just stand around saying somebody should do something. Be someone. Because on a wiki, there is no default value for somebody.

  • The Old New Thing

    Irony patrol: Recycling bins


    Microsoft has a large corporate recycling effort. Every office, every mail room, every kitchenette, every conference room has a recycling bin. The dining facilities earned Green Restaurant Certification, and there is a goal of making the cafeterias a zero-landfill facility by 2012. (Hey, that's this year!)

    A few years ago, I found one room in my building that didn't have a recycling bin, and you'd think it'd be one of the rooms near the top of the list for needing one.

    The room without a recycling bin was the copy machine room.

    As a result, people were throwing their unwanted cover sheets and other paper waste into the regular garbage.

    I decided to be somebody. I took a recycling bin from an unused office and moved it into the copy room.

    Bonus recycling bin irony: For many years, each office had three recycling bins, each labeled for its intended contents: white paper, mixed paper, and aluminum cans. Improvements in automated sorting technology removed the need to separate these recyclables manually, and in 2008, all three recycling bins were replaced with a single recycle bin, which was labeled with the simple three-arrow recycling logo.

    The irony is that Microsoft was going to toss all the old recycling bins into a landfill because they couldn't find anybody who wanted them.

    Alert Microsoft employee Tom Roth found the right people in Building Facilities and got them to stop the trucks as they were about 100 feet from dumping 40,000 perfectly good plastic bins into a landfill. Tom's son Justin works in the recycling industry, and he used his contacts to get the word out, and soon requests for recycling bins were coming in from all over the state of Washington. It took three months, but they eventually found homes for all of the recycle bins.

    Ironic disaster averted.

  • The Old New Thing

    The cries of "Oh no!" emerge from each office as the realization slowly dawns


    Today is the (approximate) 15th anniversary of the Bedlam Incident. To commemorate that event, here's a story of another email incident gone horribly awry.

    Some time ago, an email message was sent to a large mailing list. It came from somebody in the IT department and said roughly, "This is a mail sent on behalf of Person X to check if your XYZ server has migrated to the new datacenter. Please visit http://blahblah and confirm that your server name is of the form XXYYNNZZ. If not, please Reply to All."


    The seasoned Microsoft employees (and the new employees who paid attention during new employee orientation) recognized the monster that was about to be unleashed, and the cries of "Oh no!" could be heard emerging from each office as the realization dawned.

    And then it started. All the replies from people saying "I'm still on the old datacenter." And then the replies from people saying, "Stop replying!"

    What's frustrating is that you can't do anything about the catastrophe that is unfolding. Any attempt to reply to the message telling people to stop replying only makes the problem worse. All you can do is stand back and wait for the fire to burn itself out.

    Ten minutes later, Person X sent a message to the mailing list. "Please DO NOT Reply all to this email thread. I am working with the IT department to see if there is another way to get this information."

    It took another ten minutes for the messages to finally stop, but that seems to have shut things down. Now it's time for blame and speculation!

    We were never told whose brilliant idea it was to try to gather information by sending mail to 7000 people telling them to reply all. One theory was that Person X went to the IT department saying "Hi, I'd like to collect information XYZ from this large list of people. Can you help?" And some new hire in the IT department said, "Sure, I can get that information for you. I'll just send everybody some email!"

    After the dust settled, somebody made a tally of the damage.

    Number of people on the mailing list: around 7000.
    Number of replies: 70.
    Of those, number of replies saying "stop replying": 17.

    To commemorate this event, a colleague of mine who maintains a popular internal tool pushed out an upgraded version. The new version had a checkbox on the main page:

    I have not been migrated to the new datacenter.

    Bonus chatter: It so happens that the message was sent at the beginning of the summer, and most of the "I'm still on the old datacenter" replies came from summer interns. Maybe it was a test.

  • The Old New Thing

    What happens if you forget to pass an OVERLAPPED structure on an asynchronous handle?


    A customer noticed that they were accidentally issuing some I/O's against an overlapped handle without using an OVERLAPPED structure. The calls seemed to work, and the program did not appear to suffer any ill effects, but they were worried that they just being lucky and that eventually the error will come back to bite them. So what really happens if you forget to pass an OVERLAPPED structure on an asynchronous handle?

    Well, the layer of the kernel that deals with OVERLAPPED structures doesn't know whether then handle is synchronous or asynchronous. It just assumes that if you don't pass an OVERLAPPED structure, then the handle is synchronous. And the way it deals with synchronous I/O without an OVERLAPPED is that it creates a temporary OVERLAPPED structure on the stack with a null hEvent, issues an asynchronous I/O with that temporary OVERLAPPED structure, and then waits for completion with Get­Overlapped­Result(bWait = TRUE). It then returns the result.

    What does this mean for you?

    Well, what happens if the hEvent is null?

    If the hEvent member of the OVERLAPPED structure is NULL, the system uses the state of the hFile handle to signal when the operation has been completed.

    Okay, let's step back and look at what's going on here.

    First of all, a file handle is a waitable object: It becomes unsignaled when an I/O operation begins, and it becomes signaled when an I/O operation ends.

    Second of all, this behavior is not useful in general. If you are operating on a synchronous handle, you already know when the I/O operation ends: It ends when the synchronous I/O call returns. And if you are operating on an asynchronous handle, all the hFile tells you is that some I/O completed, but you don't know which one it was. That's why the documentation also says

    Use of file, named pipe, or communications-device handles for this purpose is discouraged. It is safer to use an event object because of the confusion that can occur when multiple simultaneous overlapped operations are performed on the same file, named pipe, or communications device. In this situation, there is no way to know which operation caused the object's state to be signaled.

    What's more, if somebody initiates a new I/O after your asynchronous I/O completed, the file object becomes unsignaled, and there's a possibility that this happened before you got a change to call Wait­For­Single­Object.

    So why have this weird behavior if it's not useful in general? Because it's what the system itself uses internally to implement synchronous I/O! It issues the I/O asynchronously, then waits on the handle. Since the handle is synchronous, the system already knows that there can be only one I/O in progress at a time, so it can just wait on the hFile to know when that I/O is complete.

    Okay, so let's look again at the case of the overlapped I/O issued with no OVERLAPPED structure. The layer that deals with OVERLAPPED structure assumes it has a synchronous handle and issues an asynchronous I/O, then waits until the handle is signaled, under the mistaken belief that the handle will be signaled when that I/O completes (since it "knows" that that's the only outstanding I/O request). But if your handle is actually asynchronous, what happens is that the OVERLAPPED layer waits on the hFile, and the call returns when any I/O on that handle completes. In other words, you're in the "... is discouraged" part of the documentation.

    Theoretically speaking, then, it is legal to pass NULL as the lpOverlapped to Read­File when the handle is asynchronous, but the results may not be what you want, since the call may return prematurely if there is other I/O going on at the same time. And then when the I/O actually completes, it updates the OVERLAPPED structure that was created temporarily on the stack, and we saw that that leads to memory corruption that goes away when you try to debug it.

    There are those who argue that the documentation for Read­File is overly cautious when it outright bans the use of a null lpOverlapped on asynchronous handles, because if you are really careful, you can get it to work, if you can guarantee that no I/O is outstanding on the handle at the time you issue your I/O call, and no other I/O will be issued against the handle while you're waiting for your call to complete.

    I'm of the camp that it's like telling people that it's okay to change gears on your manual transmission by just slamming the gear stick into position without using the clutch. Yes, you can do it if you are really careful and get everything to align just right, but if you mess up, your transmission explodes and spews parts all over the road.

    In the customer's case they were issuing the I/O without an OVERLAPPED structure after the handle was created and before asynchronous operations began, so it was indeed the case that nobody else was using the handle.¹ The usage was therefore technically safe, but the customer nevertheless chose to switch to using an explicit OVERLAPPED structure with an explicit hEvent, just in case future code changes resulted in asynchronous operations being performed on the handle at an earlier point. (Wise choice on the customer's part. Safety first!)

    ¹ We're assuming that there aren't any bugs that result in somebody using a handle after closing it or using an uninitialized handle variable. Even if that assumption isn't true, it would also cause problems even in the case where we we passed an explicit OVERLAPPED structure, so it's no buggier than it was before.

  • The Old New Thing

    Combo boxes have supported incremental searching for quite some time now

    Back in August 2007, I promised to post a program the following day but it appears that I never did. Oops. I discovered this as I went through my "things to blog about" pile and figured better late than never. Though five years late is pretty bad.

    Here's a program which fills a combo box with some strings.

    #include <windows.h>
    #include <windowsx.h>
    static LPCTSTR rgpszList[] = {
        TEXT("Czech Republic"),
        TEXT("United Kingdom"),
    INT_PTR CALLBACK DlgProc(HWND hdlg, UINT uMsg, WPARAM wParam, LPARAM lParam)
        switch (uMsg) {
        case WM_INITDIALOG:
            for (int i = 0; i < ARRAYSIZE(rgpszList); i++) {
                SendDlgItemMessage(hdlg, 100, CB_ADDSTRING, 0,
            return TRUE;
        case WM_COMMAND:
            if (GET_WM_COMMAND_ID(wParam, lParam) == IDCANCEL) {
                EndDialog(hdlg, 0);
        return FALSE;
    int WINAPI WinMain(HINSTANCE hinst, HINSTANCE hinstPrev,
                       LPSTR lpCmdLine, int nShowCmd)
        DialogBox(hinst, MAKEINTRESOURCE(1), NULL, DlgProc);
        return 0;
    // scratch.rc
    #include <windows.h>
    1 DIALOG 50, 50, 185, 98

    Run this program and start typing: "S"-"L"-"O"-"V"-"E"... Hey, look, the combo box is performing incremental search and once you hit the "E", it selected Slovenia, the first item in the list which begins with the letters S-L-O-V-E.

    Wait a few seconds, and try it again. This time, type "S"-"P", and hey look, it selected Spain. You didn't have to go through all those other "S" countries to get to it.

    If you hit F4 to open the combo box and then type "S"-"L"-"O"-"V"-"E", observe that there is a tiny vertical line that tells you where you are in the incremental search string.

    As I noted some time ago, the incremental search resets after four times the double-click time, or two seconds by default.

    Note: The "things to blog about" pile has over 2000 items in it, so there really isn't much need for a Suggestion Box, but I open it up once every few years just for show.

  • The Old New Thing

    Why does Windows Compressed Folders (Zip folders) reject paths that begin with a slash?


    A customer asked, "Does NTFS support files with a null string as the name?"

    No, NTFS does not support files with no name. None of the commonly-used Windows file systems do. Files must have a name. But what a strange question that is. The customer was kind enough to explain why they cared.

    "We have a zip file that the Compressed Folders (Zip folders) feature that comes with Windows cannot deal with. When we try to extract the contents of the zip file, we get the error message 'Windows has blocked access to these files to help protect your computer.' We've attached a copy of the file."

    The Compressed Folders functionality in Explorer has many known limitations, such as lack of support for ZIP64 and AES encryption. Neither of those applied to the zip file in question, however.

    The customer explained what they did. "We created the zip file with a third party zip tool. In particular, after adding a directory tree to the zip file, we renamed the root of the tree to have a blank name. In the zip file we sent you, we added A/file.txt, and then we used the zip tool to rename 'A' to the empty string."

    And indeed if you looked at the zip file in a hex editor, the file name was "/file.txt".

    Now the pieces fell into place. The Compressed Folders code was blocking the file because it was attempting to perform a directory traversal; specifically, it was trying to drop a file in the root directory. The ZIP Application Note says that the "file name" field consists of "The name of the file, with optional relative path." Note that the path must be relative. The next sentence emphasizes this point: "The path stored should not contain a drive or device letter, or leading slash." Therefore, the zip file is invalid, and the Compressed Folders code is within its rights to reject it. (And one wonders why the zip tool allowed the user to create an invalid zip file.)

    It's unclear what the customer was trying to do by renaming "A" to the empty string. So the recommendation back to the customer was "Don't do that."

  • The Old New Thing

    Usage guidance for a popcorn machine in the kitchenette


    My colleague KC Lemson tipped me off to a sign hanging next to a popcorn machine in one of the kitchens:


    Please do not leave
    popcorn kettle on

    while unattended.
    The fire truck will
    , and they don't
    want any popcorn.


    A friend of mine happened to have a chat with a fire fighter who used to be assigned to the fire station nearest to Microsoft main campus. According to him, the top three reasons for being called to a Microsoft building are (in no particular order)

    • Burnt popcorn.
    • Somebody has a panic attack and mistakes it for a heart attack.
    • Somebody pulls the fire alarm because they are under a lot of stress because they missed a deadline, and they think pulling the fire alarm will buy them some time. (Where do they think they are, college?)

    This week is Fire Prevention Week.

  • The Old New Thing

    How does Explorer deal with recent files that were renamed?


    Roni wonders how Explorer manages to keep track of files that were moved or renamed. Specifically, "opening a shortcut to a renamed file actually updates the shortcut's destination and opens the renamed file. How is this done? Is it an NTFS feature?"

    This feature has been around since Windows 95. If the target of a shortcut no longer exists, the shell tries to resolve the shortcut; i.e., find the object, wherever it ended up moving to. As I explained several months before the question was posted, the algorithm used by the shell varies depending on the operating system and the file system and your domain policies. Possibly also the phase of the moon, one can never be sure.

    It's not that Explorer actually keeps track of the files as they move around, just in case you had a shortcut to them. Rather, the shortcut remembers enough information about the file so that if the file moves, Explorer can try looking for it.

    The fact that shortcuts can resolve targets means that shortcuts are a handy tool for keeping track of files that might move around. If you want to keep track of a file, you can just create a shortcut to it (you don't even need to save it in a file), and when it comes time to find the file, you just resolve the shortcut.

  • The Old New Thing

    How do I override the default icon selection algorithm?


    The usual way of loading an icon from a resource is to use the Load­Icon or Load­Image function and letting the window manager's icon selection algorithm decide which image to use based on the images available in the icon, the desired icon size, and the current color depth. A customer wanted to override that algorithm, because the window manager uses the current display color depth to select an image, but they were obtaining the icon for printing purposes, so they wanted to get the highest-color-quality icon rather than the one that matched the screen's color depth. How do you override the default algorithm?

    You basically do the same thing the window manager does. As we saw earlier, icon resources are actually stored in multiple pieces. The thing you use to talk about icons is actually the icon directory, which in turn points to a set of images. The first step, then, is to obtain the icon directory.

    HRSRC hrsrcIcon = FindResource(hResources,
                         MAKEINTRESOURCE(IDI_MY_ICON), RT_GROUP_ICON);
    HGLOBAL hIcon = LoadResource(hResources, hrsrcIcon);
    auto lpIcon = static_cast<GRPICONDIR *>(LockResource(hIcon));

    You then take the images listed in the GRPICONDIR and apply your custom algorithm to decide which one you like best. (If you want to use the default algorithm, you can call Lookup­Icon­Id­From­Directory or Lookup­Icon­Id­From­Directory­Ex. But if you want to use the default algorithm, then just use Load­Image already!)

    When you've found the image you like, take the nId, and that's the resource ID for the RT_ICON.

    HRSRC hrsrcImage = FindResource(hResources,
                         MAKEINTRESOURCE(nId), RT_ICON);
    HGLOBAL hImage = LoadResource(hResources, hrsrcImage);
    auto lpImage = static_cast<PBYTE>(LockResource(hImage));

    You can then convert the icon image data into an icon by using the Create­Icon­From­Resource or Create­Icon­From­Resource­Ex function.

Page 2 of 3 (25 items) 123