• The Old New Thing

    I'll see (some of) you in Los Angeles, but this time it'll be October


    (It was September last time.)

    I've been asked to step in as a replacement speaker for the 2008 PDC. The details of the talk are not yet publically available, but what I can say is that the topic will be in the category of Win32 programming.

    Disclaimers: Information was believed to be accurate at press time. Speakers subject to change. Talk may be changed or cancelled for reasons outside my control, or possibly even reasons within my control. This posting is provided as is with no warranties, and confers no rights. Not recommended for children. Some settling may occur during shipping. If symptoms persist, see a doctor.

  • The Old New Thing

    On nearly getting pickpocketed in both Lisbon and Madrid


    My trip to Lisbon introduced me to another tourist phenomenon: pickpockets.

    It was around 10:30 in the morning, and I got on the train to head into town, planning to climb the steps through the Alfama district to visit the castle which looms over the city. The morning rush was over, and the Metro car was nearly empty.

    Just before the doors closed, a group of about four twentysomething guys stumbled onto the train, walking unsteadily and talking quite loudly among themselves. I found this immediately suspicious. They are acting drunk, but who is drunk at 10:30 in the morning? At 10:30, you're hung over, not drunk. And even if you are drunk, you are drunk in the comfort of your home, not stumbling around the subways.

    Even though the subway car had only about three people, and there was plenty of room to spread out, this group of pretend-drunks hung around close to me. I went on high alert.

    A few seconds later, one of the guys "stumbled" into me and thrust his hand into my pants pocket. I immediately grabbed his hand and yanked it back out, making sure he didn't get anything, adding a shout of "Hey!" (I don't speak Portuguese, so I couldn't say anything more eloquent.)

    Still keeping up the ruse of just being a bunch of loud-mouthed drunks, the group of would-be pickpockets stumbled off the train just as the doors closed. Well, three of them did. One of them didn't quite get off in time and stood with his face against the wall until the train reached its next stop, at which point he ran off.

    After Lisbon, I headed over to Madrid, and on the Metro I was on one of the long escalators connecting between two train lines, and I caught the person behind me surreptitously trying to unzip an outside pocket on my shoulder bag. He hadn't made much progress, but just to make sure he didn't get anything, I said hello and shook his hand.

    Bonus chatter: Getting targeted by thieves in Europe seems to be a tradition for me. During the Berlin phase of a previous visit to the continent, a thief tried unsuccessfully to steal the camera out of my hand. Sweden treats me well, though. I don't get targeted by pickpockets; just people trying to recruit me into some sort of organization.

    [Raymond is currently away, possibly being pickpocketed this very moment.]

  • The Old New Thing

    Proto-Microspeak: Efforting


    I have only two citations, so it may not be proper Microspeak.

    We're efforting that for you.

    They're not just trying, they're efforting.

    Solution efforting seems to fall in a gap between teams so there's no clear owner or resourcing focused on it.

    Bonus jargon: resourcing. Actually, that one sentence came from a longer document packed with management-speak. Here's another beauty from that longer document: "A lot of effort goes into availing efficient systems to streamline incident handle time." Of course, what the person really meant to write was "A lot of efforting..."

    It appears that the term has been heard around the NPR offices as well.

    My local social circle has started using the term mockingly. "The Foo object efforts the destruction of the Bar object."

  • The Old New Thing

    The alignment declaration specifier is in bytes, not bits


    Explicit object alignment is not something most people worry about when writing code, which means that when you decide to worry about it, you may be a bit rusty on how the declarations work. (After all, if it's something you worried about all the time, then you wouldn't have trouble remembering how to do it!)

    I was looking at some customer code, and there was a class who had a data member with an explicit alignment declaration.

    class Whatever {
        __declspec(align(32)) LONG m_lSomething; // Must be DWORD-aligned to make writes atomic

    I pointed out that the comment didn't match the code. The comment says that the variable needs to be DWORD-aligned (which in Windows-speak means aligned on a 32-bit boundary), but the code aligns it on a 32-byte boundary, eight times as generous as required. On the other hand, maybe they really did want the member aligned on a 32-byte boundary (say to put it on its own cache line).

    Turns out that in this case, the comment was correct and the code was wrong. To force a variable to align on a DWORD boundary, you want to say __declspec(align(4)). Save yourself a bunch of unnecessary padding bytes.

    But in fact, in this case, the customer was simply trying too hard. The code was compiled with default alignment, which aligns integer types on their natural boundaries anyway. The compiler was going to align the variable even if you didn't specify anything.

    [Raymond is currently away; this message was pre-recorded.]

  • The Old New Thing

    What happened in real-mode Windows when somebody did a longjmp into a discardable segment?


    During the discussion of how real-mode Windows handled return addresses into discarded segments, Gabe wondered, "What happens when somebody does a long­jmp into a discardable segment?"

    I'm going to assume that everybody knows how long­jmp traditionally works so I can go straight to the analysis.

    The reason long­jmp is tricky is that it has to jump to a return address that isn't on the stack. (The return address was captured in the jmp_buf.) If that segment got relocated or discarded, then the jump target is no longer valid. It would have gotten patched to a return thunk if it were on the stack, but since it's in a jmp_buf, the stack walker didn't see it, and the result is a return address that is no longer valid. (There is a similar problem if the data segment or stack segment got relocated. Exercise: Why don't you have to worry about the data segment or stack segment being discarded?)

    Recall that when a segment got discarded, all return addresses which pointed into that segment were replaced with return thunks. I didn't mention it explicitly in the original discussion, but there are three properties of return thunks which will help us here:

    • It is safe to invoke a return thunk even if the associated code segment is in memory. All that happens is that the "ensure the segment is present" step is a nop, and the return thunk simply continues with its work of recovering the original state.
    • It is safe to abandon a return thunk without needing to do any special cleanup. All the state used by the return thunk is stored in the patched stack itself, so if you want to abandon a return thunk, all you need to do is free the stack space.
    • It is safe to reuse a return thunk. Since they are statically allocated, you can use them over and over as long as the associated code segment has not been freed.

    The first property (idempotence of the return thunk) is no accident. It's required behavior in order for return thunks to work at all! After all, if the segment was loaded (say by a direct call or some other return thunk), then the return thunk needs to say, "Well, I guess that was easy," and simply skip the "load the target segment" step. (It still needs to do the rest of the work, of course.)

    The second property (abandonment) is also no accident. An application might decide to exit without returning all the way to Win­Main (the equivalent of calling Exit­Process instead of returning from Win­Main). This would abandon all the stack frames between the exit point and the Win­Main.

    The third property (reuse) is a happy accident. (Well, it was probably designed in for the purpose we're about to put it to right here.)

    Okay, now let's look at the jump buffer again. If you've been following along so far, you may have guessed the solution: Pre-patch the return address as if it had already been discarded. If it turns out that the segment was discarded, then the return thunk will restore it. If the segment is present (either because it was never discarded, or because it was discarded and reloaded, possibly at a new address), the return thunk will figure out where the code is and jump to it.

    Actually, since the state is being recorded in a jmp_buf, the tight space constraints of stack patching do not apply here. If it turns out you need 20 bytes of memory to record this information, then go ahead and make your jmp_buf 20 bytes. You don't have to try to make it all fit inside an existing stack frame.

    The jmp_buf therefore doesn't have to try to play the crazy air-squeezing games that stack patching did. It can record the return thunk, the handles to the data and stack segments, and the return IP without any encoding at all. And in fact, the long­jmp function doesn't need to invoke the return thunk directly. It can just extract the segment number after the initial INT 3Fh and pass that directly to the segment loader.

    (There is a little hitch if the address being returned to is fixed; in that case, there is no return thunk. But that just makes things easier: The lack of a return thunk means that the return address cannot be relocated, so there is no patching needed at all!)

    This magic with return thunks and segment reloading is internal to the operating system, so the core set­jmp and long­jmp functionality was provided by the kernel rather than the C runtime library in a pair of functions called Catch and Throw. The C runtime's set­jmp and long­jmp functions merely forwarded to the kernel versions.

  • The Old New Thing

    The sign that a trend is over: It shows up in a movie


    George Mannes talks with Mike Pesca on his observation that once a financial trend shows up in a movie or television show, it's over.

  • The Old New Thing

    Understanding the classical model for linking: Sometimes you don't want a symbol to come along for a ride

    Continuing our study of the classical model for linking, let's take another look at the trick of taking symbols along for the ride.

    The technique of taking symbols along for the ride is quite handy if that's what you want, but sometimes you don't actually want it. For example, a symbol taken along for the ride may create conflicts or create unwanted dependencies.

    Here's an example: Suppose you have a library called stuff.lib where you put functions that are used by various modules in different projects. One of the files in your library might look like this:

    // filedatestuff.cpp
    BOOL GetFileCreationTimeW(
            LPCWSTR pszFile,
            FILETIME *pft)
        BOOL fSuccess = GetFileAttributesExW(pszFile,
        if (fSuccess) {
            *pft = wfad.ftCreationTime;
        } else {
            pft->dwLowDateTime = 0;
            pft->dwHighDateTime = 0;
        return fSuccess;
    BOOL GetFileCreationTimeAsStringW(
             LPCWSTR pszFile,
             LPWSTR pszBuf,
             UINT cchBuf)
        FILETIME ft;
        BOOL fSuccess = GetFileCreationTimeW(pszFile, &ft);
        if (fSuccess) {
            fSuccess = SHFormatDateTimeW(&ft, NULL,
                                         pszBuf, cchBuf) > 0;
        return fSuccess;

    Things are working out great, people like the helper functions in your library, and then you get a bug report:

    When my program calls the Get­File­Creation­TimeW function, I get a linker error: unresolved external: __imp__SHFormat­Date­TimeW. If I remove my call to Get­File­Creation­TimeW, then my program builds fine.

    You scratch your head. "The program is calling Get­File­Creation­TimeW, but that function doesn't call SHFormat­Date­TimeW, so why are we getting an unresolved external error? Any why hasn't anybody else run into this problem before?"

    First question first. Why are we getting an unresolved external error for a nonexistent external dependency?

    Because the Get­File­Creation­Time­As­StringW function got taken along for the ride. When the customer's program called Get­File­Creation­TimeW, that pulled in the filedatestuff.obj file, and that OBJ file contains both Get­File­Creation­TimeW and Get­File­Creation­Time­As­StringW. Since they are in the same OBJ file, pulling in one function pulls in all of them.

    The fix is to split the filedatastuff.cpp file into two files, one for each function. That way, when you pull in one function, nobody else comes along for the ride.

    Now to the second half of the question: Why did nobody run into this problem before?

    The Get­File­Creation­TimeW function has a dependency on Get­File­Attributes­ExW, which is a function in KERNEL32.DLL. On the other hand, the Get­File­Creation­Time­As­StringW function has a dependency on SHFormat­Date­TimeW, which is a function in SHLWAPI.DLL. If somebody lists KERNEL32.LIB as a dependent library in their project, but they don't include SHLWAPI.LIB on that list, then they will encounter this problem because the linker will pull in the reference to SHFormat­Date­TimeW and have no way of resolving it.

    Nobody ran into this before because SHLWAPI.LIB has lots of cute little functions in it, so most people include it in their project. Only if somebody is being frugal and leaving SHLWAPI.LIB out of their project will they run into this problem.

    Bonus chatter: The suggestion to split the file into two will work, but if you are really clever, you can still do some consolidation. Instead of splitting up files by functional group (for example, "all FILETIME functions"), you need to split them up based on their dependencies ("functions that are dependent solely on SHLWAPI.LIB"). Of course, this type of organization may make the code harder to follow ("Why did you put Get­File­Creation­Time­As­StringW and Hash­String in the same file?"), so you have to balance this against maintainability and readability. For example, somebody who is not aware of the classical model for linking may add a function to the file that has a dependency on SHELL32.DLL, and now your careful separation has fallen apart.

  • The Old New Thing

    Bug Bash: It's funny because it's true


    Hans Bjordahl's Bug Bash is one of those it's funny because it's true comics about life inside a software company. Specifically, Microsoft, where Hans worked for many years and whose internal newsletter featured Hans's strip. (Some might argue that Bug Bash was the only part of the company newsletter worth reading.)

    Today I'm going to share two of my favorites.

    • Internal marketing campaigns. The year 2004 was a particularly good one (or in the opinion of some people: a particularly bad one) for internal marketing campaigns at Microsoft, with about forty pieces of internal marketing delivered to your maildrop during the year. That's a piece of internal junk mail nearly every week! Perhaps in response, the year 2005 was unusually sparse. But internal marketing appears to be on the rise again, though fortunately not yet to the heights of 2004.
    • Blog all you want. The subtitle on the fake newspaper headline wins it.

    Bonus discovery, Hans revealed himself as the co-founder of Mr. Cranky. Man, I loved that site.

  • The Old New Thing

    How do I launch a file as if it were a text file, even though its extension is not .txt?


    You might have a program that generates log files or other text content with an extension other than .txt. You naturally might want to open these documents in the user's default text editor.

    You might decide to ask the Windows developer support team, "How can I figure out what program is the handler for text files?" The idea being that once you get this program name, you can then run it yourself, with the document on the command line. And you would also be running into the trap of looking for the answer to a question rather than a solution to a problem.

    For one thing, the default handler for the file type might require special command line parameters, parameters which you won't get if you merely get the executable path. For example, on Windows 7, the default command line for JPG files is %SystemRoot%\System32\rundll32.exe "%ProgramFiles%\Windows Photo Viewer\PhotoViewer.dll", ImageView_Fullscreen %1, and if you merely asked for the executable, all you would get back would be rundll32.exe, and trying to execute rundll32.exe Boats.jpg doesn't get you very far. You lost all the command line arguments.

    For another thing, the default handler for the file type might not even be a command line. It might be an old program that uses DDE. Or the handler might be a drop target. Or it could be an IContext­Menu or an IExecute­Command. In these cases, there is no command line in the first place, so asking for the command line template is meaningless.

    But we saw the answer to this question before, just in a different guise. The lpClass member of the SHELL­EXECUTE­INFO lets you open a file as if it were another type of file. In that article, somebody was passing a class when they didn't mean to; here, we're passing it on purpose.

  • The Old New Thing

    News flash: Universities are more likely to admit students whose parents gave lots of money


    Earlier, we saw that alumni give money to universities in order to increase the likelihood that the university will admit their children. Today, we learn that the tactic works. The children of big-donor alumni are more likely to be accepted. In fact, you don't even have to be the child of alumni. Just make sure you parents give lots of money.

Page 370 of 425 (4,250 items) «368369370371372»