• The Old New Thing

    What happened in real-mode Windows when somebody did a longjmp into a discardable segment?


    During the discussion of how real-mode Windows handled return addresses into discarded segments, Gabe wondered, "What happens when somebody does a long­jmp into a discardable segment?"

    I'm going to assume that everybody knows how long­jmp traditionally works so I can go straight to the analysis.

    The reason long­jmp is tricky is that it has to jump to a return address that isn't on the stack. (The return address was captured in the jmp_buf.) If that segment got relocated or discarded, then the jump target is no longer valid. It would have gotten patched to a return thunk if it were on the stack, but since it's in a jmp_buf, the stack walker didn't see it, and the result is a return address that is no longer valid. (There is a similar problem if the data segment or stack segment got relocated. Exercise: Why don't you have to worry about the data segment or stack segment being discarded?)

    Recall that when a segment got discarded, all return addresses which pointed into that segment were replaced with return thunks. I didn't mention it explicitly in the original discussion, but there are three properties of return thunks which will help us here:

    • It is safe to invoke a return thunk even if the associated code segment is in memory. All that happens is that the "ensure the segment is present" step is a nop, and the return thunk simply continues with its work of recovering the original state.
    • It is safe to abandon a return thunk without needing to do any special cleanup. All the state used by the return thunk is stored in the patched stack itself, so if you want to abandon a return thunk, all you need to do is free the stack space.
    • It is safe to reuse a return thunk. Since they are statically allocated, you can use them over and over as long as the associated code segment has not been freed.

    The first property (idempotence of the return thunk) is no accident. It's required behavior in order for return thunks to work at all! After all, if the segment was loaded (say by a direct call or some other return thunk), then the return thunk needs to say, "Well, I guess that was easy," and simply skip the "load the target segment" step. (It still needs to do the rest of the work, of course.)

    The second property (abandonment) is also no accident. An application might decide to exit without returning all the way to Win­Main (the equivalent of calling Exit­Process instead of returning from Win­Main). This would abandon all the stack frames between the exit point and the Win­Main.

    The third property (reuse) is a happy accident. (Well, it was probably designed in for the purpose we're about to put it to right here.)

    Okay, now let's look at the jump buffer again. If you've been following along so far, you may have guessed the solution: Pre-patch the return address as if it had already been discarded. If it turns out that the segment was discarded, then the return thunk will restore it. If the segment is present (either because it was never discarded, or because it was discarded and reloaded, possibly at a new address), the return thunk will figure out where the code is and jump to it.

    Actually, since the state is being recorded in a jmp_buf, the tight space constraints of stack patching do not apply here. If it turns out you need 20 bytes of memory to record this information, then go ahead and make your jmp_buf 20 bytes. You don't have to try to make it all fit inside an existing stack frame.

    The jmp_buf therefore doesn't have to try to play the crazy air-squeezing games that stack patching did. It can record the return thunk, the handles to the data and stack segments, and the return IP without any encoding at all. And in fact, the long­jmp function doesn't need to invoke the return thunk directly. It can just extract the segment number after the initial INT 3Fh and pass that directly to the segment loader.

    (There is a little hitch if the address being returned to is fixed; in that case, there is no return thunk. But that just makes things easier: The lack of a return thunk means that the return address cannot be relocated, so there is no patching needed at all!)

    This magic with return thunks and segment reloading is internal to the operating system, so the core set­jmp and long­jmp functionality was provided by the kernel rather than the C runtime library in a pair of functions called Catch and Throw. The C runtime's set­jmp and long­jmp functions merely forwarded to the kernel versions.

  • The Old New Thing

    New Year's Eve is sometimes a stressful occasion


    Today is New Year's Eve, another opportunity for to mark that an approximately-integral number of revolutions of the earth have occurred since some point in time that wasn't even calculated correctly in the first place. (We retain it for backward compatibility.)

    December 31, 1999 was a particularly anxious day in the technology sector. Microsoft's Director of Y2K Readiness and vice president of Product Support Services described some of the steps that were being taken to prepare for any timekeeping-related issues that would arise as the calendar ticked over to 2000.

    We've analyzed phone capacity, IT systems backup for both data and power, and additional lab environments for enhanced product support capabilities. We have redundant power systems, including back-up generators, at key locations in the unlikely event we lose power.

    I got to see one of those key locations, or at least I saw it from a distance.

    Parked outside Building 26, the home of the Core Operating Systems Division, were semi-trailers filled with computers and back-up generators, ready to spring into action in case of a disaster of biblical proportions and somebody needed to develop a hotfix to Windows on the spot as civilized society collapsed into anarchy all around. (Personally, I would think that if civilization were collapsing, you would have more important things to worry about than patching an operating system.)

    At least to make things more pleasant for the people who had to remain at work that night, Microsoft threw "a New Year's Eve party with a disc jockey, dinner, and a champagne toast at midnight. Our employees can bring their spouses and children, and we'll even host a separate children's party."

    Actually, it sounds like they had more fun than I did that night. I think I stayed at home and watched television.

    It turns out that civilization did not collapse into anarchy. There were minor glitches, but nothing serious. "No problems were seen in Angola, Uganda and Kenya, where the telephone system was said to be functioning as erratically as usual. Italy, one of the worst-prepared countries in the West, also appeared to cross into the new century without any major trouble." Nice to know you can be snarky in a newspaper article.

    (Nitpicker's corner: January 1, 2000 was not the first day of a new century.)

    Depending on how you look at it, Y2K was either an overhyped event created to generate revenue for the technology industry, or was a demonstration of how people can solve a major problem if they just worked hard and coöperated.

    Reality is probably a mix of the two.

    Happy new year, everybody. See you on the other side.

  • The Old New Thing

    Turning off the disco lights on the Start screen


    On the Windows 8 Start screen, you can pan left and right, and if you are using touch, you can whip the entire screen left and right and send all your tiles flying past.

    One of the effects we saw when you panned quickly around the Start screen was something we called disco lights, but which formally goes by the much more mundane name stroboscopic effect.

    When you pan quickly through the Start screen, the items are moving so fast that the human eye can't register any of the fine details. It's just a bunch of colored boxes whizzing past. But video displays are not continuous. They refresh at a particular frequency, so it's as if there's a strobe light flashing on your Start screen.

    Therefore, what you perceive is not a bunch of boxes whizzing past, but rather a bunch of boxes moving around in strange patterns.

    When the panning hits a magic speed, each frame's tiles happen to land in nearly the same spot as the previous frame's tiles, but since they are different tiles, the colors are all different. At that critical speed, you don't get the effect that the tiles are whizzing past. Instead, it looks like they are blinking different colors, like disco lights.

    Photosensitive epilepsy warning: Do not view the disco lights simulation if you have or may have photosensitive epilepsy.

    Otherwise, you can click the colored box to see the disco lights. Music sold separately. I created two versions of the simulation, a more realistic one and one with an exaggerated disco effect to show you what inspired the name.

    The fix was to turn off the colors and just show plain tiles. That way you still felt the motion but you didn't get the disco effect. (It also mean that the Start page was less likely to induce photosensitive epilepsy.)

    This is an interesting example of the sorts of design problems that show up when you start implementing a feature. In the original design for the Start screen, it seemed obvious that the Start screen should show what the user is scrolling through at full speed. But when you actually did it, the hidden stroboscopic effect showed that doing a better job actually looks worse.

    Tomorrow marks the 36th anniversary of the disco dance movie Saturday Night Fever. Sorry we couldn't keep the lights on for you.

  • The Old New Thing

    Why is the Program Files directory called Program Files instead of just Programs?


    Some people suggest that one thing Microsoft Research could do with that time machine they're working on is to go back in time and change the name of the Program Files directory to simply Programs.

    No, it really should be Program Files.

    Program Files are not the same as Programs. Programs are things like Calc, Notepad, Excel, Photoshop. They are things you run.

    Program Files are things like ACRORD32.DLL and TWCUTCHR.DLL. They are files that make programs run.

    If the directory were named Programs, then people who wanted to run a program would start digging into that directory and seeing a page full of weird DLL names and wonder "What the heck kind of programs are these?" And eventually they might figure out that if they want to run PowerPoint, they need to double-click on the icon named POWERPNT. "Computers are so hard to use."


    If you want to find your programs, go to the Start menu. The Program Files directory is like the pantry of a restaurant. You aren't expected to go in there and nibble on things that look interesting. You're expected to order things from the menu.

  • The Old New Thing

    I wrote FAT on an airplane, for heaven's sake


    When you wrote code for 16-bit Windows, one of the things you spent time doing as part of performance tuning was deciding which functions should be grouped together in which segments.

    Code in 16-bit Windows executed out of code segments, each of which could be up to 64KB in size. When a code segment was loaded from disk, the entire segment was loaded, and when it was discarded, the entire segment was discarded. This meant that you could affect your application's performance in significant ways by choosing which functions go in which segments.

    For example, it was to your advantage to keep functions that are called at the same time in the same segment, so that they would be loaded as a unit (saving I/O time). If you chose poorly and put unrelated functions in the same segment, then calling any function in the segment caused all the functions to be loaded, which was a waste of I/O since you loaded a bunch of functions you didn't need.

    Even if the functions were called at the same time, you also had to keep an eye on their popularity. If you have one function that is called frequently and another that is called infrequently (but always preceded by a call to the first function), the less popular function will ride the coattails of his roommate and remain loaded in memory. This creates unnecessary memory pressure that could cause a function which is moderately-frequently-called to be discarded to make room.

    Creating lots of small segments allowed your memory usage to be managed with finer granularity, but it also costs you in general overhead as well as I/O, because each segment load was a new round trip to the disk. You therefore had to balance memory cost against I/O cost.

    The process of optimizing the grouping of functions into segments was known as segment tuning.

    During the development of Windows 3.0, it was customary to have regular meetings with Bill Gates to brief him on the status of the project. At one of the reviews, the topic was performance, and Bill complained, "You guys are spending all this time with your segment tuning tinkering. I could teach a twelve-year-old to segment-tune. I want to see some real optimization, not this segment tuning nonsense. I wrote FAT on an airplane, for heaven's sake."

    (I can't believe I had to write this: This is a dramatization, not a courtroom transcript.)

    This "I wrote FAT on an airplane" line was apparently one Bill used when he wanted to complain that what other people was doing wasn't Real Programming. But this time, the development manager decided she'd had enough.

    "Fine, Bill. We'll set you up with a machine fully enlisted in the Windows source code, and you can help us out with some of your programming magic, why don't you."

    This shut him up.

  • The Old New Thing

    The management of memory for resources in 16-bit Windows, redux


    Some time ago, I briefly ran down how 16-bit Windows managed memory for resources. But there's a detail that I neglected to mention: Ownership.

    As we saw, a resource handle HRSRC was really a pointer to the resource directory entry of the resource from the corresponding module. This could be done with a 16-bit pointer because the segment portion of the pointer could be inferred from the module the resource belonged to. In fact, since modules could be relocated in memory at run time due to compaction, you had better not try to remember the segment portion of the pointer since it could change!

    The Find­Resource function located the resource directory entry. The Load­Resource function allocated memory for the resource and loaded it from disk. The Lock­Resource function locked the memory so you could access it. If two people tried to load the same resource, the memory for the resource was re-used so there was only one copy in memory, and if both people free the resource, the resource is cached in case somebody asks for it again soon.

    Now things get interesting: When does the resource get removed from the cache? What actually controls the lifetime of the resource?

    Answer: The resource lifetime is tied to the module it came from. When the module is unloaded, all its resources are unloaded along with it. (Note that even if a resource is cached, its contents can get discarded if it is tagged as DISCARDABLE.)

    In Win32, modules are directly mapped into memory, and along with it, the resources. Therefore, accessing the resources of a module is a simple matter of figuring out where they got mapped (Find­Resource and friends will tell you), and then reading the memory directly. So despite the radical change to resources work, the basic rules haven't changed: The resources are good as long as the module is still in memory.

    But there are resources and then there are resources. So far, we've been talking about resources in the sense of Find­Resource, which I will call module resources for lack of a better term. But people often work with objects like icons and bitmaps which are not literally resources but which are derived from resources. Next time, we'll look at the relationship between module resources and resource-derived objects in 16-bit Windows.

  • The Old New Thing

    Early versions of Aero Peek: Aladdin, Bat Signal, and Squeegee


    The feature now known as Aero Peek wasn't born that way. It went through several iterations before becoming what eventually shipped in Windows 7.

    At the MIX09 conference, Stephan Hoefnagels showed some of the precursors to Aero Peek. Here are the highlights, and the corresponding time codes if you want to jump straight to the demos.

    Thumbnails in the taskbar (time code 30:20)

    How it worked: Instead of labeled icons, the taskbar showed miniatures of the windows themselves.

    How it got its name: Um, it was just named after what it was.

    Why it failed: When shrunk to taskbar size, most windows looked the same: white with unreadably tiny text.

    Aladdin (time code 42:34)

    How it worked: To bring another window into view temporarily, shake the mouse over that window. The longer you shake, the longer the window stays visible.

    How it got its name: From the folk tale in which the hero summons a genie by rubbing a magic lamp.

    Why it failed: It was too tiresome having to keep shaking the mouse while you read a document.

    Bat Signal (time code 31:10)

    How it worked: When you hover over a taskbar button, the screen darkens, and a spotlight shines on the corresponding window.

    How it got its name: After the distress signal used in the Batman series.

    Why it failed: People liked the way it helped them find their windows, but it was too easy to trigger accidentally. Also, people tried to "ride the light beam", which inadvertently canceled the bat signal.

    Squeegee (time code 33:10)

    How it worked: When you hover over the pop-up thumbnail, the corresponding window comes to the front, and all the other windows turn into glass sheets.

    How it got its name: It makes your screen look like somebody took a squeegee to all the other windows and washed all the pixels off them, leaving clean sheets of glass.

    Squeegee won the day, and it was given the official name Aero Peek.

    Bonus chatter: Although the Wikipedia entry for Batman clocks in at 4.3 Ginsburgs, the entry for Bat-Signal is only 857 milliGinsburgs. Hey, Wikipedians, you're falling down on the job!

  • The Old New Thing

    From Microsoft's mail room to the board room


    In the early 1980's, Microsoft moved its headquarters from downtown Bellevue to Northup Way in Bellevue, a building which later served as a training center (Microsoft University) and is now the North Campus of Bellevue Community College. This is the building in which the infamous Bill Gates Tiger Beat-style photo was taken, and the one I told a story about some years ago.

    At the time, the company was still very small, and the task of managing the relocation was shared among all the employees. After everybody settled in at the new building, it became apparent that the mail volume was barely a trickle. In particular, checks were not coming in, which is kind of an important part of running a business.

    In the excitement of moving, nobody remembered to file a Change of Address form with the Post Office. A visit to the old Bellevue offices revealed a huge mound of mail at the old location.

    A Change of Address was quickly filed, but in the meantime, Steve Ballmer became the company mailman. Every day, he would drive to the old offices, pick up the mail that had accumulated, dump it in the trunk of his car, then deliver it to the Northup offices.

    (Today's story is in recognition of the 238th birthday of the United States Postal Service.)

  • The Old New Thing

    Why is the syntax for touching a file from the command prompt so strange?


    The magic incantation for updating the last-modified date on a file is

    COPY /B FILE+,,

    What strange syntax! What's with the plus sign and the commas, anyway?

    The formal syntax is the much more straightforward

    COPY /B A+B+C+D

    This means to start with the file A, then append the files B, C, and D, treating them all as binary files.

    If you omit the B+C+D part, then you get

    COPY /B A+

    This means "Start with A, then append nothing." The side effect is that the last-write time gets updated, because the command processor opens A for append, writes nothing, then closes the handle.

    That syntax has worked since at least MS-DOS 2.1 (the earliest version I still have a virtual machine for).

    I dont know where the two-comma version came from, but it most likely exploited a parsing glitch in COMMAND.COM, and somehow this variant gained traction and became the version everybody used (even though the other version is two keystrokes shorter). As a result, this weird syntax has become grandfathered as a special-case in the CMD.EXE parser. Here's some actual code from the part of CMD.EXE which parses the arguments to the COPY command:

    if (parse_state == SEEN_TWO_COMMAS)
        copy_mode = TOUCH;
  • The Old New Thing

    The credit card with a half-million-dollar credit limit


    Corporate policies for acquiring hardware typically require going through a bunch of procedures, like issuing purchase orders, getting appropriate approvals, all the usual red tape with the purpose of preventing fraud. But the Windows 95 project was so notoriously behind schedule that upper management removed some procedural roadblocks.

    To expedite the acquisition of hardware for members of the development team, the administrative assistant for the core development team was issued a corporate credit card with a credit limit of $500,000. (You can calculate what $500,000 in 1994 dollars corresponds to in today's money.) I assume the theory here was "buy first, fill out paperwork later."

    "It's kind of weird having a credit card in your pocket that you could buy a house with."

Page 4 of 50 (500 items) «23456»