March, 2013

  • The Old New Thing

    "Adjust visual effects for best performance" should really be called "Adjust visual effects for crappiest appearance"


    In the Performance Options control panel, on the tab labeled Visual Effects, there is a radio button called Adjust for best performance. If you select it, then all the visual effects are disabled.

    But the name of that radio button has been wrong for a long time. It doesn't actually adjust your visual effects for best performance. It just adjusts them for crappiest appearance.

    Starting in Windows Vista, a lot of visual effects were offloaded to the graphics card. Consequently, the impact on system performance for those visual effects is negligible, and sometimes turning off the effect actually makes your system run slower because you disabled hardware acceleration, forcing operations to be performed in software.

    For example, if desktop composition is enabled, then a backup copy of a window's entire contents is kept in video memory, even if the window is covered by other windows. Without desktop composition, the window manager uses the classic model which follows the principle don't save anything you can recalculate: The contents of an occluded window are not saved anywhere, and when the window becomes exposed, the window receives a WM_PAINT message to tell it to regenerate its contents.

    This means that, for example, when you remove a window from the screen and expose the window underneath, the desktop compositor can show the contents of the underlying window immediately because it saved a copy of the window in video memory and has been keeping it up to date. On the other hand, if you disable desktop composition, you will just stare at a blank window underneath, and then you have to sit and wait for that window to repaint itself.

    Congratulations: By disabling desktop composition, you made the act of uncovering a window run slower. (You will see the same effect when switching between maximized windows.)

    Okay, so if unchecking these visual effects has negligible and sometimes even a negative effect on performance, why do we still present them in the Visual Effects control panel for people to disable?

    Because enthusiasts who think they are so awesomely clever looooove disabling anything that makes the computer look pretty, because they're convinced that effort making the computer look pretty is clearly done at the expense of performance. They observe that if all the visual effects are disabled, their machine runs fast, but that's not a controlled experiment because they failed to measure how fast the computer runs when the effects are enabled. (By similar logic, sticking a banana in your ear keeps the alligators away.)

    These are the people who think that a bare computer motherboard on a table with components hanging off it runs faster than a computer packed into an attractive case. And even if you demonstrate that it doesn't actually run faster, they will still keep their computer in its partially-disassembled state because it adds to their street cred.

    The Visual Effects settings page turned into a sort of unintended psychology experiment.

  • The Old New Thing

    The source of much confusion: "backed by the system paging file"


    Perhaps one of the most misunderstood sentences in the Win32 documentation is this little bit in the documentation for Create­File­Mapping:

    If hFile is INVALID_HANDLE_VALUE, the calling process must also specify a size for the file mapping object in the dwMaximum­Size­High and dwMaximum­Size­Low parameters. In this scenario, Create­File­Mapping creates a file mapping object of a specified size that is backed by the system paging file instead of by a file in the file system.

    When people read the underlined portion, they interpret this to mean "The data in the file mapping object will be written to the system paging file." But that's not what it says. It says that it is backed by the system paging file. In other words, "If I need to page this memory out, I will store it in the system paging file."

    Note the word "if".

    Usually, people get all worked up about the description because "I don't want this data to be written to disk by the creator, and then read from the disk by the consumer. I want this to be stored in RAM, just like the memory I allocate with Heap­Allocate or Virtual­Alloc." Of course, what they didn't realize is that memory allocated with Heap­Allocate and Virtual­Alloc is also backed by the system paging file. If memory allocated by Heap­Allocate and Virtual­Alloc needs to be paged out, the memory manager will write it to the paging file.

    In other words, "backed by the system paging file" just means "handled like regular virtual memory."

    If the memory is freed before it ever gets paged out, then it will never get written to the system paging file. Just like you wanted.

    The documentation was written with kernel-colored glasses. They figured that you knew that paging file-backed memory was just a way of saying "normal pageable memory."

    Exercise: What happens if paging is disabled? Where is the memory backed if there is no paging file?

  • The Old New Thing

    There's no law that says two people can't have the same thing to eat


    Some time ago, my group went out for a team lunch. It was to a restaurant we were not familiar with, so there was quite a bit of time studying the menu. As everybody looked over the menu, discussion naturally turned to "So what are you going to have?"

    "I think I'll have the salmon sandwich."

    One of my colleagues replied, "Oh, rats. I was thinking of having that."

    I remarked, "There's no law that says two people can't order the same thing."

    My colleague disagreed.

    Not if you ask my wife. Whenever we go out to eat, she'll ask me what I'm having, and then she'll say "Oh, rats. I was thinking of having that. Now I'll have to order something else."

    I'll say, "You can order it too, that's okay. Or I'll change my order, no big deal."

    But she'll say, "No, that's okay. I'll just find something else."

    I've tried many times without success to convince her that it's okay for two people to have the same thing to eat. Now I just accept it.

    Update: A few months later, I received an update from my colleague.

    The other night, my wife and I went out to dinner, and my wife really wanted the same that I had already said that I was going to order. But instead of switching to something else, she ordered it anyway. I think this is the first time this has ever happened. And you know what? The world did not end.
  • The Old New Thing

    Microspeak: Science project


    A science project is a feature that is really cool and challenging from a technological standpoint but is way overkill for the end-user scenario at hand.

    Back in the late 1990's, a bunch of us cooked up this idea for a networked screen saver that ran at night after most people had gone home from work. You told it the physical location and compass orientation of everybody's monitor. The networked screen saver created a virtual red bouncing ball that traveled around the building in three dimensions. The screen saver showed a viewport into the three-dimensional virtual world that contained the bouncing ball.

    This is a clear example of a science project: Nobody's going to sit there and type the positions and orientations of every computer monitor in the building. Even if we wrote this screen saver, nobody would actually use it. Most of the enjoyment is in actually writing the screen saver than in actually running it.

    One type of science project has high set-up costs for low benefit, like our bouncing ball screen saver.

    Another type of science project requires hardware that very few people have right now. For example, "If you have a tablet connected to at least two touch-enabled external monitors, then you can..."

    A third type of science project is simply trying to solve a problem that nobody really considers to be a problem. You're doing it just for the Gee Whiz factor. For example, "If you have a pair of Bluetooth headphones, and you walk back into range of your computer, the computer can automatically unpause your music." Yeah, I guess you could do that, but it also means that while you are away from your computer, you're walking around looking like an idiot because you're wearing headphones.

    Now, there may be an actual useful feature hiding inside a science project, but until you find that feature and bring it to the surface, what you basically have is a science project.

  • The Old New Thing

    What are the conventions for managing standard handles?


    Consider this function:

    void ChangeConsoleColor(WORD wColor)
     HANDLE h = GetStdHandle(STD_OUTPUT_HANDLE);
     if (h != INVALID_HANDLE_VALUE) {
      SetConsoleTextAttribute(h, wColor);

    "When I call this function, it works the first time, but when I call it a second time, Get­Std­Handle returns a handle numerically identical to the one returned by the first call, but the handle is now invalid, presumably because I closed it. I closed it because I was taught to clean up after myself. Is this a case where I shouldn't?"

    Yes, you should clean up after yourself, but you should also have been taught to be respectful of community property. In this case, you walked into the TV room of your dormitory, watched an episode of Friends, and then smashed the television with a baseball bat. Later, you came back to the room to watch another episode of Friends and said, "Hey, what happened to our television?" (You can tell I'm old because I'm talking about the TV room of a dormitory.)

    The standard handle values are sort of like a global variable for your process. Anybody can call Get­Std­Handle to read the variable, and anybody can call Set­Std­Handle to set it. But as with any other global handle variable, you need to observe certain rules to ensure that the value is always valid.

    Suppose you had a global variable called HANDLE hSomeFile. What invariants would you want to apply?

    • If the value is INVALID_HANDLE_VALUE, then there is no active file. (You might also have decided to use NULL as your special value, but INVALID_HANDLE_VALUE works better here because that is the conventional sentinel value for file handles.)
    • If the value is not the special value above, then it refers to a valid file handle.

    That second invariant above already establishes a rule:

    • If you close the handle held in the global variable, you must also set the global variable to a new valid value.

    As I noted some time ago, programming is a game of stepping-stone from one island of consistency to another. You start with a consistent system, you perturb it (temporarily violating consistency), and then you re-establish consistency. Closing the handle makes the value invalid, so you need to follow up by making the value valid again. Otherwise you left your system in an inconsistent state.

    Okay, now instead of talking about that global variable hSomeFile, let's talk about the global handle hidden behind Get­Std­Handle and Set­Std­Handle. Congratulations, we just established the rules for managing standard handles.

    • If Get­Std­Handle returns INVALID_HANDLE_VALUE, then there is no active file.
    • If the value is not the special value above, then it refers to a valid file handle. (Note that file handles can refer to things that aren't files. In our case, it often will refer to a console.)
    • If you call Close­Handle on a standard handle, then you must also call Set­Std­Handle to set a new value for the standard handle.

    Note that these rules are just conventions. If you want to violate them by, say, closing the handle and then leaving a garbage handle in the hidden global variable for the next guy to trip over, then that's your problem. For example, you might choose to violate the rules temporarily, and then fix things up before anybody notices.

  • The Old New Thing

    Why do Explorer and the command prompt interpret file times differently?


    A customer observed that if they use Explorer to view the timestamp on a file, it is not always in agreement with the value shown if they run a plain DIR in a command prompt. They are sometimes off by an hour. Why is that?

    Whenever you hear the phrase "off by an hour" you should immediately think "Daylight Saving Time".

    The formatting of file timestamps shown by Explorer has changed over time. The most recent algorithm (at the time of this writing) is to use the time zone that was in effect at your current location at the time the timestamp was created. For example, a file created at noon in June 22 will show its timestamp as noon, even if you view it in the middle of December. That's because Explorer says, "Well, on June 22, Daylight Saving Time was not in effect, even though it's in effect now, so I will interpret that time zone as if Daylight Saving Time were not active." (Hey, Raymond, didn't you get that backward? Answer: The customer who asked this question is in New Zealand.)¹

    The old-style function for converting a UTC timestamp into a local timestmap is File­Time­To­Local­File­Time. The documentation for that function points you at the sequence of operations you need to perform if you want to use the time zone of the timestamp instead of the current time zone.

    Explorer switched to using the time zone of the timestamp, but the command processor continues using the old-style conversion.

    Why doesn't the command processor get with the program?

    Well, for one thing, the people who decide what Explorer does are not the same people who decide what the command processor does. (The Explorer folks can certainly make suggestions, but they can't force the command processor to do anything.) It's like asking why Taco Bell puts the men's room on the left, but Pizza Hut puts it on the right.

    The command processor is an old and cranky program. The command processor yells at Explorer to get off his lawn. The command processor gets upset when his Internet connection flakes out while he's watching Matlock online. The command processor doesn't care about your fancy-pants localized file names; it shows the raw file system names. The command processor has hundreds of thousands of scripts, and there's no way of knowing how many of them depend on the exact way it formats dates.

    You may be able to wheedle the command processor into making some changes for you, but you'd better have a really good reason, and he's going to be really grumpy about it. The command processor was once cajoled into changing its date format to four-digit years back in the late 20th century, and he did it only because everybody insisted that it was soooooooo important. But he was so grumpy about it, he had an option to go back.

    ¹ Actually, that's not true. The customer who asked the question was in Texas, but I moved him to New Zealand for the purpose of the story. People in the Southern Hemisphere always have to put up with us Northerners assuming that summer starts in June, so I figured I'd turn the tables for once.

  • The Old New Thing

    Redistributing computers among offices for heating purposes

    Some time ago, I joked about the people who rearrange computers in their house during the winter in order to use them as space heaters.

    Turns out this happens a lot at Microsoft. One of my friends said that one of his coworkers used a small heater in her office to keep warm. On the other hand, his office always ran warm because of all the computers in it. They hit upon a simple solution to both problems: "Now she's using a 12 core/24 thread space heater that's a lot quieter than her old one."

    At one point in time, I had a large number of computers in my office, including an Itanium prototype. (You knew it was a prototype because it contained Engineering Styrofoam.) The thing generated a lot of heat. My friend across the hall, on the other hand, had a cold office. Solution: With some help from colleagues, we moved the Itanium across the hall. Two problems solved.

  • The Old New Thing

    Does this operation work when file system redirection is disabled? The default answer is NO


    A customer reported that when their program called SH­Get­File­Info to get the icon for a folder, the call failed. "It works on some machines but not others. We don't know what the difference is between the working and non-working machines." They included the offending function from their program, but everything in the function looked good. The problem was something outside the function itself.

    Eventually, the customer confessed that they had called the Wow64­Disable­Wow64­Fs­Redi­rection function to disable file system redirection, and the call to SH­Get­File­Info took place while redirection was disabled. "We found that if we re-enable file system redirection before calling SH­Get­File­Info, then everything works properly."

    That's right, because, like impersonation, nothing works when file system redirection is disabled unless it is specifically documented as supporting disabled redirection. This is even called out in the documentation for Wow64­Disable­Wow64­Fs­Redi­rection:

    Note  The Wow64­Disable­Wow64­Fs­Redi­rection function affects all file operations performed by the current thread, which can have unintended consequences if file system redirection is disabled for any length of time. For example, DLL loading depends on file system redirection, so disabling file system redirection will cause DLL loading to fail. Also, many feature implementations use delayed loading and will fail while redirection is disabled. The failure state of the initial delay-load operation is persisted, so any subsequent use of the delay-load function will fail even after file system redirection is re-enabled. To avoid these problems, disable file system redirection immediately before calls to specific file I/O functions (such as Create­File) that must not be redirected, and re-enable file system redirection immediately afterward using Wow64­Revert­Wow64­Fs­Redi­rection.

    Whenever you use one of these "global solutions to a local problem" types of solutions that change some fundamental behavior of the system, you have to make sure that everybody is on board with your decision.

    The local solution would be to use the C:\Windows\Sys­Native virtual directory for files you want to look up in the native system directory rather than the emulated system directory.

  • The Old New Thing

    The x86 architecture is the weirdo: Structured exception handling


    If your reference architecture is x86, then you will think that everything it does is normal and the rest of the world is weird. Except it's the other way around: The x86 architecture is the weirdo.

    I was reminded of this when commenter 640k complained, on the subject of what I think is table-based structured exception handling, "It would be interesting to know why this 'invention' was introduced in 64-bit Windows when no other version of Windows requires it." (The original text was "when no other OS requires it", but I'm assuming that this was in the context of Windows-based OS, since unix doesn't have structured exception handling in the first place.)

    This has a very narrow definition of "no other OS", because it really means "No other non-x86-based version of Windows." In this world, the color of the sky is x86.

    In fact, x86 is the only architecture for which Windows uses stack-based exception chaining. All other architectures use table-based exception unwinding. The prologue and epilogue of each function must follow a particular format so that the actions performed therein can be unwound during exception handling. At the very introduction of Win32, it was only the x86 which used stack-based unwinding. The Alpha AXP, MIPS, and PowerPC all used used table-based exception unwinding. And as new architectures were added by Windows, they all used table-based exception unwinding as well. Itanium? Table-based. Alpha AXP 64-bit? Table-based. ARM? Table-based.

    The use of table-based exception handling was not "introduced" with x64. It was introduced back in 1992, and has in fact been the exception unwinding mechanism for all architectures.

    Well, almost all. Not the x86, because the x86 is the weirdo.

  • The Old New Thing

    The C language specification describes an abstract computer, not a real one


    If a null pointer is zero, how do you access the memory whose address is zero? And if C allows you to take the address one past the end of an array, how do you make an array that ends at 0xFFFFFFFF, since adding one to that value would wrap around?

    First of all, who says that there is a byte zero? Or a byte 0xFFFFFFFF?

    The C language does not describe an actual computer. It describes a theoretical one. On this theoretical computer, it must be possible to do certain things, like generate the address of one item past the end of an array, and that address must compare greater than the address of any member of the array.

    But how the C language implementation chooses to map these theoretical operations to actual operations is at the discretion of the C language implementation.

    Now, most implementations will do the "obvious" thing and say, "Well, a pointer is represented as a numerical value which is equal to the low-level memory address." But they are not required to do so. For example, you might have an implementation that says, "You know what? I'm just going to mess with you, and every pointer is represented as a numerical value which is equal to the low-level memory address minus 4194304. In other words, if you try to dereference a pointer whose numeric value is 4096, you actually access the memory at 4194304 + 4096 = 4198400. On such a system, you could have an array that goes all the way to 0xFFFFFFFF, because the numeric value of the pointer to that address is 0xFFBFFFFF, and the pointer to one past the end of the array is therefore a perfectly happy 0xFFC00000.

    Before you scoff and say "That's a stupid example because nobody would actually do that," think again. Win32s did exactly this. (The 4194304-byte offset was done in hardware by manipulating the base address of the flat selectors.) This technique was important because byte 0 was the start of the MS-DOS interrupt table, and corrupting that memory was a sure way to mess up your system pretty bad. By shifting all the pointers, it meant that a Win32s program which dereferenced a null pointer ended up accessing byte 4194304 rather than byte 0, and Win32s made sure that there was no memory mapped there, so that the program took an access violation rather than corrupting your system.

    But let's set aside implementations which play games with pointer representations and limit ourselves to implementations which map pointers to memory addresses directly.

    "A 32-bit processor allegedly can access up to 2³² memory locations. But if zero and 0xFFFFFFFF can't be used, then shouldn't we say that a 32-bit processor can access only 2³² − 2 memory locations? Is everybody getting ripped off by two bytes? (And if so, then who is pocketing all those lost bytes?)"

    A 32-bit processor can address 2³² memory locations. There are no "off-limits" addresses from the processor's point of view. The guy that made addresses zero and 0xFFFFFFFF off-limits was the C language specification, not the processor. That a language fails to expose the full capabilities of the underlying processor shouldn't be a surprise. For example, you probably would have difficulty accessing the byte at 0xFFFFFFFF from JavaScript.

    There is no rule in the C language specification that the language must permit you to access any byte of memory in the computer. Implementations typically leave certain portions of the address space intentionally unused so that they have wiggle room to do the things the C language specification requires them to do. For example, the implementation can arrange never to allocate an object at address zero, so that it can conform to the requirement that the address of an object never compares equal to the null pointer. It also can arrange never to allocate an object that goes all the way to 0xFFFFFFFF, so that it can safely generate a pointer one past the end of the object which behaves as required with respect to comparison.

    So you're not getting ripped off. Those bytes are still addressable in general. But you cannot get to them in C without leaving the C abstract machine.

    A related assertion turns this argument around. "It is impossible to write a conforming C compiler for MS-DOS because the C language demands that the address of a valid object cannot be zero, but in MS-DOS, the interrupt table has address zero."

    There is a step missing from this logical argument: It assumes that the interrupt table is a C object. But there is no requirement that the C language provide access to the interrupt table. (Indeed, there is no mention of the interrupt table anywhere in the C language specification.) All a conforming implementation needs to do is say, "The interrupt table is not part of the standard-conforming portion of this implementation."

    "Aha, so you admit that a conforming implementation cannot provide access to the interrupt table."

    Well, certainly a conforming implementation can provide language extensions which permit access to the interrupt table. It may even decide that dereferencing a null pointer grants you access to the interrupt table. This is permitted because dereferencing a null pointer invokes undefined behavior, and one legal interpretation of undefined behavior is "grants access to the interrupt table."

Page 1 of 3 (30 items) 123