The Old New Thing

The path-searching algorithm is not a backtracking algorithm

Raymond Chen - MSFT — Wed, 08 Feb 2012 15:00:00 GMT

Suppose your PATH environment variable looks like this:

C:\dir1;\\server\share;C:\dir2

Suppose that you call LoadLibrary("foo.dll") intending to load the library at C:\dir2\foo.dll. If the network server is down, the LoadLibrary call will fail. Why doesn't it just skip the bad directory in the PATH and continue searching?

Suppose the LoadLibrary function skipped the bad network directory and kept searching. Suppose that the code which called LoadLibrary("foo.dll") was really after the file \\server\share\foo.dll. By taking the server down, you have tricked the LoadLibrary function into loading c:\dir2\foo.dll instead. (And maybe that was your DLL planting attack: If you can convince the system to reject all the versions on the PATH by some means, you can then get LoadLibrary to look in the current directory, which is where you put your attack version of foo.dll.)

This can manifest itself in very strange ways if the two copies of foo.dll are not identical, because the program is now running with a version of foo.dll it was not designed to use. "My program works okay during the day, but it starts returning bad data when I try to run between midnight and 3am." Reason: The server is taken down for maintenance every night, so the program ends up running with the version in c:\dir2\foo.dll, which happens to be an incompatible version of the file.

When the LoadLibrary function is unable to contact \\server\share\foo.dll, it doesn't know whether it's in the "don't worry, I wasn't expecting the file to be there anyway" case or in the "I was hoping to get that version of the file, don't substitute any bogus ones" case. So it plays it safe and assumes it's in the "don't substitute any bogus ones" and fails the call. The program can then perform whatever recovery it deems appropriate when it cannot load its precious foo.dll file.

Now consider the case where there is also a c:\dir1\foo.dll file, but it's corrupted. If you do a LoadLibrary("foo.dll"), the call will fail with the error ERROR_BAD_EXE_FORMAT because it found the C:\dir1\foo.dll file, determined that it was corrupted, and gave up. It doesn't continue searching the path for a better version. The path-searching algorithm is not a backtracking algorithm. Once a file is found, the algorithm commits to trying to load that file (a "cut" in logic programming parlance), and if it fails, it doesn't backtrack and return to a previous state to try something else.

Discussion: Why does the LoadLibrary search algorithm continue if an invalid directory or drive letter is put on the PATH?

Vaguely related chatter: No backtracking, Part One

Microspeak: fit

Raymond Chen - MSFT — Tue, 07 Feb 2012 15:00:00 GMT

In Microspeak, fit is a predicate noun which is never used on its own but always comes with a modifying adjective. For something to be a good fit is for something to be appropriate or suitable for a particular situation. The opposite of a good fit is not a bad fit, because that's pejorative. Rather, something that is not a good fit is referred to as a poor fit.

The purpose of a previewer plug-in is to allow users to view the media without opening it. An image editing tool would not be a good fit for the previewing feature. (Alternatively, "would be a poor fit for the previewing feature.")

To be a good fit with a particular group is to mesh well with that group's existing practices and conventions.

The Datacenter Edition of the product is a poor fit for most small businesses.

The group in question need not consist of people.

The results are obtained incrementally, which makes it a good fit for IQueryable<T> and LINQ.

Microsoft Human Resources loves to apply the concept of "fit" to people fitting into a job position.

The story of the mysterious WINA20.386 file

Raymond Chen - MSFT — Mon, 06 Feb 2012 15:00:00 GMT

matushorvath was curious about the WINA20.386 file that came with some versions of MS-DOS.

The WINA20.386 file predates my involvement, but I was able to find some information on the Internet that explained what it was for. And it's right there in KB article Q68655: Windows 3.0 Enhanced Mode Requires WINA20.386:

Windows 3.0 Enhanced Mode Requires WINA20.386

Windows 3.0 enhanced mode uses a modular architecture based on what are called virtual device drivers, or VxDs. VxDs allow pieces of Windows to be replaced to add additional functionality. WINA20.386 is such a VxD. (VxDs could be called "structured" patches for Windows.)

Windows 3.0 enhanced mode considers the state of the A20 line to be the same in all MS-DOS virtual machines (VMs). When MS-DOS is loaded in the high memory area (HMA), this can cause the machine to stop responding (hang) because of MS-DOS controlling the A20 line. If one VM is running inside the MS-DOS kernel (in the HMA) and Windows task switches to another VM in which MS-DOS turns off A20, the machine hangs when switching back to the VM that is currently attempting to execute code in the HMA.

WINA20.386 changes the way Windows 3.0 enhanced mode handles the A20 line so that Windows treats the A20 status as local to each VM, instead of global to all VMs. This corrects the problem.

(At the time I wrote this, a certain popular Web search engine kicks up as the top hit for the exact phrase "Windows 3.0 Enhanced Mode Requires WINA20.386" a spam site that copies KB articles in order to drive traffic. Meanwhile, the actual KB article doesn't show up in the search results. Fortunately, Bing got it right.)

That explanation is clearly written for a technical audience with deep knowledge of MS-DOS, Windows, and the High Memory Area. matushorvath suggested that "a more detailed explanation could be interesting." I don't know if it's interesting; to me, it's actually quite boring. But here goes.

The A20 line is a signal on the address bus that specifies the contents of bit 20 of the linear address of memory being accessed. If you aren't familiar with the significance of the A20 line, this Wikipedia article provides the necessary background.

The High Memory Area is a 64KB-sized block of memory (really, 64KB minus 16 bytes) that becomes accessible when the CPU is in 8086 mode but the A20 line is enabled. To free up conventional memory, large portions of MS-DOS relocate themselves into the HMA. When a program calls into MS-DOS, it really calls into a stub which enables the A20 line, calls the real function in the HMA, and then disables the A20 line before returning to the program. (The value of the HMA was discovered by my colleague who also discovered the fastest way to get out of virtual-8086 mode.)

The issue is that by default, Windows treats all MS-DOS device drivers and MS-DOS itself as global. A change in one virtual machine affects all virtual machines. This is done for compatibility reasons; after all, those old 16-bit device drivers assume that they are running on a single-tasking operating system. If you were to run a separate copy of each driver in each virtual machine, each copy would try to talk to the same physical device, and bad things would happen because each copy assumed it was the only code that communicated with that device.

Suppose MS-DOS device drivers were treated as local to each virtual machine. Suppose you had a device driver that controlled a traffic signal, and as we all know, one of the cardinal rules of traffic signals is that you never show green in both directions. The device driver has two variables: NorthSouthColor and EastWestColor, and initially both are set to Red. The copy of the device driver running in the first virtual machine decides to let traffic flow in the north/south direction, and it executes code like this:

if (EastWestColor != Red) {
 SetEastWestColor(Red);
}
SetNorthSouthColor(Green);

Since both variables are initially set to Red, this code sets the north/south lights to green.

Meanwhile, the copy of the device driver in the second virtual machine wants to let traffic flow in the east/west direction:

if (NorthSouthColor != Red) {
 SetNorthSouthColor(Red);
}
SetEastWestColor(Green);

Since we have a separate copy of the device driver in each virtual machine, the changes made in the first virtual machine do not affect the values in the second virtual machine. The second virtual machine sees that both variables are set to Red, so it merely sets the east/west color to green.

On the other hand, both of these device drivers are unwittingly controlling the same physical traffic light, and it just got told to set the lights in both directions to Green.

Oops.

Okay, so Windows defaults drivers to global. That way, you don't run into the double-bookkeeping problem. But this causes problems for the code which manages the A20 line:

Consider a system with two virtual machines. The first one calls into MS-DOS. The MS-DOS dispatcher enables the A20 line and calls the real function, but before the function returns, the virtual machine gets pre-empted. The second virtual machine now runs, and it too calls into MS-DOS. The MS-DOS dispatcher in the second virtual machine enables the A20 line and calls into the real function, and after the function returns, the second virtual machine disables the A20 line and returns to its caller. The second virtual machine now gets pre-empted, and the first virtual machine resumes execution. Oops: It tries to resume execution in the HMA, but the HMA is no longer there because the second virtual machine disabled the A20 line!

The WINA20.386 driver teaches Windows that the state of the A20 should be treated as a per-virtual-machine state rather than a global state. With this new information, the above scenario does not run into a problem because the changes to the A20 line made by one virtual machine have no effect on the A20 line in another virtual machine.

matushorvath goes on to add, "I would be very interested in how Windows 3.0 found and loaded this file. It seems to me there must have been some magic happening, e.g. DOS somehow forcing the driver to be loaded by Windows."

Yup, that's what happened, and there's nothing secret about it. When Windows starts up, it broadcasts an interrupt. TSRs and device drivers can listen for this interrupt and respond by specifying that Windows should load a custom driver or request that certain ranges of data should be treated as per-virtual-machine state rather than global state (known to the Windows virtual machine manager as instance data). MS-DOS itself listens for this interrupt, and when Windows sends out the "Does anybody have any special requests?" broadcast, MS-DOS responds, "Yeah, please load this WINA20.386 driver."

So there you have it, the story of WINA20.386. Interesting or boring?

The compatibility constraints of error codes, episode 2

Raymond Chen - MSFT — Fri, 03 Feb 2012 15:00:00 GMT

A customer reported an incompatibility in Windows 7: If A: is a floppy drive and they call LoadLibrary("A:\\foo.dll") and there is no disk in the drive, the LoadLibrary call fails with the error ERROR_NOT_READY. Previous versions of Windows failed with the error ERROR_MOD_NOT_FOUND.

Both error codes are reasonable responses to the situation. "The module couldn't be found because the drive is not ready." Programs should treat a failed LoadLibrary as a failed library load and shouldn't be sensitive to the precise reason for the error. (They can display a more specific error to the user based on the error code, but overall program logic shouldn't depend on the error code.)

Fortunately, the customer discovered this discrepancy during their pre-release testing and were able to accommodate this change in their program before ever releasing it. A sigh of relief from the application compatibility team.

Episode 1.

When you are looking for more information, it helps to say what you need the information for

Raymond Chen - MSFT — Thu, 02 Feb 2012 15:00:00 GMT

It's often the case that when a question from a customer gets filtered through a customer liaison, some context gets lost. (I'm giving the customer the benefit of the doubt here and assuming that it's the customer liaison that removed the context rather than the customer who never provided it.) Consider the following request:

We would like to know more information about the method the shell uses to resolve shortcuts.

This is kind of a vague question. It's like asking "I'd like to know more about the anti-lock braking system in my car." There are any number of pieces of information that could be provided about the anti-lock braking system.

"It requires a Class C data bus."
"The tire position sensors are on the wheel-axis."
"It is connected to the brakes."
"It is shiny."

When we ask the customer, "Could you be more specific what type of information you are looking for?" the response is sometimes

We want to know everything.

This is not a helpful clarification. Do they want to start with Maxwell's Equations and build up from there?

As it happened, in the case of wanting more information about the method the shell uses to resolve shortcuts, they just wanted to know how to disable the search-based algorithm.

This sort of "ask for everything and figure it out later" phenomenon is quite common. I remember another customer who wanted to know "everything" about changing network passwords, and they wouldn't be any more specific than that, so we said, "Well, you can start with these documents, perhaps paying particular attention to this one, but if they tell us what they are going to be doing with the information, we can help steer them to the specific parts that will be most useful to them."

As it turned out, all the customer really wanted to know was "When users change their password, is the new password encrypted on the wire?"

Third example, and then I'll stop. Another customer wanted to know everything about how Explorer takes information from the file system and displays it in an Explorer window. After asking a series of questions, we eventually figured out that they in fact didn't want or need a walkthrough of the entire code path that puts results in the Explorer window. The customer simply wanted to know why two specific folders show up in their Explorer window with names that didn't match the file system name.

When you ask for more information, explain what you need the information for, or at least be more specific what kind of "more information" you need. That way, you save everybody lots of time. The people answering your question don't waste their time gathering information you don't need (and gathering that information can be quite time-consuming), and you don't waste your time sifting through all the information you don't want.

You might say that these people are employing the for-if anti-pattern:

foreach (document d in GetAllPossibleDocumentation())
{
 if (d.Topic == "password encryption on the wire") return d;
}

Things I've written that have amused other people, Episode 9

Raymond Chen - MSFT — Wed, 01 Feb 2012 15:00:00 GMT

A customer liaison reported that their customer wants to be able to access their machine without needing a password. They just want to be able to net use * \\machine\share and be able to access the files right away. I guess because passwords are confusing, easy to forget, and just get in the way. Anyway, the customer discovered that they could do so on Windows XP by going to the folder they want to share, going to the Sharing tab, then clicking on the If you understand the security risks but want to share files without running the wizard link,

and then on the Enable File Sharing dialog, clicking Just enable file sharing.

What the customer wanted to know was if there was a way they could automate this process.

My response to the customer liaison went like this:

Your customer has chosen to ignore not one but two security warnings. Furthermore, since they are looking for an automated way of doing this, it sounds like they intend on deploying this "feature" to all the computers in their organization. Maybe they just enjoy being part of a botnet? Your customer is basically saying "I wish my computer to have no network security." They should at least restrict access to authenticated users. But if they if they insist on having their corporate network turned into a spam farm, they can enable the Guest account and say that it can "Access this computer from the network." Congratulations, your computers will soon be filled with malware and porn.

That last sentence made it into some people's quotes file.

The Freudian typo that will not die: Enchanced video quality

Raymond Chen - MSFT — Tue, 31 Jan 2012 15:00:00 GMT

While ~~wasting time~~ doing valuable background research on my computer, I received the following suggestion:

For enchanced video quality, click here.

It's good to know that the typo that I first encountered in 1993 is still alive and kicking.

(And even though it's not important to the story, people will demand some sort of follow-up, so here it is: I submitted feedback to the vendor, who said that it was a known issue fixed in the next update.)

Why does it take Task Manager longer to appear when you start it from the Ctrl+Alt+Del dialog?

Raymond Chen - MSFT — Mon, 30 Jan 2012 15:00:00 GMT

Amit was curious why it takes longer for Task Manager to appear when you start it from the Ctrl+Alt+Del dialog compared to launching it from the taskbar.

Well, you can see the reason right there on the screen: You're launching it the long way around.

If you launch Task Manager from the taskbar, Explorer just launches taskmgr.exe via the usual CreateProcess mechanism, and Task Manager launches under the same credentials on the same desktop.

On the other hand, when you use the secure attention sequence, the winlogon program receives the notification, switches to the secure desktop, and displays the Ctrl+Alt+Del dialog. When you select Task Manager from that dialog, it then has to launch taskmgr.exe, but it can't use the normal CreateProcess because it's on the wrong desktop and it's running under the wrong security context. (Because winlogon runs as SYSTEM, as Task Manager will tell you.)

Clearly, in order to get Task Manager running on your desktop with your credentials, winlogon needs to change its security context, change desktops, and then launch taskmgr.exe. The desktop switch is probably the slowest part, since it involves the video driver, and video drivers are not known for their blazingly fast mode changes.

It's like asking why an international package takes longer to deliver than a domestic one. Because it's starting from further away, and it also has to go through customs.

Does mapping the same shared memory two times in a process lead to double the address space usage?

Raymond Chen - MSFT — Fri, 27 Jan 2012 15:00:00 GMT

A customer designed a system which uses shared memory. Specifically, for each database file, they create a corresponding shared memory block of, say, 200MB. Multiple clients which connect to the same database file use the same shared memory block. Naturally, if two processes each access the same database file, each process will map the shared memory block into their respective address space. The question arose regarding what happens if one process connects to the same database file twice. Will the two calls to MapViewOfFile share the same address space, or will each one allocate a separate chunk of address space?

Win32 makes no guarantees what will happen. All that you can be sure of is that the memory will be mapped into your address space, and you will get a pointer to it, and when you're done, you call UnmapViewOfFile. Whether the two calls return the same pointer is unspecified.

In fact, Windows 95 returned the same pointer, whereas Windows NT returns a different pointer. We saw this earlier when we intentionally mapped the same shared memory block multiple times, and observed somebody actually taking a dependency on this behavior in order to effect the strangest way of detecting Windows NT. Don't take a dependency on this behavior; who knows, maybe a future version of Windows NT will consolidate multiple mappings in order to conserve address space.

If you want force this consolidation behavior, you'll have to roll it yourself, say with a lookup table of active mappings and a reference count.

The 2012/2013 Seattle Symphony subscription season at a glance

Raymond Chen - MSFT — Thu, 26 Jan 2012 15:00:01 GMT

Every year, I put together a little pocket guide to the Seattle Symphony subscription season for my symphony friends to help them decide which ticket package they want. As before, you might find it helpful, you might not, but either way, you're going to have to suffer through it. Here's the at-a-glance season guide for the 2012/2013 season. (Full brochure. Seattle Times coverage.)

Week	Program	Comments	21	13	7A 7B	7C 7D	7E 7F	7G	4A	BS	SU	WG
Week	Program	Comments	21	13	7A 7B	7C 7D	7E 7F	7G	4A	BS	SU	WG
09/20 2012	Berlioz: Roman Carnival Overture Martinů: Symphony #6 Fantaises symphoniques Debussy: Nocturnes: Nuages and Fêtes Respighi: Pines of Rome	Good Nervous Excellent Excellent
10/04 2012	Mussorgsky: Night on Bald Mountain Tchaikovsky: Rococo Variations Sibelius: Symphony #1	Awesome Good Excellent
10/18 2012	Beethoven: Coriolan Overture Mozart: Sinfonia Concertante K297b Fujikura: Mina† Haydn: Symphony #103 Drum Roll	Awesome Excellent Wildcard Awesome
10/26	Sonic Evolution	Wildcard
11/01 2012	Tchaikovsky: The Snow Maiden Suite Tchaikovsky: Violin Concerto Prokofiev: Symphony #6	Good Awesome Nervous
11/08 2012	Adams: Harmonielehre Beethoven: Piano Concerto #5	Polarizing Awesome
11/15 2012	Brahms: Piano Concerto #2 Dutilleux: The Shadow of Time R. Strauss: Till Eulenspiegel	Awesome Nervous Good
11/29 2012	Berg: Violin Concerto Mahler: Symphony #4	Polarizing Polarizing
01/10 2013	Stravinsky: Pulcinella Suite Mendelssohn: Piano Concerto #1 Mozart: Symphony #39	Excellent Excellent Awesome
01/31	Messiaen: Turangalîla Symphony	Polarizing
02/07 2013	Rossini: William Tell Overture Schumann: Piano Concerto Brahms: Symphony #4	Awesome Excellent Awesome
02/14 2013	(Rossini: William Tell Overture) Faure: Pelléas et Mélisande Suite Mozart: Piano Concerto #21 Ravel: Shéherazade Szymanowski: Symphony #4	Awesome Good Awesome Nervous Nervous
03/14 2013	Tippett: The Midsummer Marriage: Ritual Dances Bruch: Violin Concerto #1 Elgar: Enigma Variations	Nervous Awesome Excellent
03/21 2013	Mozart: Don Giovanni Overture Britten: Cello Symphony Beethoven: Symphony #5	Awesome Nervous Awesome
03/28 2013	Liadov: The Enchanted Lake Kancheli: Styx Rimsky-Korsakov: Scheherezade	Okay Nervous Awesome
04/13 2013	Mozart: Piano Concerto #9 Bruckner: Symphony #4 Romantic	Awesome Excellent
04/18 2013	Antheil: A Jazz Symphony Gruber: Percussion Concerto Rough Music Bernstein: On the Waterfront Suite Stravinsky: The Firebird Suite	Excellent Okay Good Excellent
04/25 2013	Sibelius: Karelia Overture Sibelius: Violin Concerto Zavaro: New Work† Beethoven: Symphony #7	Good Awesome Wildcard Awesome
05/30 2013	Smetana: Wallenstein's Camp Beethoven: Violin Concerto Dvořák: Symphony #6	Excellent Awesome Awesome
06/13	Britten: War Requiem	Nervous
06/20 2013	Shostakovich: Violin Concerto #1 John Luther Adams: Become Ocean†	Nervous Okay
06/27 2013	Saint-Saëns: Organ Symphony Wagner: Tristan: Prelude and Liebestod Wagner: Tannhäuser Overture and Venusberg Music	Awesome Okay Excellent

Legend:

21	Masterworks 21-concert series (Choice of Thursdays or Saturdays)
13	Masterworks 13-concert series (Choice of Thursdays or Saturdays)
7A	Masterworks 7-concert series A (Thursdays)
7B	Masterworks 7-concert series B (Saturdays)
7C	Masterworks 7-concert series C (Thursdays)
7D	Masterworks 7-concert series D (Saturdays)
7E	Masterworks 7-concert series E (Thursdays)
7F	Masterworks 7-concert series F (Saturdays)
7G	Masterworks 7-concert series G (Sunday afternoons)
4A	Masterworks 4-concert series A (Friday afternoons)
BS	Beyond the Score multimedia lecture-concert (Sunday afternoons)
SU	Symphony Untuxed (Fridays)
WG	WolfGang (Various evenings)
†	Premiere

For those not familiar with the Seattle Symphony ticket package line-ups: Most of the ticket packages are named Masterworks nX where n is the number is the number of concerts in the package, and the letter indicates which variation. Ticket packages have been combined if they are identical save for the day of the week. For example, 7C and 7D are the same concerts; the only difference is that 7C is for Thursday nights, while 7D is for Saturday nights. The Beyond the Score concerts focus on only one of the pieces. The WolfGang series is available only to members of the WolfGang club.

This chart doesn't include "one-off" concert series such as the Mainly Mozart or Distinguished Artists series. A "one-off" series is a concert series which shares no concerts with any other series.

Changes from last season:

The Rush Hour series was redesigned and became the Symphony Untuxed series.
Symphony Specials was dropped.
The Baroque & Wine Series and Mainly Mozart concerts still exist, but I dropped them from the table since they are now one-off series.
WolfGang already existed but I added it to the list now that there's room for it.

The comments column very crudely categorizes the works to assist my less-classically-aware friends. This is, of course, a highly subjective rating system, but I tried to view each piece from the ears of my symphony friends. Thus, I rated downward pieces that I personally like but which others might not and rated up pieces that I may not find musically satisfying but which nevertheless tend to be crowd-pleasers.

These predictions have, of course, proven wrong in the past.

Here's what the comments mean. Note that they do not indicate whether the piece is significant in a musicological sense; they're just my guess as to whether my friends are going to like it. (For example, I know that my friends don't like minimalism, and I suspect they don't like serialism, so I rated the Adams and Berg down even though I think they're quite good. They also don't like vocal pieces. On the other hand, it turns out that I overcame my Bruckner jinx, so I can at least give the Bruckner a positive score this time. Let's just hope it's not the Hass edition.)

Awesome: Guaranteed crowd-pleaser.
Excellent: You will definitely like this piece.
Good: You will probably like this piece.
Okay: You may like this piece.
Nervous: I have a bad feeling about this one.
Polarizing: Some people will love it; others will hate it.
Wildcard: I have no idea what will happen.

In many cases, I am not familiar with the piece and am basing my evaluation on what I know about the composer (or am just guessing).

Why doesn't the Windows 7 Start menu have a pushpin for pinning items?

Raymond Chen - MSFT — Thu, 26 Jan 2012 15:00:00 GMT

You may have noticed a minor inconsistency between pinning a program to the Start menu and pinning a destination to a program's Jump List. Although pinned items appear at the top of the respective lists, and both the Start menu and Jump List let you right-click an item and select Pin/Unpin, the Jump List also lets you pin and unpin an item by clicking on the pushpin. Why doesn't the Start menu have a pushpin in addition to the right-click menu?

For a time, items on the Start menu did have a pushpin, just like items on Jump Lists. The design had a few problems, however. Start menu items can also have a triangle indicating the presence of a flyout menu, and the presence of two indicators next to an item made the interface look awkward and too busy. And what do you do if an item has only one indicator? Do you right-justify all the indicators? Or do you place the indicators in columns and reserve blank space for the missing ones?

Internet Explorer	¤ ▶
Command Prompt	¤
Notepad	▶
Calculator

Internet Explorer	¤	▶
Command Prompt	¤
Notepad		▶
Calculator

Both look ugly for different reasons. The right-justify-everything version looks ugly because the pushpin appears to keep moving around. The blank-space-if-no-flyout version looks ugly because you have a pushpin hovering in the middle of nowhere. (Imagine trying to click on one of these things: You just have to "know" that the magic click spot for pinning an item is 20 pixels to the left of the far right edge.)

But the real death blow to showing a pushpin for pinning items to the Start menu was the usability testing. Users had trouble figuring out where to click to pin an item or to open the Jump List and frequently got the opposite of what they wanted. Since opening the Jump List is by far the more common operation, it won the battle of the prominent UI affordance, and the option for pinning and unpinning was left to a context menu.

Which, as it happens, is where the pin/unpin option started in the first place.

How do I disable the fault-tolerant heap?

Raymond Chen - MSFT — Wed, 25 Jan 2012 15:00:00 GMT

A while back, I linked to a talk by Silviu Calinoiu on the fault-tolerant heap. But what if you don't want the fault-tolerant heap? For example, during program development, you probably want to disable the fault-tolerant heap for your program: If the program is crashing, then it should crash so you can debug it!

Method 1 is to disable the fault-tolerant heap globally. While this prevents the fault-tolerant heap from auto-activating in the future, it does not go back and undo activations that were enabled in the past. In other words, you have to remember to do this before your application crashes for the first time.

Therefore, you probably want to combine Method 1 with Method 2 on the same page, where it gives instructions on how to reset the list of applications for which the fault-tolerant heap is enabled.

Mario Raccagni provides a third way of disabling the fault tolerant heap, this time for one specific process instead of globally. His explanation is in Italian, so you get to exercise your translation skills.

tl;dr version: Go to the HKEY_LOCAL_MACHINE and HKEY_CURRENT_USER versions of Software\Microsoft\Windows NT\CurrentVersion\AppCompatFlags\Layers\your_application.exe and delete the FaultTolerantHeap entry.

A single-handed effort to keep the memory of $2 bills alive

Raymond Chen - MSFT — Tue, 24 Jan 2012 15:00:00 GMT

As I noted when I told the story of the computer programmer who dabbled in making change that my colleague had a lot of money-related quirks.

For some reason my colleague felt the $2 bill deserved more attention. Every so often, he would go to the bank and buy $100 in $2 bills, then reintroduce the bills into circulation and enjoy people's reactions to them. (Most cashiers looked at it and recognized that it was legal tender, but couldn't find a good place to put it in the till. It usually got tossed under the drawer with all the checks.)

It was a regular occurrence that the bank didn't have that many $2 bills on hand, but they managed to find them and let him know when he could come pick them up.

One time, the bank called him back. "Hi, we asked all our branches in the entire county, but all together we can't find enough $2 bills. If you want, we can place an order with the Federal Reserve. The catch is, though, that the minimum order is $2000."

"Sure, go ahead and place the order."

Some time later, he went in to pick up his huge stack of $2 bills.

My colleague now found himself in a situation where something fun turned into an ordeal, like a smoker who is forced to smoke an entire pack of cigarettes at one sitting. Or in this case, more like 1000 cigarettes.

At the end of group meals at a restaurant, after everybody had calculated their share and put their money in the bill holder (this being the days when people actually paid cash for things), he would raid the bill holder for change, taking out all the notes greater than $2 and replacing them with the appropriate number of $2 bills. As a result, when the servers came to collect the bill holders, they found them stuffed with $1 and $2 bills (mostly $2).

Too bad he didn't make a pad out of them.

Bonus reading: $1 billion that nobody wants. Follow-up.

Can OANOCACHE be used for non-debug purposes?

Raymond Chen - MSFT — Mon, 23 Jan 2012 15:00:00 GMT

Friday asks whether OANOCACHE can be used for non-debug purposes, say to improve stability and/or speed.

You can try, but it's not recommended. For one thing, it probably damages stability, because there are many applications out there which unwittingly rely on the BSTR cache to protect them from heap corruption bugs. The Windows team has for years wanted to tweak the BSTR cache (even going so far as getting rid of it entirely), but the compatibility issues always return and quash any attempts at radical restructuring.

Identifying applications that rely on the BSTR cache and deploying an appropriate compatibility shim would be one thing. It's applications which support third party plug-ins that are the sticky point. You say, "Okay, this application is fine, we'll use the new BSTR cache behavior" and then it loads a plug-in DLL that requires the old BSTR cache behavior. Now you're stuck since you don't have a time machine.

How do FILE_FLAG_SEQUENTIAL_SCAN and FILE_FLAG_RANDOM_ACCESS affect how the operating system treats my file?

Raymond Chen - MSFT — Fri, 20 Jan 2012 15:00:00 GMT

There are two flags you can pass to the CreateFile function to provide hints regarding your program's file access pattern. What happens if you pass either of them, or neither?

Note that the following description is not contractual. It's just an explanation of the current heuristics (where "current" means "Windows 7"). These heuristics have changed at each version of Windows, so consider this information as a tip to help you choose an appropriate access pattern flag in your program, not a guarantee that the cache manager will behave in a specific way if you do a specific thing.

If you pass the FILE_FLAG_SEQUENTIAL_SCAN flag, then the cache manager alters its behavior in two ways: First, the amount of prefetch is doubled compared to what it would have been if you hadn't passed the flag. Second, the cache manager marks as available for re-use those cache pages which lie entirely behind the current file pointer (assuming there are no other applications using the file). After all, by saying that you are accessing the file sequentially, you're promising that the file pointer will always move forward.

At the opposite extreme is FILE_FLAG_RANDOM_ACCESS. In the random access case, the cache manager performs no prefetching, and it does not aggressively evict pages that lie behind the file pointer. Those pages (as well as the pages that lie ahead of the file pointer which you already read from or wrote to) will age out of the cache according to the usual most-recently-used policy, which means that heavy random reads against a file will not pollute the cache (the new pages will replace the old ones).

In between is the case where you pass neither flag.

If you pass neither flag, then the cache manager tries to detect your program's file access pattern. This is where things get weird.

If you issue a read that begins where the previous read left off, then the cache manager performs some prefetching, but not as much as if you had passed FILE_FLAG_SEQUENTIAL_SCAN. If sequential access is detected, then pages behind the file pointer are also evicted from the cache. If you issue around six reads in a row, each of which begins where the previous one left off, then the cache manager switches to FILE_FLAG_SEQUENTIAL_SCAN behavior for your file, but once you issue a read that no longer begins where the previous read left off, the cache manager revokes your temporary FILE_FLAG_SEQUENTIAL_SCAN status.

If your reads are not sequential, but they still follow a pattern where the file offset changes by the same amount between each operation (for example, you seek to position 100,000 and read some data, then seek to position 150,000 and read some data, then seek to position 200,000 and read some data), then the cache manager will use that pattern to predict the next read. In the above example, the cache manager will predict that your next read will begin at position 250,000. (This prediction works for decreasing offsets, too!) As with auto-detected sequential scans, the prediction stops as soon as you break the pattern.

Since people like charts, here's a summary of the above in tabular form:

Access pattern	Prefetch	Evict-behind
Explicit random	No	No
Explicit sequential	Yes (2×)	Yes
Autodetected sequential	Yes	Yes
Autodetected very sequential	Yes (2×)	Yes
Autodetected linear	Yes	?
None	No	?

There are some question marks in the above table where I'm not sure exactly what the answer is.

Note: These cache hints apply only if you use ReadFile (or moral equivalents). Memory-mapped file access does not go through the cache manager, and consequently these cache hints have no effect.

Why do Microsoft customer records use the abbreviation "cx" for customer?

Raymond Chen - MSFT — Thu, 19 Jan 2012 15:00:00 GMT

As is common in many industries, Microsoft customer service records employ abbreviations for many commonly-used words. In the travel industry, for example, pax is used as an abbreviation for passenger. The term appears to have spread to the hotel industry, even though people who stay at a hotel aren't technically passengers. (Well, unless you think that with the outrageous prices charged by the hotels, the people are being taken for a ride.)

For a time, the standard abbreviation for customer in Microsoft's customer service records was cu. This changed, however, when it was pointed out to the people in charge of such things that cu is a swear word in Portuguese. The standard abbreviation was therefore changed to cx.

If you're reading through old customer records and you know Portuguese and you see the word cu, please understand that we are not calling the customer a rude name.

The person who introduced me to this abbreviation added, "I just spell out the word. It's not that much more work, and it's a lot easier to read."

Some years ago, I was asked to review a technical book, and one of the items of feedback I returned was that the comments in the code fragments were full of mysterious abbreviations. "Sgnl evt before lv cs." I suggested that the words be spelled out or, if you really want to use abbreviations, at least have somewhere in the text where the abbreviations are explained.

If I had wanted to demonstrate the social skills of a thermonuclear device, my feedback might have read "unls wrtg pzl bk, avd unxplnd n unnec abbvs."

Don't try to allocate memory until there is only x% free

Raymond Chen - MSFT — Wed, 18 Jan 2012 15:00:00 GMT

I have an ongoing conflict with my in-laws. Their concept of the correct amount of food to have in the refrigerator is "more than will comfortably fit." Whenever they come to visit (which is quite often), they make sure to bring enough food so that my refrigerator bursts at the seams, with vegetables and eggs and other foodstuffs crammed into every available nook and cranny. If I'm lucky, the amount of food manages to get down to "only slightly overfull" before their next visit. And the problem isn't restricted to the refrigerator. I once cleared out some space in the garage, only to find that they decided to use that space to store more food. (Who knows, maybe one day I will return from an errand to find that my parking space has been filled with still more food while I was gone.)

Occasionally, a customer will ask for a way to design their program so it continues consuming RAM until there is only x% free. The idea is that their program should use RAM aggressively, while still leaving enough RAM available (x%) for other use. Unless you are designing a system where you are the only program running on the computer, this is a bad idea.

Consider what happens if two programs try to be "good programs" and leave x% of RAM available for other purposes. Let's call the programs Program 10 (which wants to keep 10% of the RAM free) Program 20 (which wants to keep 20% of the RAM free). For simplicity, let's suppose that they are the only two programs on the system.

Initially, the computer is not under memory pressure, so both programs can allocate all the memory they want without any hassle. But as time passes, the amount of free memory slowly decreases.

Program 10 (20%)

Free (60%)

Program 20 (20%)

Program 10 (30%)

Free (40%)

Program 20 (30%)

Program 10 (40%)

Free (20%)

Program 20 (40%)

And then we hit a critical point: The amount of free memory drops below 20%.

Program 10 (41%)

Free (18%)

Program 20 (41%)

At this point, Program 20 backs off in order to restore the amount of free memory back to 20%.

Program 10 (41%)

Free (20%)

Program 20 (39%)

Now, each time Program 10 and Program 20 think about allocating more memory, Program 20 will say "Nope, I can't do that because it would send the amount of free memory below 20%." On the other hand, Program 10 will happily allocate some more memory since it sees that there's a whole 10% it can allocate before it needs to stop. And as soon as Program 10 allocates that memory, Program 20 will free some memory to bring the amount of free memory back up to 20%.

Program 10 (42%)

Free (19%)

Program 20 (39%)

Program 10 (42%)

Free (20%)

Program 20 (38%)

Program 10 (43%)

Free (19%)

Program 20 (38%)

Program 10 (43%)

Free (20%)

Program 20 (37%)

Program 10 (44%)

Free (19%)

Program 20 (37%)

Program 10 (44%)

Free (20%)

Program 20 (36%)

I think you see where this is going. Each time Program 10 allocates a little more memory, Program 20 frees the same amount of memory in order to get the total free memory back up to 20%. Eventually, we reach a situation like this:

Program 10 (75%)

Free (20%)

P20 (5%)

Program 20 is now curled up in the corner of the computer in a fetal position. Program 10 meanwhile continues allocating memory, and Program 20, having shrunk as much as it can, is forced to just sit there and whimper.

Program 10 (76%)

Free (19%)

P20 (5%)

Program 10 (77%)

Free (18%)

P20 (5%)

Program 10 (78%)

Free (17%)

P20 (5%)

Program 10 (79%)

Free (16%)

P20 (5%)

Program 10 (80%)

Free (15%)

P20 (5%)

Program 10 (81%)

Free (14%)

P20 (5%)

Program 10 (82%)

Free (13%)

P20 (5%)

Program 10 (83%)

Free (12%)

P20 (5%)

Program 10 (84%)

Free (11%)

P20 (5%)

Program 10 (85%)

Free (10%)

P20 (5%)

Finally, Program 10 stops allocating memory since it has reached its own personal limit of not allocating the last 10% of the computer's RAM. But it's too little too late. Program 20 has already been forced into the corner, thrashing its brains out trying to survive on only 5% of the computer's memory.

It's sort of like when people from two different cultures with different concepts of personal space have a face-to-face conversation. The person from the not-so-close culture will try to back away in order to preserve the necessary distance, while the person from the closer-is-better culture will move forward in order to close the gap. Eventually, the person from the not-so-close culture will end up with his back against the wall anxiously looking for an escape route.

Microspeak: Walls and ladders

Raymond Chen - MSFT — Tue, 17 Jan 2012 15:00:00 GMT

Reader laonianren wanted to know more about this game Walls and Ladders.

"Walls and Ladders" is not a game. It's just a metaphor for a conflict in which one side wants to perform some action and the other side wants to prevent it. The defending side builds a wall, and the attacking side builds a taller ladder. In response, the defending side builds a taller wall, and the attacking side builds an even taller ladder. The result of this conflict is that the defending side constructs an ever-more-elaborate wall and the attacking side constructs a more-and-more complex ladder [link possible NSFW], both sides expending ridiculous amounts of resources and ultimately ending up back where they started.

There is a closely-related metaphor known as an arms race. In an arms race, each participant wants to be the most X, for some property X. An arms race tends to be all-attack, whereas wall-and-ladders tends to have one side attacking and the other defending.

Since many conflicts can be phrased either as an attack-attack scenario or an attack-defend scenario (some defenses may include counter-attacks), I tend to get the two confused. Notice, for example, that my arms race article contains mostly walls-and-ladders scenarios; for example, a case where one side wants to terminate a process and another wants to prevent it from being terminated. On the other hand, my wall and ladders example was really more of an arms race, with both sides wanting to take control of the screen.

Depending on which group you work with at Microsoft, you may find a preference for walls and ladders over arms race, probably due to the same sensitivity to military terms that led to the War Room being renamed Ship Room. (I seem to recall that there was a lawsuit that among other things alleged that the fact that a Microsoft project called its daily meeting room the War Room was proof of Microsoft's evil essence.)

Cultural arbitrage: The food-related sucker bet

Raymond Chen - MSFT — Mon, 16 Jan 2012 15:00:01 GMT

While I was at a group dinner at a Chinese restaurant, a whole fish was brought to our table. One of the other people at the table told a story of another time a whole fish was brought to the table.

He attended the wedding rehearsal dinner of a family member. The bride is Chinese, but the groom is not. (Or maybe it was the other way around. Doesn't matter to the story.) The dinner was banquet-style at a Chinese restaurant, and one of the many courses was a whole fish.

Two of the non-Chinese attendees marveled at the presence of an entire fish right there in front of them, head, tail, fins, and all. I guess they had up until then only been served fish that had already been filleted, or at least had the head cut off. One of them nudged my acquaintance and said, "We'll give you $500 if you eat the eyeball."

These guys inadvertently created their own sucker bet.

For you see, eating the eyeball is common in many parts of Asia. In fact, whenever their family has fish, my nieces fight over who gets the honor of eating the eyeballs!

I don't know whether my acquaintance cheerfully accepted the bet or whether he explained that their bet was a poor choice to offer a Chinese person.

What food-related sucker bets exist in your culture? (I'm not talking about foods like chicken feet or tongue, which are clearly prepared and served to be eaten. I'm talking about things that an uninitiated person might consider to be a garnish or an inedible by-product, like shrimp heads.)

Update: I remind you that the question is not asking for foods which are served as dishes on their own.

Why was there a font just for drawing symbols on buttons?

Raymond Chen - MSFT — Mon, 16 Jan 2012 15:00:00 GMT

Henke37 wonders why the Marlett font was introduced. Why use a font for drawing symbols on window buttons?

Using a font was a convenient way to have scalable graphics.

It's not like Windows could've used VML or SVG since they hadn't been invented yet. EMFs would have been overkill as well. Fonts were very convenient because the technology to render scalable fonts already existed and was well-established. It's always good to build on something that has been proven, and TrueType scalable font technology proved itself very nicely in Windows 3.1. TrueType has the added benefit of supporting hinting, allowing tweaks to the glyph outlines to be made for particular pixel sizes. (A feature not available in most vector drawing languages, but also a feature very important when rendering at small font sizes.)

Keys duplicated from photo: Delayed reaction

Raymond Chen - MSFT — Fri, 13 Jan 2012 15:00:01 GMT

There was a report some time ago that researchers have developed a way to duplicate keys given only a photograph. When I read this story, I was reminded of an incident that occurred to a colleague of mine.

He accidentally locked his keys in his car and called a locksmith. Frustratingly, the keys were sitting right there on the driver's seat. The locksmith arrived and assessed the situation. "Well, since you already paid for me to come all the way out here, how would you like a spare key?"

"Huh? What do you mean?"

The locksmith looked at the key on the driver's seat, studied it intently for a few seconds, then returned to his truck. A short while later, he returned with a freshly-cut key, which he inserted into the door lock.

The key worked.

How do I print non-error messages during compilation?

Raymond Chen - MSFT — Fri, 13 Jan 2012 15:00:00 GMT

Commenter Worf remarked, "My one wish is that #warning would be supported."

I always find it interesting when people say "I wish that Microsoft would stop following standards," since the #warning directive is nonstandard.

The Microsoft C/C++ compiler implements the feature in a method compatible with the standard, namely via a #pragma directive.


#pragma message("You really shouldn't be doing that.")

If you want to warn people away from deprecated functionality, you can use the #pragma deprecated() directive or the even more convenient (but more standards-troublesome) __declspec(deprecated) declaration specifier. The declaration specifier is much more convenient than the preprocessor directive because you can use it in a macro, and you can attach it to specific overloads of a function. (It's also more standards-troublesome because, while it is still permitted by the standard because it begins with a double-underscore, it is also not required to be ignored by compilers which do not understand it.)

In my experience, however, printing messages during compilation is of little consequence. Print all the messages during compilation as you want; nobody will read them. The only thing that gets attention is an actual warning or error. (And in many cases, only the error will get any attention at all.)

Puzzling out the upsell-o-meter

Raymond Chen - MSFT — Thu, 12 Jan 2012 15:00:01 GMT

As I noted before, many grocery stores in the United States have a printer next to the cash register which prints out coupons customized to your purchases. Here's a purchase and the accompanying coupon. What is the story behind this pairing?

Purchased: Diapers for newborn baby.
Coupon: Save 75 cents on ice cream.

Bonus chatter: While waiting in line, I read the warning label on the diapers. It went on for quite a bit, but one part triggered my "I wonder what lawsuit led to this warning" sensor: "Like most articles of clothing, XYZ brand diapers will burn if exposed to flame." Did somebody say, "Oh no, there's a fire, what will I do? I know, I'll smother the fire with my baby's diapered bottom!"

Why does CreateEvent fail with ERROR_PATH_NOT_FOUND if I give it a name with a backslash?

Raymond Chen - MSFT — Thu, 12 Jan 2012 15:00:00 GMT

A customer reported that the CreateEvent function was failing with the unusual error code ERROR_PATH_NOT_FOUND:

HANDLE h = CreateEvent(0, FALSE, TRUE, "willy\\wonka");
if (h == NULL) {
 DWORD dwError = GetLastError(); // returns ERROR_PATH_NOT_FOUND
 ...
}

The customer continued, "The documentation for CreateEvent says that the lpName parameter must not contain the backslash character. Clearly we are in error for having passed an illegal character, but why are we getting the strange error code? There is no file path involved. Right now, we've added ERROR_PATH_NOT_FOUND to our list of possible error codes, but we'd like an explanation of what the error means."

Okay, first of all, building a table of all known error codes is another compatibility problem waiting to happen. Suppose in the next version of Windows, a new error code is added, say, ERROR_REJECTED_BY_SLASHDOT. What will your program do when it gets this new error code?

Now back to the error code. There is no file path involved here, so why is there a path-not-found error?

Because it's not a file system path that failed. It's an object namespace path.

If a backslash appears in the name of a named object, it is treated as a namespace separator. (If there is no backslash, the name is interpreted as part of the Local namespace.) And the call fails with a path-not-found error since there is no namespace called willy, so the path traversal inside the object namespace fails.

The treatment of the backslash as a namespace separator is sort of alluded to in the very next sentence of the documentation: "For more information, see Kernel Object Namespaces." The following paragraph also expands upon this idea: "The object can be created in a private namespace. For more information, see Object Namespaces." The documentation sort of assumes you'll follow the links and learn more about those namespacey things, at which point you'll learn what that backslash in the object name really means (and why there is the rule about not allowing backslashes).

But here it is if you don't want to try to figure it out:

"If you put a backslash in the name, it is treated as a namespace separator, and if you don't know what a namespace is, then that's probably not what you want. So don't use backslashes unless you know what you're doing."

What a steal: A house for only ten dollars!

Raymond Chen - MSFT — Wed, 11 Jan 2012 15:00:01 GMT

When I was signing the papers for a house purchase many years ago, I noticed that the deed papers read

The Grantor «names of people selling the house»
for and in consideration of TEN DOLLARS AND OTHER GOOD AND VALUABLE CONSIDERATION
in hand paid, conveys and warrants to «me»
the following described real estate...

I noticed that I technically was buying the house for ten dollars.

The closing agent explained, "Well, ten dollars and other consideration. This is just a convention, so that the actual amount paid for the house doesn't go into the record." It also saves them from having to revise the document each time the price changes. (I don't buy the "doesn't go into the record" argument, because you can still look up the actual sale price from public records. But this back in the days before Zillow, where this sort of information was not available online and if you wanted to look up this information, you had to gasp physically visit the county records office.)

When I told this story to one of my friends who was also buying a house, he said, "Cool. I'm going to ask them to put down $20 when they draw up the papers for my house."

Therefore, according to the title transfer records, he paid twice as much for his house than I did mine.