The Old New Thing

Fancy use of exception handling in FormatMessage leads to repeated "discovery" of security flaw

2012-02-10T15:00:00Z

Every so often, somebody "discovers" an alleged security vulnerability in the FormatMessage function. You can try it yourself:

#include <windows.h>
#include <stdio.h>

char buf[2048];
char extralong[128*1024];

int __cdecl main(int argc, char **argv)
{
 memset(extralong, 'x', 128 * 1024 - 1);
 DWORD_PTR args[] = { (DWORD_PTR)extralong };
 FormatMessage(FORMAT_MESSAGE_FROM_STRING |
               FORMAT_MESSAGE_ARGUMENT_ARRAY, "%1", 0, 0,
               buf, 2048, (va_list*)args);
 return 0;
}

If you run this program under the debugger and you tell it to break on all exceptions, then you will find that it breaks on an access violation trying to write to an invalid address.

eax=00060078 ebx=fffe0001 ecx=0006fa34 edx=00781000 esi=0006fa08 edi=01004330
eip=77f5b279 esp=0006f5ac ebp=0006fa1c iopl=0         nv up ei pl nz na pe cy
cs=001b  ss=0023  ds=0023  es=0023  fs=0038  gs=0000             efl=00010203
ntdll!fputwc+0x14:
77f5b279 668902           mov     [edx],ax              ds:0023:00781000=????

Did you just find a buffer overflow security vulnerability?

The FormatMessage function was part of the original Win32 interface, back in the days when you had lots of address space (two whole gigabytes) but not a lot of RAM (12 megabytes, or 16 if you were running Server). The implementation of FormatMessage reflects this historical reality by working hard to conserve RAM but not worrying too much about conserving address space. And it takes advantage of this fancy new structured exception handling feature.

The FormatMessage uses the reserve a bunch of address space but commit pages only as they are necessary pattern, illustrated in MSDN under the topic Reserving and Committing Memory. Except that the sample code on that page contains serious errors. For example, if the sample code encounters an exception other than STATUS_ACCESS_VIOLATION, it still "handles" it by doing nothing and returning EXCEPTION_EXECUTE_HANDLER. It fails to handle random access to the buffer or access violations caused by DEP. Though in the very specific sample, it mostly works since the protected region does only one thing, so there aren't many opportunities for the other types of exceptions to occur. (Though if you're really unlucky, you might get an STATUS_IN_PAGE_ERROR.) But enough complaining about that sample.

The FormatMessage function reserves 64KB of address space, commits the first page, and then calls an internal helper function whose job it is to generate the output, passing the start of the 64KB block of address space as the starting address and telling it to give up when it reaches 64KB. Something like this:

struct DEMANDBUFFER
{
  void *Base;
  SIZE_T Length;
};

int
PageFaultExceptionFilter(DEMANDBUFFER *Buffer,
                         EXCEPTION_RECORD ExceptionRecord)
{
  int Result;

  DWORD dwLastError = GetLastError();

  // The only exception we handle is a continuable read/write
  // access violation inside our demand-commit buffer.
  if (ExceptionRecord->ExceptionFlags & EXCEPTION_NONCONTINUABLE)
    Result = EXCEPTION_CONTINUE_SEARCH;

  else if (ExceptionRecord->ExceptionCode != EXCEPTION_ACCESS_VIOLATION)
    Result = EXCEPTION_CONTINUE_SEARCH;

  else if (ExceptionRecord->NumberParameters < 2)
    Result = EXCEPTION_CONTINUE_SEARCH;

  else if (ExceptionRecord->ExceptionInformation[0] &
      ~(EXCEPTION_READ_FAULT | EXCEPTION_WRITE_FAULT))
    Result = EXCEPTION_CONTINUE_SEARCH;

  else if (ExceptionRecord->ExceptionInformation[1] -
      (ULONG_PTR)Buffer->Base >= Buffer->Length)
    Result = EXCEPTION_CONTINUE_SEARCH;

  else {
    // If the memory is already committed, then committing memory won't help!
    // (The problem is something like writing to a read-only page.)
    void *ExceptionAddress = (void*)ExceptionInformation[1];
    MEMORY_BASIC_INFORMATION Information;
    if (VirtualQuery(ExceptionAddress, &Information,
                     sizeof(Information)) != sizeof(Information))
      Result = EXCEPTION_CONTINUE_SEARCH;

    else if (Information.State != MEM_RESERVE)
      Result = EXCEPTION_CONTINUE_SEARCH;

    // Okay, handle the exception by committing the page.
    // Exercise: What happens if the faulting memory access
    // spans two pages?
    else if (!VirtualAlloc(ExceptionAddress, 1, MEM_COMMIT, PAGE_READWRITE))
      Result = EXCEPTION_CONTINUE_SEARCH;

    // We successfully committed the memory - retry the operation
    Result = EXCEPTION_CONTINUE_EXECUTION;
  }

  RestoreLastError(dwLastError);
  return Result;
}

DWORD FormatMessage(...)
{
  DWORD Result = 0;
  DWORD Error;
  DEMANDBUFFER Buffer;
  Error = InitializeDemandBuffer(&Buffer, FORMATMESSAGE_MAXIMUM_OUTPUT);
  if (Error == ERROR_SUCCESS) {
    __try {
     Error = FormatMessageIntoBuffer(&Result,
                                     Buffer.Base, Buffer.Length, ...);
    } __except (PageFaultExceptionFilter(&Buffer,
                   GetExceptionInformation()->ExceptionRecord)) {
     // never reached - we never handle the exception
    }
  }
  if (Error == ERROR_SUCCESS) {
   Error = CopyResultsOutOfBuffer(...);
  }
  DeleteDemandBuffer(&Buffer);
  if (Result == 0) {
    SetLastError(Error);
  }
  return Result;
}

The FormatMessageIntoBuffer function takes an output buffer and a buffer size, and it writes the result to the output buffer, stopping when the buffer is full. The DEMANDBUFFER structure and the PageFaultExceptionHandler work together to create the output buffer on demand as the FormatMessageIntoBuffer function does its work.

To make discussion easier, let's say that the FormatMessage function merely took printf-style arguments and supported only FORMAT_MESSAGE_FROM_STRING | FORMAT_MESSAGE_ALLOCATE_BUFFER.

DWORD FormatMessageFromStringPrintfAllocateBuffer(
    PWSTR *ResultBuffer,
    PCWSTR FormatString,
    ...)
{
  DWORD Result = 0;
  DWORD ResultString = NULL;
  DWORD Error;
  DEMANDBUFFER Buffer;
  va_list ap;
  va_start(ap, FormatString);
  Error = InitializeDemandBuffer(&Buffer, FORMATMESSAGE_MAXIMUM_OUTPUT);
  if (Error == ERROR_SUCCESS) {
    __try {
     SIZE_T MaxChars = Buffer.Length / sizeof(WCHAR);
     int i = _vsnwprintf((WCHAR*)Buffer.Base, MaxChars,
                         FormatString, ap);
     if (i < 0 || i >= MaxChars) Error = ERROR_MORE_DATA;
     else Result = i;
    } __except (PageFaultExceptionFilter(&Buffer,
                   GetExceptionInformation()->ExceptionRecord)) {
     // never reached - we never handle the exception
    }
  }
  if (Error == ERROR_SUCCESS) {
   // Exercise: Why don't we need to worry about integer overflow?
   DWORD BytesNeeded = sizeof(WCHAR) * (Result + 1);
   ResultString = (PWSTR)LocalAlloc(LMEM_FIXED, BytesNeeded);
   if (ResultBuffer) {
    // Exercise: Why CopyMemory and not StringCchCopy?
    CopyMemory(ResultString, Buffer.Base, BytesNeeded);
   } else Error = ERROR_NOT_ENOUGH_MEMORY;
  }
  DeleteDemandBuffer(&Buffer);
  if (Result == 0) {
    SetLastError(Error);
  }
  *ResultBuffer = ResultString;
  va_end(ap);
  return Result;
}

Let's run this function in our head to see what happens if somebody triggers the alleged buffer overflow by calling

PWSTR ResultString;
DWORD Result = FormatMessageFromStringPrintfAllocateBuffer(
                   &ResultString, L"%s", VeryLongString);

After setting up the demand buffer, we call _vsnwprintf to format the output into the demand buffer, but telling it not to go past the buffer's total length. The _vsnwprintf function parses the format string and sees that it needs to copy VeryLongString to the output buffer. Let's say that the DEMANDBUFFER was allocated at address 0x00780000 on a system with 4KB pages. At the start of the copy, the address space looks like this:

64KB

0078 0000
C 0078 1000
R 0078 2000
R 0078 3000
R 0078 4000
R 0078 5000
R 0078 6000
R 0078 7000
R 0078 8000
R 0078 9000
R 0078 A000
R 0078 B000
R 0078 C000
R 0078 D000
R 0078 E000
R 0078 F000
R 0079 0000
X

^ output pointer

"C" stands for a committed page, "R" stands for a reserved page, and "X" stands for a page that, if accessed, would be a buffer overflow. We start copying VeryLongString into the output buffer. After copying 2048 characters, we fill the first committed page; copying character 2049 raises a page fault exception.

64KB

^ output pointer

This is the point at which over-eager people observe the first-chance exception, capture the register dump above, and begin writing up their security vulnerability report, cackling with glee. (Observe that in the register dump, the address we are writing to is of the form 0x####1000.)

As with all first-chance exceptions, it goes down the exception chain. Our custom PageFaultExceptionFilter recognizes this as an access violation in a page that it is responsible for, and the page hasn't yet been committed, so it commits the page as read/write and resumes execution.

64KB

0078 0000
C 0078 1000
C 0078 2000
R 0078 3000
R 0078 4000
R 0078 5000
R 0078 6000
R 0078 7000
R 0078 8000
R 0078 9000
R 0078 A000
R 0078 B000
R 0078 C000
R 0078 D000
R 0078 E000
R 0078 F000
R 0079 0000
X

^ output pointer

Copying character 2049 now succeeds, as does the copying of characters 2050 through 4096. When we hit character 4097, the cycle repeats:

64KB

^ output pointer

Again, the first-chance exception is sent down the chain, our PageFaultExceptionFilter recognizes this as a page it is responsible for, and it commits the page and resumes execution.

64KB

0078 0000
C 0078 1000
C 0078 2000
C 0078 3000
R 0078 4000
R 0078 5000
R 0078 6000
R 0078 7000
R 0078 8000
R 0078 9000
R 0078 A000
R 0078 B000
R 0078 C000
R 0078 D000
R 0078 E000
R 0078 F000
R 0079 0000
X

^ output pointer

If you think about it, this is exactly what the memory manager does with memory that has been allocated but not yet accessed: The memory is not present, and the moment an application tries to access it, the not-present page fault is raised, the memory manager commits the page, and then execution resumes normally. It's memory-on-demand, which is one of the essential elements of virtual memory. What's going on with the DEMANDBUFFER is that we are simulating in user mode what the memory manager does in kernel mode. (The difference is that while the memory manager takes committed memory and makes it present on demand, the DEMANDBUFFER takes reserved address space and commits it on demand.)

The cycle repeats 13 more times, and then we reach another interesting part of the scenario:

64KB

0078 0000
C 0078 1000
C 0078 2000
C 0078 3000
C 0078 4000
C 0078 5000
C 0078 6000
C 0078 7000
C 0078 8000
C 0078 9000
C 0078 A000
C 0078 B000
C 0078 C000
C 0078 D000
C 0078 E000
C 0078 F000
C 0079 0000
X

output pointer ^

We are about to write 32768th character into the DEMANDBUFFER. Once that's done, the buffer will be completely full. One more byte and we will overflow the buffer. (Not even a wafer-thin byte will fit.)

Let's write that last character and cover our ears in anticipation.

64KB

output pointer ^

Oh noes! Completely full! Run for cover!

But wait. We passed a buffer size to the _vsnwprintf function, remember? We already told it never to write more than 32768 characters. As it's about to write character 32769, it realizes, "Wait a second, this would overflow the buffer I was given. I'll return a failure code instead."

The feared write of the 32769th character never takes place. We never write to the "X" page. Instead, the _vnswprintf call returns that the buffer was not large enough, which is converted into ERROR_MORE_DATA and returned to the caller.

If you follow through the entire story, you see that everything worked as it was supposed to and no overflow took place. The _vnswprintf function ran up to the brink of disaster but stopped before taking that last step. This is hardly anything surprising; it happens whenever the _vnswprintf function encounters a buffer too small to hold the output. The only difference is that along the way, we saw a few first-chance exceptions, exceptions that had nothing to do with avoiding the buffer overflow in the first place. They were just part of FormatMessage's fancy buffer management.

It so happens that in Windows Vista, the fancy buffer management technique was abandoned, and the code just allocates 64KB of memory up front and doesn't try any fancy commit-on-demand games. Computer memory has become plentiful enough that a momentary allocation of 64KB has less of an impact than it did twenty years ago, and performance measurements showed that the new "Stop trying to be so clever" technique was now about 80 times faster than the "gotta scrimp and save every last byte of memory" technique.

The change had more than just a performance effect. It also removed the first-chance exception from FormatMessage, which means that it no longer does that thing which everybody mistakes for a security vulnerability. The good news is that nobody reports this as a vulnerability in Windows Vista any more. The bad news is that people still report it as a vulnerability in Windows XP, and each time this issue comes up, somebody (possibly me) has to sit down and reverify that the previous analysis is still correct, in the specific scenario being reported, because who knows, maybe this time they really did find a problem.

What is the effect of memory-mapped file access on GetLastError()?

2012-02-09T15:00:00Z

A customer was using memory-mapped files and was looking for information as to whether access to the memory-mapped data modifies the value returned by GetLastError. A member of the kernel team replied, "No, memory-mapped I/O does not ever change the value returned by GetLastError."

That answer is simultaneously correct and wrong, a case of looking at the world through kernel-colored glasses.

While it's true that the kernel does not ever change the value returned by GetLastError, it's also the case that you might change it.

If you set up an exception handler, then your exception handler might perform operations that affect the last-error code, and those changes will be visible after the exception handler returns. (This applies to all exception handlers and filters, not just ones related to memory-mapped files.)

If you intend to return EXCEPTION_CONTINUE_EXECUTION because you handled the exception, then you probably should make sure to leave the last-error code the way you found it. Otherwise, the code that you interrupted and then resumed will have had its last-error code changed asynchronously. You just sabotaged it from above.

// Code in italics is wrong

LONG ExceptionFilter(LPEXCEPTION_POINTERS ExceptionPointers)
{
 if (IsAnExceptionICanRepair(ExceptionPointers)) {
   RepairException(ExceptionPointers);
   // fixed up error; continuing
   return EXCEPTION_CONTINUE_EXECUTION;
 }

 if (IsAnExceptionICanHandle(ExceptionPointers)) {
  // We cannot repair it, but we can handle it.
  return EXCEPTION_EXECUTE_HANDLER;
 }

 // Not our exception. Keep looking.
 return EXCEPTION_CONTINUE_SEARCH;
}

If the IsAnExceptionICanRepair function or RepairException function does anything that affects the last-error code, then when the exception filter is executed for a repairable exception, the last-error code is magically changed without the mainline code's knowledge. All the mainline code did was execute stuff normally, and somehow during a memory access or a floating point operation or some other seemingly-harmless action, the last-error code spontaneously changed!

If you are going to continue execution at the point the exception was raised, then you need to "put things back the way you found them" (except of course for the part where you repair the exception itself).

LONG ExceptionFilter(LPEXCEPTION_POINTERS ExceptionPointers)
{
 PreserveLastError preserveLastError;

 if (IsAnExceptionICanRepair(ExceptionPointers)) {
   RepairException(ExceptionPointers);
   // fixed up error; continuing
   return EXCEPTION_CONTINUE_EXECUTION;
 }

 if (IsAnExceptionICanHandle(ExceptionPointers)) {
  // We cannot repair it, but we can handle it.
  return EXCEPTION_EXECUTE_HANDLER;
 }

 // Not our exception. Keep looking.
 return EXCEPTION_CONTINUE_SEARCH;
}

Exercise: Why isn't it important to restore the last error code if you return EXCEPTION_EXECUTE_HANDLER?

Exercise: Is it important to restore the last error code if you return EXCEPTION_CONTINUE_SEARCH?

The path-searching algorithm is not a backtracking algorithm

2012-02-08T15:00:00Z

Suppose your PATH environment variable looks like this:

C:\dir1;\\server\share;C:\dir2

Suppose that you call LoadLibrary("foo.dll") intending to load the library at C:\dir2\foo.dll. If the network server is down, the LoadLibrary call will fail. Why doesn't it just skip the bad directory in the PATH and continue searching?

Suppose the LoadLibrary function skipped the bad network directory and kept searching. Suppose that the code which called LoadLibrary("foo.dll") was really after the file \\server\share\foo.dll. By taking the server down, you have tricked the LoadLibrary function into loading c:\dir2\foo.dll instead. (And maybe that was your DLL planting attack: If you can convince the system to reject all the versions on the PATH by some means, you can then get LoadLibrary to look in the current directory, which is where you put your attack version of foo.dll.)

This can manifest itself in very strange ways if the two copies of foo.dll are not identical, because the program is now running with a version of foo.dll it was not designed to use. "My program works okay during the day, but it starts returning bad data when I try to run between midnight and 3am." Reason: The server is taken down for maintenance every night, so the program ends up running with the version in c:\dir2\foo.dll, which happens to be an incompatible version of the file.

When the LoadLibrary function is unable to contact \\server\share\foo.dll, it doesn't know whether it's in the "don't worry, I wasn't expecting the file to be there anyway" case or in the "I was hoping to get that version of the file, don't substitute any bogus ones" case. So it plays it safe and assumes it's in the "don't substitute any bogus ones" and fails the call. The program can then perform whatever recovery it deems appropriate when it cannot load its precious foo.dll file.

Now consider the case where there is also a c:\dir1\foo.dll file, but it's corrupted. If you do a LoadLibrary("foo.dll"), the call will fail with the error ERROR_BAD_EXE_FORMAT because it found the C:\dir1\foo.dll file, determined that it was corrupted, and gave up. It doesn't continue searching the path for a better version. The path-searching algorithm is not a backtracking algorithm. Once a file is found, the algorithm commits to trying to load that file (a "cut" in logic programming parlance), and if it fails, it doesn't backtrack and return to a previous state to try something else.

Discussion: Why does the LoadLibrary search algorithm continue if an invalid directory or drive letter is put on the PATH?

Vaguely related chatter: No backtracking, Part One

Microspeak: fit

2012-02-07T15:00:00Z

In Microspeak, fit is a predicate noun which is never used on its own but always comes with a modifying adjective. For something to be a good fit is for something to be appropriate or suitable for a particular situation. The opposite of a good fit is not a bad fit, because that's pejorative. Rather, something that is not a good fit is referred to as a poor fit.

The purpose of a previewer plug-in is to allow users to view the media without opening it. An image editing tool would not be a good fit for the previewing feature. (Alternatively, "would be a poor fit for the previewing feature.")

To be a good fit with a particular group is to mesh well with that group's existing practices and conventions.

The Datacenter Edition of the product is a poor fit for most small businesses.

The group in question need not consist of people.

The results are obtained incrementally, which makes it a good fit for IQueryable<T> and LINQ.

Microsoft Human Resources loves to apply the concept of "fit" to people fitting into a job position.

The story of the mysterious WINA20.386 file

2012-02-06T15:00:00Z

matushorvath was curious about the WINA20.386 file that came with some versions of MS-DOS.

The WINA20.386 file predates my involvement, but I was able to find some information on the Internet that explained what it was for. And it's right there in KB article Q68655: Windows 3.0 Enhanced Mode Requires WINA20.386:

Windows 3.0 Enhanced Mode Requires WINA20.386

Windows 3.0 enhanced mode uses a modular architecture based on what are called virtual device drivers, or VxDs. VxDs allow pieces of Windows to be replaced to add additional functionality. WINA20.386 is such a VxD. (VxDs could be called "structured" patches for Windows.)

Windows 3.0 enhanced mode considers the state of the A20 line to be the same in all MS-DOS virtual machines (VMs). When MS-DOS is loaded in the high memory area (HMA), this can cause the machine to stop responding (hang) because of MS-DOS controlling the A20 line. If one VM is running inside the MS-DOS kernel (in the HMA) and Windows task switches to another VM in which MS-DOS turns off A20, the machine hangs when switching back to the VM that is currently attempting to execute code in the HMA.

WINA20.386 changes the way Windows 3.0 enhanced mode handles the A20 line so that Windows treats the A20 status as local to each VM, instead of global to all VMs. This corrects the problem.

(At the time I wrote this, a certain popular Web search engine kicks up as the top hit for the exact phrase "Windows 3.0 Enhanced Mode Requires WINA20.386" a spam site that copies KB articles in order to drive traffic. Meanwhile, the actual KB article doesn't show up in the search results. Fortunately, Bing got it right.)

That explanation is clearly written for a technical audience with deep knowledge of MS-DOS, Windows, and the High Memory Area. matushorvath suggested that "a more detailed explanation could be interesting." I don't know if it's interesting; to me, it's actually quite boring. But here goes.

The A20 line is a signal on the address bus that specifies the contents of bit 20 of the linear address of memory being accessed. If you aren't familiar with the significance of the A20 line, this Wikipedia article provides the necessary background.

The High Memory Area is a 64KB-sized block of memory (really, 64KB minus 16 bytes) that becomes accessible when the CPU is in 8086 mode but the A20 line is enabled. To free up conventional memory, large portions of MS-DOS relocate themselves into the HMA. When a program calls into MS-DOS, it really calls into a stub which enables the A20 line, calls the real function in the HMA, and then disables the A20 line before returning to the program. (The value of the HMA was discovered by my colleague who also discovered the fastest way to get out of virtual-8086 mode.)

The issue is that by default, Windows treats all MS-DOS device drivers and MS-DOS itself as global. A change in one virtual machine affects all virtual machines. This is done for compatibility reasons; after all, those old 16-bit device drivers assume that they are running on a single-tasking operating system. If you were to run a separate copy of each driver in each virtual machine, each copy would try to talk to the same physical device, and bad things would happen because each copy assumed it was the only code that communicated with that device.

Suppose MS-DOS device drivers were treated as local to each virtual machine. Suppose you had a device driver that controlled a traffic signal, and as we all know, one of the cardinal rules of traffic signals is that you never show green in both directions. The device driver has two variables: NorthSouthColor and EastWestColor, and initially both are set to Red. The copy of the device driver running in the first virtual machine decides to let traffic flow in the north/south direction, and it executes code like this:

if (EastWestColor != Red) {
 SetEastWestColor(Red);
}
SetNorthSouthColor(Green);

Since both variables are initially set to Red, this code sets the north/south lights to green.

Meanwhile, the copy of the device driver in the second virtual machine wants to let traffic flow in the east/west direction:

if (NorthSouthColor != Red) {
 SetNorthSouthColor(Red);
}
SetEastWestColor(Green);

Since we have a separate copy of the device driver in each virtual machine, the changes made in the first virtual machine do not affect the values in the second virtual machine. The second virtual machine sees that both variables are set to Red, so it merely sets the east/west color to green.

On the other hand, both of these device drivers are unwittingly controlling the same physical traffic light, and it just got told to set the lights in both directions to Green.

Oops.

Okay, so Windows defaults drivers to global. That way, you don't run into the double-bookkeeping problem. But this causes problems for the code which manages the A20 line:

Consider a system with two virtual machines. The first one calls into MS-DOS. The MS-DOS dispatcher enables the A20 line and calls the real function, but before the function returns, the virtual machine gets pre-empted. The second virtual machine now runs, and it too calls into MS-DOS. The MS-DOS dispatcher in the second virtual machine enables the A20 line and calls into the real function, and after the function returns, the second virtual machine disables the A20 line and returns to its caller. The second virtual machine now gets pre-empted, and the first virtual machine resumes execution. Oops: It tries to resume execution in the HMA, but the HMA is no longer there because the second virtual machine disabled the A20 line!

The WINA20.386 driver teaches Windows that the state of the A20 should be treated as a per-virtual-machine state rather than a global state. With this new information, the above scenario does not run into a problem because the changes to the A20 line made by one virtual machine have no effect on the A20 line in another virtual machine.

matushorvath goes on to add, "I would be very interested in how Windows 3.0 found and loaded this file. It seems to me there must have been some magic happening, e.g. DOS somehow forcing the driver to be loaded by Windows."

Yup, that's what happened, and there's nothing secret about it. When Windows starts up, it broadcasts an interrupt. TSRs and device drivers can listen for this interrupt and respond by specifying that Windows should load a custom driver or request that certain ranges of data should be treated as per-virtual-machine state rather than global state (known to the Windows virtual machine manager as instance data). MS-DOS itself listens for this interrupt, and when Windows sends out the "Does anybody have any special requests?" broadcast, MS-DOS responds, "Yeah, please load this WINA20.386 driver."

So there you have it, the story of WINA20.386. Interesting or boring?

The compatibility constraints of error codes, episode 2

2012-02-03T15:00:00Z

A customer reported an incompatibility in Windows 7: If A: is a floppy drive and they call LoadLibrary("A:\\foo.dll") and there is no disk in the drive, the LoadLibrary call fails with the error ERROR_NOT_READY. Previous versions of Windows failed with the error ERROR_MOD_NOT_FOUND.

Both error codes are reasonable responses to the situation. "The module couldn't be found because the drive is not ready." Programs should treat a failed LoadLibrary as a failed library load and shouldn't be sensitive to the precise reason for the error. (They can display a more specific error to the user based on the error code, but overall program logic shouldn't depend on the error code.)

Fortunately, the customer discovered this discrepancy during their pre-release testing and were able to accommodate this change in their program before ever releasing it. A sigh of relief from the application compatibility team.

Episode 1.

When you are looking for more information, it helps to say what you need the information for

2012-02-02T15:00:00Z

It's often the case that when a question from a customer gets filtered through a customer liaison, some context gets lost. (I'm giving the customer the benefit of the doubt here and assuming that it's the customer liaison that removed the context rather than the customer who never provided it.) Consider the following request:

We would like to know more information about the method the shell uses to resolve shortcuts.

This is kind of a vague question. It's like asking "I'd like to know more about the anti-lock braking system in my car." There are any number of pieces of information that could be provided about the anti-lock braking system.

"It requires a Class C data bus."
"The tire position sensors are on the wheel-axis."
"It is connected to the brakes."
"It is shiny."

When we ask the customer, "Could you be more specific what type of information you are looking for?" the response is sometimes

We want to know everything.

This is not a helpful clarification. Do they want to start with Maxwell's Equations and build up from there?

As it happened, in the case of wanting more information about the method the shell uses to resolve shortcuts, they just wanted to know how to disable the search-based algorithm.

This sort of "ask for everything and figure it out later" phenomenon is quite common. I remember another customer who wanted to know "everything" about changing network passwords, and they wouldn't be any more specific than that, so we said, "Well, you can start with these documents, perhaps paying particular attention to this one, but if they tell us what they are going to be doing with the information, we can help steer them to the specific parts that will be most useful to them."

As it turned out, all the customer really wanted to know was "When users change their password, is the new password encrypted on the wire?"

Third example, and then I'll stop. Another customer wanted to know everything about how Explorer takes information from the file system and displays it in an Explorer window. After asking a series of questions, we eventually figured out that they in fact didn't want or need a walkthrough of the entire code path that puts results in the Explorer window. The customer simply wanted to know why two specific folders show up in their Explorer window with names that didn't match the file system name.

When you ask for more information, explain what you need the information for, or at least be more specific what kind of "more information" you need. That way, you save everybody lots of time. The people answering your question don't waste their time gathering information you don't need (and gathering that information can be quite time-consuming), and you don't waste your time sifting through all the information you don't want.

You might say that these people are employing the for-if anti-pattern:

foreach (document d in GetAllPossibleDocumentation())
{
 if (d.Topic == "password encryption on the wire") return d;
}

Things I've written that have amused other people, Episode 9

2012-02-01T15:00:00Z

A customer liaison reported that their customer wants to be able to access their machine without needing a password. They just want to be able to net use * \\machine\share and be able to access the files right away. I guess because passwords are confusing, easy to forget, and just get in the way. Anyway, the customer discovered that they could do so on Windows XP by going to the folder they want to share, going to the Sharing tab, then clicking on the If you understand the security risks but want to share files without running the wizard link,

and then on the Enable File Sharing dialog, clicking Just enable file sharing.

What the customer wanted to know was if there was a way they could automate this process.

My response to the customer liaison went like this:

Your customer has chosen to ignore not one but two security warnings. Furthermore, since they are looking for an automated way of doing this, it sounds like they intend on deploying this "feature" to all the computers in their organization. Maybe they just enjoy being part of a botnet? Your customer is basically saying "I wish my computer to have no network security." They should at least restrict access to authenticated users. But if they if they insist on having their corporate network turned into a spam farm, they can enable the Guest account and say that it can "Access this computer from the network." Congratulations, your computers will soon be filled with malware and porn.

That last sentence made it into some people's quotes file.

The Freudian typo that will not die: Enchanced video quality

2012-01-31T15:00:00Z

While ~~wasting time~~ doing valuable background research on my computer, I received the following suggestion:

For enchanced video quality, click here.

It's good to know that the typo that I first encountered in 1993 is still alive and kicking.

(And even though it's not important to the story, people will demand some sort of follow-up, so here it is: I submitted feedback to the vendor, who said that it was a known issue fixed in the next update.)

Why does it take Task Manager longer to appear when you start it from the Ctrl+Alt+Del dialog?

2012-01-30T15:00:00Z

Amit was curious why it takes longer for Task Manager to appear when you start it from the Ctrl+Alt+Del dialog compared to launching it from the taskbar.

Well, you can see the reason right there on the screen: You're launching it the long way around.

If you launch Task Manager from the taskbar, Explorer just launches taskmgr.exe via the usual CreateProcess mechanism, and Task Manager launches under the same credentials on the same desktop.

On the other hand, when you use the secure attention sequence, the winlogon program receives the notification, switches to the secure desktop, and displays the Ctrl+Alt+Del dialog. When you select Task Manager from that dialog, it then has to launch taskmgr.exe, but it can't use the normal CreateProcess because it's on the wrong desktop and it's running under the wrong security context. (Because winlogon runs as SYSTEM, as Task Manager will tell you.)

Clearly, in order to get Task Manager running on your desktop with your credentials, winlogon needs to change its security context, change desktops, and then launch taskmgr.exe. The desktop switch is probably the slowest part, since it involves the video driver, and video drivers are not known for their blazingly fast mode changes.

It's like asking why an international package takes longer to deliver than a domestic one. Because it's starting from further away, and it also has to go through customs.

Does mapping the same shared memory two times in a process lead to double the address space usage?

2012-01-27T15:00:00Z

A customer designed a system which uses shared memory. Specifically, for each database file, they create a corresponding shared memory block of, say, 200MB. Multiple clients which connect to the same database file use the same shared memory block. Naturally, if two processes each access the same database file, each process will map the shared memory block into their respective address space. The question arose regarding what happens if one process connects to the same database file twice. Will the two calls to MapViewOfFile share the same address space, or will each one allocate a separate chunk of address space?

Win32 makes no guarantees what will happen. All that you can be sure of is that the memory will be mapped into your address space, and you will get a pointer to it, and when you're done, you call UnmapViewOfFile. Whether the two calls return the same pointer is unspecified.

In fact, Windows 95 returned the same pointer, whereas Windows NT returns a different pointer. We saw this earlier when we intentionally mapped the same shared memory block multiple times, and observed somebody actually taking a dependency on this behavior in order to effect the strangest way of detecting Windows NT. Don't take a dependency on this behavior; who knows, maybe a future version of Windows NT will consolidate multiple mappings in order to conserve address space.

If you want force this consolidation behavior, you'll have to roll it yourself, say with a lookup table of active mappings and a reference count.

The 2012/2013 Seattle Symphony subscription season at a glance

2012-01-26T15:00:01Z

Every year, I put together a little pocket guide to the Seattle Symphony subscription season for my symphony friends to help them decide which ticket package they want. As before, you might find it helpful, you might not, but either way, you're going to have to suffer through it. Here's the at-a-glance season guide for the 2012/2013 season. (Full brochure. Seattle Times coverage.)

Week	Program	Comments	21	13	7A 7B	7C 7D	7E 7F	7G	4A	BS	SU	WG
Week	Program	Comments	21	13	7A 7B	7C 7D	7E 7F	7G	4A	BS	SU	WG
09/20 2012	Berlioz: Roman Carnival Overture Martinů: Symphony #6 Fantaises symphoniques Debussy: Nocturnes: Nuages and Fêtes Respighi: Pines of Rome	Good Nervous Excellent Excellent
10/04 2012	Mussorgsky: Night on Bald Mountain Tchaikovsky: Rococo Variations Sibelius: Symphony #1	Awesome Good Excellent
10/18 2012	Beethoven: Coriolan Overture Mozart: Sinfonia Concertante K297b Fujikura: Mina† Haydn: Symphony #103 Drum Roll	Awesome Excellent Wildcard Awesome
10/26	Sonic Evolution	Wildcard
11/01 2012	Tchaikovsky: The Snow Maiden Suite Tchaikovsky: Violin Concerto Prokofiev: Symphony #6	Good Awesome Nervous
11/08 2012	Adams: Harmonielehre Beethoven: Piano Concerto #5	Polarizing Awesome
11/15 2012	Brahms: Piano Concerto #2 Dutilleux: The Shadow of Time R. Strauss: Till Eulenspiegel	Awesome Nervous Good
11/29 2012	Berg: Violin Concerto Mahler: Symphony #4	Polarizing Polarizing
01/10 2013	Stravinsky: Pulcinella Suite Mendelssohn: Piano Concerto #1 Mozart: Symphony #39	Excellent Excellent Awesome
01/31	Messiaen: Turangalîla Symphony	Polarizing
02/07 2013	Rossini: William Tell Overture Schumann: Piano Concerto Brahms: Symphony #4	Awesome Excellent Awesome
02/14 2013	(Rossini: William Tell Overture) Faure: Pelléas et Mélisande Suite Mozart: Piano Concerto #21 Ravel: Shéherazade Szymanowski: Symphony #4	Awesome Good Awesome Nervous Nervous
03/14 2013	Tippett: The Midsummer Marriage: Ritual Dances Bruch: Violin Concerto #1 Elgar: Enigma Variations	Nervous Awesome Excellent
03/21 2013	Mozart: Don Giovanni Overture Britten: Cello Symphony Beethoven: Symphony #5	Awesome Nervous Awesome
03/28 2013	Liadov: The Enchanted Lake Kancheli: Styx Rimsky-Korsakov: Scheherezade	Okay Nervous Awesome
04/13 2013	Mozart: Piano Concerto #9 Bruckner: Symphony #4 Romantic	Awesome Excellent
04/18 2013	Antheil: A Jazz Symphony Gruber: Percussion Concerto Rough Music Bernstein: On the Waterfront Suite Stravinsky: The Firebird Suite	Excellent Okay Good Excellent
04/25 2013	Sibelius: Karelia Overture Sibelius: Violin Concerto Zavaro: New Work† Beethoven: Symphony #7	Good Awesome Wildcard Awesome
05/30 2013	Smetana: Wallenstein's Camp Beethoven: Violin Concerto Dvořák: Symphony #6	Excellent Awesome Awesome
06/13	Britten: War Requiem	Nervous
06/20 2013	Shostakovich: Violin Concerto #1 John Luther Adams: Become Ocean†	Nervous Okay
06/27 2013	Saint-Saëns: Organ Symphony Wagner: Tristan: Prelude and Liebestod Wagner: Tannhäuser Overture and Venusberg Music	Awesome Okay Excellent

Legend:

21	Masterworks 21-concert series (Choice of Thursdays or Saturdays)
13	Masterworks 13-concert series (Choice of Thursdays or Saturdays)
7A	Masterworks 7-concert series A (Thursdays)
7B	Masterworks 7-concert series B (Saturdays)
7C	Masterworks 7-concert series C (Thursdays)
7D	Masterworks 7-concert series D (Saturdays)
7E	Masterworks 7-concert series E (Thursdays)
7F	Masterworks 7-concert series F (Saturdays)
7G	Masterworks 7-concert series G (Sunday afternoons)
4A	Masterworks 4-concert series A (Friday afternoons)
BS	Beyond the Score multimedia lecture-concert (Sunday afternoons)
SU	Symphony Untuxed (Fridays)
WG	WolfGang (Various evenings)
†	Premiere

For those not familiar with the Seattle Symphony ticket package line-ups: Most of the ticket packages are named Masterworks nX where n is the number is the number of concerts in the package, and the letter indicates which variation. Ticket packages have been combined if they are identical save for the day of the week. For example, 7C and 7D are the same concerts; the only difference is that 7C is for Thursday nights, while 7D is for Saturday nights. The Beyond the Score concerts focus on only one of the pieces. The WolfGang series is available only to members of the WolfGang club.

This chart doesn't include "one-off" concert series such as the Mainly Mozart or Distinguished Artists series. A "one-off" series is a concert series which shares no concerts with any other series.

Changes from last season:

The Rush Hour series was redesigned and became the Symphony Untuxed series.
Symphony Specials was dropped.
The Baroque & Wine Series and Mainly Mozart concerts still exist, but I dropped them from the table since they are now one-off series.
WolfGang already existed but I added it to the list now that there's room for it.

The comments column very crudely categorizes the works to assist my less-classically-aware friends. This is, of course, a highly subjective rating system, but I tried to view each piece from the ears of my symphony friends. Thus, I rated downward pieces that I personally like but which others might not and rated up pieces that I may not find musically satisfying but which nevertheless tend to be crowd-pleasers.

These predictions have, of course, proven wrong in the past.

Here's what the comments mean. Note that they do not indicate whether the piece is significant in a musicological sense; they're just my guess as to whether my friends are going to like it. (For example, I know that my friends don't like minimalism, and I suspect they don't like serialism, so I rated the Adams and Berg down even though I think they're quite good. They also don't like vocal pieces. On the other hand, it turns out that I overcame my Bruckner jinx, so I can at least give the Bruckner a positive score this time. Let's just hope it's not the Hass edition.)

Awesome: Guaranteed crowd-pleaser.
Excellent: You will definitely like this piece.
Good: You will probably like this piece.
Okay: You may like this piece.
Nervous: I have a bad feeling about this one.
Polarizing: Some people will love it; others will hate it.
Wildcard: I have no idea what will happen.

In many cases, I am not familiar with the piece and am basing my evaluation on what I know about the composer (or am just guessing).

Why doesn't the Windows 7 Start menu have a pushpin for pinning items?

2012-01-26T15:00:00Z

You may have noticed a minor inconsistency between pinning a program to the Start menu and pinning a destination to a program's Jump List. Although pinned items appear at the top of the respective lists, and both the Start menu and Jump List let you right-click an item and select Pin/Unpin, the Jump List also lets you pin and unpin an item by clicking on the pushpin. Why doesn't the Start menu have a pushpin in addition to the right-click menu?

For a time, items on the Start menu did have a pushpin, just like items on Jump Lists. The design had a few problems, however. Start menu items can also have a triangle indicating the presence of a flyout menu, and the presence of two indicators next to an item made the interface look awkward and too busy. And what do you do if an item has only one indicator? Do you right-justify all the indicators? Or do you place the indicators in columns and reserve blank space for the missing ones?

Internet Explorer	¤ ▶
Command Prompt	¤
Notepad	▶
Calculator

Internet Explorer	¤	▶
Command Prompt	¤
Notepad		▶
Calculator

Both look ugly for different reasons. The right-justify-everything version looks ugly because the pushpin appears to keep moving around. The blank-space-if-no-flyout version looks ugly because you have a pushpin hovering in the middle of nowhere. (Imagine trying to click on one of these things: You just have to "know" that the magic click spot for pinning an item is 20 pixels to the left of the far right edge.)

But the real death blow to showing a pushpin for pinning items to the Start menu was the usability testing. Users had trouble figuring out where to click to pin an item or to open the Jump List and frequently got the opposite of what they wanted. Since opening the Jump List is by far the more common operation, it won the battle of the prominent UI affordance, and the option for pinning and unpinning was left to a context menu.

Which, as it happens, is where the pin/unpin option started in the first place.

How do I disable the fault-tolerant heap?

2012-01-25T15:00:00Z

A while back, I linked to a talk by Silviu Calinoiu on the fault-tolerant heap. But what if you don't want the fault-tolerant heap? For example, during program development, you probably want to disable the fault-tolerant heap for your program: If the program is crashing, then it should crash so you can debug it!

Method 1 is to disable the fault-tolerant heap globally. While this prevents the fault-tolerant heap from auto-activating in the future, it does not go back and undo activations that were enabled in the past. In other words, you have to remember to do this before your application crashes for the first time.

Therefore, you probably want to combine Method 1 with Method 2 on the same page, where it gives instructions on how to reset the list of applications for which the fault-tolerant heap is enabled.

Mario Raccagni provides a third way of disabling the fault tolerant heap, this time for one specific process instead of globally. His explanation is in Italian, so you get to exercise your translation skills.

tl;dr version: Go to the HKEY_LOCAL_MACHINE and HKEY_CURRENT_USER versions of Software\Microsoft\Windows NT\CurrentVersion\AppCompatFlags\Layers\your_application.exe and delete the FaultTolerantHeap entry.

A single-handed effort to keep the memory of $2 bills alive

2012-01-24T15:00:00Z

As I noted when I told the story of the computer programmer who dabbled in making change that my colleague had a lot of money-related quirks.

For some reason my colleague felt the $2 bill deserved more attention. Every so often, he would go to the bank and buy $100 in $2 bills, then reintroduce the bills into circulation and enjoy people's reactions to them. (Most cashiers looked at it and recognized that it was legal tender, but couldn't find a good place to put it in the till. It usually got tossed under the drawer with all the checks.)

It was a regular occurrence that the bank didn't have that many $2 bills on hand, but they managed to find them and let him know when he could come pick them up.

One time, the bank called him back. "Hi, we asked all our branches in the entire county, but all together we can't find enough $2 bills. If you want, we can place an order with the Federal Reserve. The catch is, though, that the minimum order is $2000."

"Sure, go ahead and place the order."

Some time later, he went in to pick up his huge stack of $2 bills.

My colleague now found himself in a situation where something fun turned into an ordeal, like a smoker who is forced to smoke an entire pack of cigarettes at one sitting. Or in this case, more like 1000 cigarettes.

At the end of group meals at a restaurant, after everybody had calculated their share and put their money in the bill holder (this being the days when people actually paid cash for things), he would raid the bill holder for change, taking out all the notes greater than $2 and replacing them with the appropriate number of $2 bills. As a result, when the servers came to collect the bill holders, they found them stuffed with $1 and $2 bills (mostly $2).

Too bad he didn't make a pad out of them.

Bonus reading: $1 billion that nobody wants. Follow-up.

Can OANOCACHE be used for non-debug purposes?

2012-01-23T15:00:00Z

Friday asks whether OANOCACHE can be used for non-debug purposes, say to improve stability and/or speed.

You can try, but it's not recommended. For one thing, it probably damages stability, because there are many applications out there which unwittingly rely on the BSTR cache to protect them from heap corruption bugs. The Windows team has for years wanted to tweak the BSTR cache (even going so far as getting rid of it entirely), but the compatibility issues always return and quash any attempts at radical restructuring.

Identifying applications that rely on the BSTR cache and deploying an appropriate compatibility shim would be one thing. It's applications which support third party plug-ins that are the sticky point. You say, "Okay, this application is fine, we'll use the new BSTR cache behavior" and then it loads a plug-in DLL that requires the old BSTR cache behavior. Now you're stuck since you don't have a time machine.

How do FILE_FLAG_SEQUENTIAL_SCAN and FILE_FLAG_RANDOM_ACCESS affect how the operating system treats my file?

2012-01-20T15:00:00Z

There are two flags you can pass to the CreateFile function to provide hints regarding your program's file access pattern. What happens if you pass either of them, or neither?

Note that the following description is not contractual. It's just an explanation of the current heuristics (where "current" means "Windows 7"). These heuristics have changed at each version of Windows, so consider this information as a tip to help you choose an appropriate access pattern flag in your program, not a guarantee that the cache manager will behave in a specific way if you do a specific thing.

If you pass the FILE_FLAG_SEQUENTIAL_SCAN flag, then the cache manager alters its behavior in two ways: First, the amount of prefetch is doubled compared to what it would have been if you hadn't passed the flag. Second, the cache manager marks as available for re-use those cache pages which lie entirely behind the current file pointer (assuming there are no other applications using the file). After all, by saying that you are accessing the file sequentially, you're promising that the file pointer will always move forward.

At the opposite extreme is FILE_FLAG_RANDOM_ACCESS. In the random access case, the cache manager performs no prefetching, and it does not aggressively evict pages that lie behind the file pointer. Those pages (as well as the pages that lie ahead of the file pointer which you already read from or wrote to) will age out of the cache according to the usual most-recently-used policy, which means that heavy random reads against a file will not pollute the cache (the new pages will replace the old ones).

In between is the case where you pass neither flag.

If you pass neither flag, then the cache manager tries to detect your program's file access pattern. This is where things get weird.

If you issue a read that begins where the previous read left off, then the cache manager performs some prefetching, but not as much as if you had passed FILE_FLAG_SEQUENTIAL_SCAN. If sequential access is detected, then pages behind the file pointer are also evicted from the cache. If you issue around six reads in a row, each of which begins where the previous one left off, then the cache manager switches to FILE_FLAG_SEQUENTIAL_SCAN behavior for your file, but once you issue a read that no longer begins where the previous read left off, the cache manager revokes your temporary FILE_FLAG_SEQUENTIAL_SCAN status.

If your reads are not sequential, but they still follow a pattern where the file offset changes by the same amount between each operation (for example, you seek to position 100,000 and read some data, then seek to position 150,000 and read some data, then seek to position 200,000 and read some data), then the cache manager will use that pattern to predict the next read. In the above example, the cache manager will predict that your next read will begin at position 250,000. (This prediction works for decreasing offsets, too!) As with auto-detected sequential scans, the prediction stops as soon as you break the pattern.

Since people like charts, here's a summary of the above in tabular form:

Access pattern	Prefetch	Evict-behind
Explicit random	No	No
Explicit sequential	Yes (2×)	Yes
Autodetected sequential	Yes	Yes
Autodetected very sequential	Yes (2×)	Yes
Autodetected linear	Yes	?
None	No	?

There are some question marks in the above table where I'm not sure exactly what the answer is.

Note: These cache hints apply only if you use ReadFile (or moral equivalents). Memory-mapped file access does not go through the cache manager, and consequently these cache hints have no effect.

Why do Microsoft customer records use the abbreviation "cx" for customer?

2012-01-19T15:00:00Z

As is common in many industries, Microsoft customer service records employ abbreviations for many commonly-used words. In the travel industry, for example, pax is used as an abbreviation for passenger. The term appears to have spread to the hotel industry, even though people who stay at a hotel aren't technically passengers. (Well, unless you think that with the outrageous prices charged by the hotels, the people are being taken for a ride.)

For a time, the standard abbreviation for customer in Microsoft's customer service records was cu. This changed, however, when it was pointed out to the people in charge of such things that cu is a swear word in Portuguese. The standard abbreviation was therefore changed to cx.

If you're reading through old customer records and you know Portuguese and you see the word cu, please understand that we are not calling the customer a rude name.

The person who introduced me to this abbreviation added, "I just spell out the word. It's not that much more work, and it's a lot easier to read."

Some years ago, I was asked to review a technical book, and one of the items of feedback I returned was that the comments in the code fragments were full of mysterious abbreviations. "Sgnl evt before lv cs." I suggested that the words be spelled out or, if you really want to use abbreviations, at least have somewhere in the text where the abbreviations are explained.

If I had wanted to demonstrate the social skills of a thermonuclear device, my feedback might have read "unls wrtg pzl bk, avd unxplnd n unnec abbvs."

Don't try to allocate memory until there is only x% free

2012-01-18T15:00:00Z

I have an ongoing conflict with my in-laws. Their concept of the correct amount of food to have in the refrigerator is "more than will comfortably fit." Whenever they come to visit (which is quite often), they make sure to bring enough food so that my refrigerator bursts at the seams, with vegetables and eggs and other foodstuffs crammed into every available nook and cranny. If I'm lucky, the amount of food manages to get down to "only slightly overfull" before their next visit. And the problem isn't restricted to the refrigerator. I once cleared out some space in the garage, only to find that they decided to use that space to store more food. (Who knows, maybe one day I will return from an errand to find that my parking space has been filled with still more food while I was gone.)

Occasionally, a customer will ask for a way to design their program so it continues consuming RAM until there is only x% free. The idea is that their program should use RAM aggressively, while still leaving enough RAM available (x%) for other use. Unless you are designing a system where you are the only program running on the computer, this is a bad idea.

Consider what happens if two programs try to be "good programs" and leave x% of RAM available for other purposes. Let's call the programs Program 10 (which wants to keep 10% of the RAM free) Program 20 (which wants to keep 20% of the RAM free). For simplicity, let's suppose that they are the only two programs on the system.

Initially, the computer is not under memory pressure, so both programs can allocate all the memory they want without any hassle. But as time passes, the amount of free memory slowly decreases.

Program 10 (20%)

Free (60%)

Program 20 (20%)

Program 10 (30%)

Free (40%)

Program 20 (30%)

Program 10 (40%)

Free (20%)

Program 20 (40%)

And then we hit a critical point: The amount of free memory drops below 20%.

Program 10 (41%)

Free (18%)

Program 20 (41%)

At this point, Program 20 backs off in order to restore the amount of free memory back to 20%.

Program 10 (41%)

Free (20%)

Program 20 (39%)

Now, each time Program 10 and Program 20 think about allocating more memory, Program 20 will say "Nope, I can't do that because it would send the amount of free memory below 20%." On the other hand, Program 10 will happily allocate some more memory since it sees that there's a whole 10% it can allocate before it needs to stop. And as soon as Program 10 allocates that memory, Program 20 will free some memory to bring the amount of free memory back up to 20%.

Program 10 (42%)

Free (19%)

Program 20 (39%)

Program 10 (42%)

Free (20%)

Program 20 (38%)

Program 10 (43%)

Free (19%)

Program 20 (38%)

Program 10 (43%)

Free (20%)

Program 20 (37%)

Program 10 (44%)

Free (19%)

Program 20 (37%)

Program 10 (44%)

Free (20%)

Program 20 (36%)

I think you see where this is going. Each time Program 10 allocates a little more memory, Program 20 frees the same amount of memory in order to get the total free memory back up to 20%. Eventually, we reach a situation like this:

Program 10 (75%)

Free (20%)

P20 (5%)

Program 20 is now curled up in the corner of the computer in a fetal position. Program 10 meanwhile continues allocating memory, and Program 20, having shrunk as much as it can, is forced to just sit there and whimper.

Program 10 (76%)

Free (19%)

P20 (5%)

Program 10 (77%)

Free (18%)

P20 (5%)

Program 10 (78%)

Free (17%)

P20 (5%)

Program 10 (79%)

Free (16%)

P20 (5%)

Program 10 (80%)

Free (15%)

P20 (5%)

Program 10 (81%)

Free (14%)

P20 (5%)

Program 10 (82%)

Free (13%)

P20 (5%)

Program 10 (83%)

Free (12%)

P20 (5%)

Program 10 (84%)

Free (11%)

P20 (5%)

Program 10 (85%)

Free (10%)

P20 (5%)

Finally, Program 10 stops allocating memory since it has reached its own personal limit of not allocating the last 10% of the computer's RAM. But it's too little too late. Program 20 has already been forced into the corner, thrashing its brains out trying to survive on only 5% of the computer's memory.

It's sort of like when people from two different cultures with different concepts of personal space have a face-to-face conversation. The person from the not-so-close culture will try to back away in order to preserve the necessary distance, while the person from the closer-is-better culture will move forward in order to close the gap. Eventually, the person from the not-so-close culture will end up with his back against the wall anxiously looking for an escape route.

Microspeak: Walls and ladders

2012-01-17T15:00:00Z

Reader laonianren wanted to know more about this game Walls and Ladders.

"Walls and Ladders" is not a game. It's just a metaphor for a conflict in which one side wants to perform some action and the other side wants to prevent it. The defending side builds a wall, and the attacking side builds a taller ladder. In response, the defending side builds a taller wall, and the attacking side builds an even taller ladder. The result of this conflict is that the defending side constructs an ever-more-elaborate wall and the attacking side constructs a more-and-more complex ladder [link possible NSFW], both sides expending ridiculous amounts of resources and ultimately ending up back where they started.

There is a closely-related metaphor known as an arms race. In an arms race, each participant wants to be the most X, for some property X. An arms race tends to be all-attack, whereas wall-and-ladders tends to have one side attacking and the other defending.

Since many conflicts can be phrased either as an attack-attack scenario or an attack-defend scenario (some defenses may include counter-attacks), I tend to get the two confused. Notice, for example, that my arms race article contains mostly walls-and-ladders scenarios; for example, a case where one side wants to terminate a process and another wants to prevent it from being terminated. On the other hand, my wall and ladders example was really more of an arms race, with both sides wanting to take control of the screen.

Depending on which group you work with at Microsoft, you may find a preference for walls and ladders over arms race, probably due to the same sensitivity to military terms that led to the War Room being renamed Ship Room. (I seem to recall that there was a lawsuit that among other things alleged that the fact that a Microsoft project called its daily meeting room the War Room was proof of Microsoft's evil essence.)

Cultural arbitrage: The food-related sucker bet

2012-01-16T15:00:01Z

While I was at a group dinner at a Chinese restaurant, a whole fish was brought to our table. One of the other people at the table told a story of another time a whole fish was brought to the table.

He attended the wedding rehearsal dinner of a family member. The bride is Chinese, but the groom is not. (Or maybe it was the other way around. Doesn't matter to the story.) The dinner was banquet-style at a Chinese restaurant, and one of the many courses was a whole fish.

Two of the non-Chinese attendees marveled at the presence of an entire fish right there in front of them, head, tail, fins, and all. I guess they had up until then only been served fish that had already been filleted, or at least had the head cut off. One of them nudged my acquaintance and said, "We'll give you $500 if you eat the eyeball."

These guys inadvertently created their own sucker bet.

For you see, eating the eyeball is common in many parts of Asia. In fact, whenever their family has fish, my nieces fight over who gets the honor of eating the eyeballs!

I don't know whether my acquaintance cheerfully accepted the bet or whether he explained that their bet was a poor choice to offer a Chinese person.

What food-related sucker bets exist in your culture? (I'm not talking about foods like chicken feet or tongue, which are clearly prepared and served to be eaten. I'm talking about things that an uninitiated person might consider to be a garnish or an inedible by-product, like shrimp heads.)

Update: I remind you that the question is not asking for foods which are served as dishes on their own.

Why was there a font just for drawing symbols on buttons?

2012-01-16T15:00:00Z

Henke37 wonders why the Marlett font was introduced. Why use a font for drawing symbols on window buttons?

Using a font was a convenient way to have scalable graphics.

It's not like Windows could've used VML or SVG since they hadn't been invented yet. EMFs would have been overkill as well. Fonts were very convenient because the technology to render scalable fonts already existed and was well-established. It's always good to build on something that has been proven, and TrueType scalable font technology proved itself very nicely in Windows 3.1. TrueType has the added benefit of supporting hinting, allowing tweaks to the glyph outlines to be made for particular pixel sizes. (A feature not available in most vector drawing languages, but also a feature very important when rendering at small font sizes.)

Keys duplicated from photo: Delayed reaction

2012-01-13T15:00:01Z

There was a report some time ago that researchers have developed a way to duplicate keys given only a photograph. When I read this story, I was reminded of an incident that occurred to a colleague of mine.

He accidentally locked his keys in his car and called a locksmith. Frustratingly, the keys were sitting right there on the driver's seat. The locksmith arrived and assessed the situation. "Well, since you already paid for me to come all the way out here, how would you like a spare key?"

"Huh? What do you mean?"

The locksmith looked at the key on the driver's seat, studied it intently for a few seconds, then returned to his truck. A short while later, he returned with a freshly-cut key, which he inserted into the door lock.

The key worked.

How do I print non-error messages during compilation?

2012-01-13T15:00:00Z

Commenter Worf remarked, "My one wish is that #warning would be supported."

I always find it interesting when people say "I wish that Microsoft would stop following standards," since the #warning directive is nonstandard.

The Microsoft C/C++ compiler implements the feature in a method compatible with the standard, namely via a #pragma directive.


#pragma message("You really shouldn't be doing that.")

If you want to warn people away from deprecated functionality, you can use the #pragma deprecated() directive or the even more convenient (but more standards-troublesome) __declspec(deprecated) declaration specifier. The declaration specifier is much more convenient than the preprocessor directive because you can use it in a macro, and you can attach it to specific overloads of a function. (It's also more standards-troublesome because, while it is still permitted by the standard because it begins with a double-underscore, it is also not required to be ignored by compilers which do not understand it.)

In my experience, however, printing messages during compilation is of little consequence. Print all the messages during compilation as you want; nobody will read them. The only thing that gets attention is an actual warning or error. (And in many cases, only the error will get any attention at all.)

Puzzling out the upsell-o-meter

2012-01-12T15:00:01Z

As I noted before, many grocery stores in the United States have a printer next to the cash register which prints out coupons customized to your purchases. Here's a purchase and the accompanying coupon. What is the story behind this pairing?

Purchased: Diapers for newborn baby.
Coupon: Save 75 cents on ice cream.

Bonus chatter: While waiting in line, I read the warning label on the diapers. It went on for quite a bit, but one part triggered my "I wonder what lawsuit led to this warning" sensor: "Like most articles of clothing, XYZ brand diapers will burn if exposed to flame." Did somebody say, "Oh no, there's a fire, what will I do? I know, I'll smother the fire with my baby's diapered bottom!"