The items in the All Programs section of the Start menu are grouped into two sections, although there are no visible divider lines between them.

1. Non-folders, sorted alphabetically.
2. Folders, sorted alphabetically.

We saw earlier that the Fast Items lost their special status in Windows Vista and are sorted with the regular items. Another change from Windows XP is the order of the remaining two groups: Windows XP put folders above non-folders, because that was the sort order imposed by the IShellFolder::CompareIDs method so that folders sorted above files in regular Explorer windows. This deviation from standard sort order starting in Windows Vista was introduced because the guidance for application developers regarding Start menu shortcuts is to place program shortcuts in the Programs folder, with other supporting stuff in subfolders. Given that guidance, it is the program shortcuts in the Start menu that are more important, so they go at the top.

If you don't like the alphabetical ordering, then you can go the Start menu Properties, select Customize, and then scroll down to the bottom of the options tree and uncheck Sort All Programs by name. If you do this, then you can manually rearrange the items in the All Programs menu via drag/drop to put them in whatever order you like.

Pre-emptive snarky comment: "Changing the Start menu from a cascading menu to a tree navigation model was the stupidest idea since unsliced bread." Yes, I know you all hate it. Old news. Consider this a tip on how to cope with adversity.

In an interview with the Seattle Times, Rod Hamlin of Opera Software claimed,

We put a big red billboard out by Microsoft last year that said, "Want to be a real Internet explorer? www.opera.com." We got some interesting feedback on that. All of the AT&T executives could see it and all the Microsoft guys driving back home past Marymoor Park.

Okay, so where was this billboard? He says it was near Marymoor Park, and that it could be seen from AT&T executive offices, which makes sense so far because AT&T Wireless has offices in the Redmond Town Center business and shopping center, which lies right across the highway from the park.

But then things fall apart. First of all, there is no billboard stand anywhere along the stretch of highway that goes between Marymoor Park and Redmond Town Center.

Second, if you go to the regulations governing highway advertising in the State of Washington [easier-to-read PDF version], section 47.42.040 describes the types of signs allowed, and the alleged Opera billboard does not appear to be any of permissible types. (The closest match would be 47.42.040(4), if Opera had offices within twelve miles of Redmond Town Center.)

Third, you'd think there'd be plenty of pictures of an advertising campaign this cheeky. But I haven't been able to find any online. POIDH.

Here's an actual cheeky prank (and the response). The Internet Explorer team since learned their lesson, and now they send congratulatory cake.

Stop the presses: A colleague of mine says that he saw the sign. But it wasn't a billboard. Actually, it wasn't even a sign. It was a sponsorship banner hung on the fence of one of the sports fields at Marymoor Park, the sort of sign that more traditionally might read Bob's Auto Repair proudly supports youth sports. Go Mustangs!) I asked him why he didn't take a picture. "I guess we've all become pretty jaded. Either that or everybody figured somebody else would take a picture (so then nobody did)."

After our conversation, he went and took a picture.

A customer had a question about named kernel objects:

I understand that handles in a process if leaked will be destroyed by the kernel when the process exits. My question would be around named objects.

Would named objects hold their value indefinitely? If I run a small utility app to increment a named counting semaphore, the count of that named semaphore could be lost when that app exits?

I would expect it to always hold its current value so that transactions across processes and across time could be held even if no process is holding on to it.

When the last handle to a named kernel object (such as a named semaphore or a named shared memory block) is closed, the object itself is destroyed. Doesn't matter whether you explicitly closed the handle by calling CloseHandle or the kernel closed the handle for you when it cleaned up the mess you left behind. The object manager doesn't say, "Well, if the application explicitly called CloseHandle, then I'll also delete the named object, but if the application leaked the handle, then I'll leave the named object around."

First of all, that would kind of belie the whole concept of clean-up. Cleaning up means destroying the resources the application neglected to.

Second, this would create a bizarre situation where the way to access a new feature is to intentionally do something wrong. (Namely, to leak a handle to a named object.)

Okay, so maybe the expectation was that named objects persisted after all handles to them are closed, even if the handle is closed via the normal CloseHandle mechanism. But then how would you delete a named object? There is no DeleteNamedEvent function, after all. You could write a process that created 2 billion named objects and then leaked them. Boom, now you can't clean up by killing the process; you have to restart the computer.

Kernel objects all follow the same lifetime rules, whether they are named or anonymous: The object is destroyed when the last reference to it is removed (when the handle is closed, noting also that running threads and processes keep a reference to the corresponding kernel object).

If you want something that survives after all its handles are closed, then use something with a persistence model, like a file.

Today's dubious security vulnerability is another example of If you reconfigure your computer to be insecure, don't be surprised that there's a security vulnerability.

This example comes from by an actual security vulnerability report submitted to Microsoft:

I have found a critical security vulnerability that allows arbitrary elevation to administrator from unprivileged accounts.

1. Grant Full Control of the Windows directory (and all its contents and subdirectories) to Everyone.
2. Log on as an unprivileged user and perform these actions...

I can just stop there because your brain has already stopped processing input because of all the alarm bells ringing after you read that first step. That first step gives away the farm. If you grant control to the entire contents of the Windows directory to non-administrators, then don't be surprised that they can run around and do bad things!

"If I remove all the locks from my doors, then bad guys can steal my stuff."

Yeah, so don't do that. This is not a security vulnerability in the door.

Bonus chatter: There are many variations on this dubious security vulnerability. Actual vulnerability reports submitted to Microsoft include

• "First, grant world-write permission to this registry key..."
• "First, reconfigure Internet Explorer to allow scripting of ActiveX controls not marked safe for scripting..."
• "On a compromised machine, you can..."

That last one is impressive for its directness. "Starting on the other side of this airtight hatchway..."

It wasn't long before people realized that the fish in the default wallpaper for the Windows 7 beta was a betta fish. This was intended to be a cute little pun, though some people took the semiotics to an extreme,

Dude, this is Windows, not The Da Vinci Code. It's not like the people who chose the wallpaper are using a backchannel to pass secret messages to you like "I know I'm not supposed to tell you, but here's the Windows product schedule" or "Help, I'm trapped in a wallpaper factory!" They're just having a bit of fun.

I have yet to see anybody point out that the fish was blowing seven bubbles. And no, I don't know what it "signifies".

(I always wonder about the people who claim that the government embeds subtle messages about their complex conspiracies in plain sight. If you want to keep a secret, you don't run around talking about it!)

A customer liaison asked the following question on behalf of his customer:

My customer is finding that their Windows Server 2003 system has restarted, and they want to find out why. I've found some event log similar to the ones below, but I don't know what error code 0x84020004 is. I've searched the Knowledge Base but couldn't find anything relevant. Please let me know why the system restarted.

Event Type: Information
Event Source: USER32
Event Category: None
Event ID: 1074
Date: 3/20/2009
Time: 11:51:30 AM
User: GROUP1\infraadmin
Computer: DATA2
Description:
The process Explorer.EXE has initiated the shutdown of computer SYSP1 on behalf
of user GROUP1\infraadmin for the following reason: Operating System: 
Reconfiguration (Planned)
Reason Code: 0x84020004
Shutdown Type: restart
Comment:

The value 0x84020004 is not an error code. It says right there that it's a reason code:

Reason Code: 0x84020004

The system shutdown reason codes are documented in MSDN under the devious heading System Shutdown Reason Codes. In this case, the value 0x84020004 is a combination of

SHTDN_REASON_FLAG_PLANNED               0x80000000
SHTDN_REASON_FLAG_CLEAN_UI              0x04000000 // reason.h
SHTDN_REASON_MAJOR_OPERATINGSYSTEM      0x00020000
SHTDN_REASON_MINOR_RECONFIG             0x00000004

That value for SHTDN_REASON_FLAG_CLEAN_UI is missing from the MSDN documentation for some reason, but's listed in reason.h. The flag means that the system was shut down in a controlled manner, as opposed to SHDTN_REASON_FLAG_DIRTY_UI which means that the system lost power and did not go through a clean shutdown.

In other words, this was a planned shutdown that was the result of an operating system reconfiguration. Perhaps somebody changed a system setting in the Control Panel, and in response to the question "The change you made requires that the system be restarted. Restart now?", the person said Yes.

I was in the supermarket looking for cold medicine, and as is my wont, I like to read the fine print before choosing a product. Most of the products listed their active ingredients in the form Active Ingredient: XYZ 150mg. But there were a few that said Active Ingredient: XYZ 6X.

What is this 6X? How much is 6X? Six times what?

A closer look at the box reveals the word Homeopathic unobtrusively written towards the bottom of the box. The 6X notation means that the active ingredient's concentration is one part in 10⁶, or one part in a million.

Suppose the dosage is one teaspoon. That's about five milliliters, or about five grams of medicine. One millionth of that is 5 × 10^-6 of a gram or 0.005 milligrams. By comparison, non-homeopathic medicines contain 13mg of the same medicine per dose. But that's okay, because practitioners of homeopathy believe that a medicine becomes more powerful the more heavily it is diluted.

The Food and Drug Administration does not regulate homeopathic medicines very much at all because they contain little or no active ingredients.

Do practitioners of homeopathy not wash their hands? After all, diluting the germs with water makes them stronger, right? The more you wash, the more powerful they become!

Bonus homeopathy video: Homeopathic A&E as performed by British comedy show That Mitchell and Webb Look. (A&E = Accidents and Emergencies, what in the United States is commonly called the Emergency Room.)

Bonus homeopathy satire: New Age terrorists develop homeopathic bomb.

Obligatory XKCD link.

In other parts of the world, religious images emerge from random patterns. Out here, we get Microsoft marketing.

If I were on it, I could've charged admission.

Andy Visser posted to the Suggestion Box something that wasn't so much a suggestion as a comment, presumably to get around the fact that comments on the original item had been closed: "I've found that the start bar seems to behave like it may be using this call incorrectly. I put my start bar on the left hand side of the screen. When I try to resize the bar (dragging its edge left and right), the system tray will dynamically move icons (based on tray width), seemingly disregarding the lock. The rest of the bar waits until MouseUp to redraw."

Actually, the taskbar (that's the name of the thing you're referring to) does not call LockWindowUpdate at all, so what you're seeing is not the result of any incorrect use of LockWindowUpdate. I've also been unable to reproduce the behavior you describe. I tried Windows XP, Windows Vista, and Windows 7 both with and without Show window contents while dragging enabled, and the notification area (that's the name of the thing you're referring to) repaints at the same time as the rest of the taskbar. I don't see it "bypassing" anything.

This comment demonstrates one of the common problems with bug reports submitted from the field: The description of the problem rarely includes the system configuration under which the problem occurs—they often don't even mention what operating system they're running! The person submitting the comment generally assumes that you will somehow know how their computer is configured (or believes that the problem is not configuration-dependent). This leaves people like me trying to reproduce the problem on various systems with various configurations before finally giving up and saying "Sorry, I don't see it."

In some cases, the configuration setting that created the unwanted behavior is a setting whose sole purpose is to establish the unwanted behavior. For example, a customer reported, "Windows Vista is not displaying the user's picture on the Start menu or the logon screen. We discovered that the Apply the default user logon picture to all users policy prevents the user's customized picture from being displayed. We removed that policy, but that did not solve the problem. Is there anything else that might be causing this?"

I found it interesting that the customer initially wondered why the user's custom picture wasn't showing up, when they had explicitly set the policy that says "Do not use the user's custom picture." But at least they figured that part out on their own.

After some more questions, the customer explained that removing the policy restored the user's custom picture to the Start menu, but it nevertheless did not appear on the logon screen. (Thereby illustrating the problem of vague antecedents. When they wrote "that did not solve the problem", they weren't clear what the problem was. It turns out that they meant "that only solved part of the problem": The user picture appears on the Start menu, but not on the logon screen.)

After additional rounds of troubleshooting, the customer eventually revealed that they had also set the policy Do not display last logged on user name. Um, if you disable showing the name of the last logged-on user, then the picture of the user doesn't appear either. (Maybe they took too literal a reading of the policy, expecting it to hide the name but not the picture? What would be the purpose of that? Exercise: Why isn't the policy called the more accurate Do not display information about last logged on user?)

If instead of calling ExitProcess you merely return from the main thread of a process, does the process terminate?

No, but maybe yes.

This is another one of the places where the C runtime behaves differently from raw Win32.

Under raw Win32, a process exits when any thread chooses to exit the process explicitly (usually by calling ExitProcess) or when all threads have exited. Exiting the main thread will not result in the process exiting if there are any other threads still active. According to the old-fashioned model of how processes exit, a process was in control of all its threads and could mediate the shutdown of those threads, thereby controlling the shutdown of the process. (Of course, nowadays, with the thread pool, COM worker threads, and other threads doing random background work, the idea of being in control of all the threads in the process is now just a reminder of those simpler days.)

On the other hand, the C runtime library automatically calls ExitProcess when you exit the main thread, regardless of whether there are any worker threads still active. This behavior for console programs is mandated by the C language, which says that (5.1.2.2.3) "a return from the initial call to the main function is equivalent to calling the exit function with the value returned by the main function as its argument." The C++ language has an equivalent requirement (3.6.1). Presumably, the C runtime folks carried this behavior to WinMain for consistency.

This also means that if you decide to exit your main thread by calling ExitThread directly, then you aren't returning from the main function. Instead, you've leapt into the Win32 world where the process will not exit until all threads are gone.

A customer had the following question:

I'm using the ShellExecute function to launch a new process and am passing the handle to my application's main window as the hwnd parameter. From the new process, I want to get information from the old process, and to do that, I need the window handle. How can I recover that window handle from the new process?

You can't.

That window handle is used by the ShellExecute function only to host any user interface operations that occur as a result of the attempt to execute the program. For example, it is the owner window used for any error dialogs. The ShellExecute function does not pass the window handle to the launched process. (It couldn't even if it wanted to: There is nowhere to pass it. There is no window handle among the parameters to CreateProcess nor is there a window handle in the STARTUPINFO structure.)

If you want to pass this information to the process being launched, you'll have to come up with your own mechanism for transferring this information. For example, you can pass it on the command line, or if you have a lot of information to pass, you can use a shared memory block.

My young nieces live in a Chinese-speaking household, which is great for them because it means that when they grow up, they will be fluent in two languages. But it makes things a bit tricky at the beginning.

The niece who is the subject of this story had just turned two at the time this story takes place, so her language skills even in Chinese are pretty rudimentary. Her language skills in English are restricted to a collection of set phrases like Excuse me!, I'm sorry!, What'you doing?, I want ice cream!, and any catch phrase from the character Dora the Explorer.

(I'm also fairly sure she doesn't know what What'you doing? actually means. She'll come into a room and say, What'you doing? and then appear completely uninterested in the answer. I think she believes it to be a form of greeting and not an actual question.)

She also loves to answer the phone, and this usually isn't a problem since most callers are relatives who can speak Chinese. But occasionally, it'll be somebody who speaks only English. (In general, these are just telemarketers, since most members of the household use their mobile phones as their main number.)

Sometimes she'll run to the phone, pick it up, say "喂" (Hello), listen for a few seconds, and then just hang up.

— Who was that on the phone? we'll ask.

"人" is her one-word reply.

It's hard to explain why this is a funny answer.

The word 人 means man, person, so her response was a casual "A person." The offhand way she says it expresses her attitude that "The purpose of the telephone is to amuse me, but this was just some guy who provided no entertainment at all."

The 人 phase lasted for only a month or so. In the next phase, she still picked up the phone and hung up when there was somebody speaking English on the other end, but when we asked her who it was, she gave a more detailed reply:

"有人說ＡＢＣ", which translates roughly as "It's some guy speaking A-B-C." ("A-B-C" being her word for the English language.)

A customer noticed that when you right-click on Computer in the second column of the Start menu on Windows Vista, the first two options are Open and Explore. On the other hand, in Windows 7, the Explore option is gone, leaving just Open. The customer also noticed that in Windows Vista, the two commands had the same effect and wondered if Explore was removed because it was redundant.

The response from the product team was a very simple "Yes."

It's interesting when a customer notices a relatively insignificant UI change, figures out the likely reason for the change, and then asks for confirmation. It's not like the reason for the change affects anything. My guess is that the customer already paid for a support contract so they're just going to use it, even when the issue wouldn't normally be worth raising a support incident over.

Today marks the 15th anniversary of the public release of Windows 95.

During the development of Windows 95, one of the team members attended a Mac conference. And not as a secret agent, either. He proudly wore a Windows 95 team T-shirt as he strolled among the booths.

The rest of us back at the mother ship wished him well and started discussing how we could get access to his dental records so we could identify his remains when they were sent back to us from the conference.

When he returned, we didn't kill a calf in his honor, but we did marvel at his survival skills and asked him how it went.

— I got a lot of funny looks. And one guy, upon confirming that I really did work on the Windows 95 project, said to me, "I have to commend you guys on Windows 95 so far. It sucks less."

That backwards compliment tickled the team's funny bone, and it quickly became the unofficial team motto: Windows 95: It sucks less.

Commenter Erbi has a program which creates a splash screen on a background thread while the main thread initializes. "I create and then destroy this splash screen window just before creating and displaying the main window." The problem is that the main window fails to obtain foreground activation. Commenting out the code that creates the splash screen fixes the problem, but then there isn't a splash screen any more (obviously). "Is there an explanation for this behavior?"

This behavior is explained by two earlier blog posts, plus a PDC talk. The first blog post came out years before this question was asked: The correct order for disabling and enabling windows. Destroying a window is a rather extreme case of disabling it, but the effect is the same. When you destroy the splash screen, foreground activation needs to move to some other window, and since your main window isn't around to inherit it, foreground activation leaves your program. When the main window appears, it's too late.

The PDC talk came next, followed shortly thereafter by a blog post version of the same talk. As marketing folks like to remind you, "You get only one chance to make a first impression." Similarly, you get only one chance to use your foreground activation permission, and you decided to blow it on a splash screen. That's fine as far as it goes, but if you want to transfer that permission to another window, you have to manage it yourself. The recommended way is to establish an owner/owned relationship between them; that's the case that the "disabling and enabling windows" article focuses on.

Miss Universe 2010 National Costumes, Part 1
Miss Universe 2010 National Costumes, Part 2

Commentary in parts NSFW but they so deserve it.

By definition, the primary monitor is the monitor that has (0,0) as its upper left corner. Why can't the primary monitor be positioned somewhere else?

Well, sure you could do that, but then you'd have to invent a new name for the monitor whose upper left corner is at (0,0), and then you're back where you started.

In other words, it's just a name. You could ask, "Why can't starboard be on the left-hand side of the boat?" Well, sure you could do that, but then you'd have to come up with a new name for the right-hand side of the boat, and then things are pretty much the same as they were, just with different names.

Perhaps a more descriptive (but clumsier) name for the primary monitor would be the backward-compatibility monitor (for applications which do not support multiple monitors), because that's what the primary monitor is. If an application is not multiple-monitor aware, then any time it asks for properties of "the" monitor, it gets information about the backward-compatibility monitor. A call to GetSystemMetrics(SM_CXSCREEN) gives the width of the backward-compatibility monitor, GetSystemMetrics(SM_CYMAXIMIZED) gives the height of a window maximized on the backward-compatibility monitor, and positioning a window at (0,0) puts it at the upper left corner of the backward-compatibility monitor.

The window manager still respects window coordinates passed to functions like CreateWindow or SetWindowPos. If you pass coordinates that are on a secondary monitor—oops—a monitor different from the backward-compatibility monitor, then the window manager will happily put the window there. These coordinates might be the result of a program that is multiple-monitor aware, or possibly merely from a program which is multiple-monitor agnostic.

Multiple-monitor agnosticism is a term I just made up which refers to programs which might not explicitly support multiple monitors, but at least were open to the possibility of multiple monitors by not making assumptions about the number of monitors but instead using functions like RectVisible to determine what the visible portions of the screen are. These techniques were hot topics many years ago when you wanted to write a program that ran both on single-monitor-only versions of Windows as well as multiple-monitor versions of windows; nowadays there are rather old-fashioned, like coming up with mnemonics for all your friends' telephone numbers so you didn't have to keep looking them up. (Today, you just go ahead and call the multiple monitor functions and use the address book function in your mobile phone to remember your friends' phone numbers.)

It is not the case that the primary monitor is the applications show up here first monitor. As noted earlier, applications show up on whatever monitor they ask for, whether they asked for it explicitly (hunting around for a monitor and using it) or implicitly (restoring to the same coordinates they were on when they were last run).

Of course, programs which pass CW_USEDEFAULT to the CreateWindow function explicitly abdicated the choice of the window position and therefore the monitor. In that case, the window manager tries to guess an appropriate monitor. If the new window has a parent or owner, then it is placed on the same monitor as that parent or owner; otherwise, the window manager just puts the window on the backward-compatible monitor, for lack of a better idea.

The other day, I was in the office kitchenette, and two of my colleagues both named Paul happened to be there getting coffee. I quipped, "Oh no, is this legal? I think it's a violation of the Paul Exclusion Principle."

It was a horrible physics pun, perhaps one of the worst I've made in a long time. My challenge to you is to come up with an even worse one that you've told.

Note: You have to have actually made the pun to an appropriate audience. No fair just making one up for the purpose of the challenge.

In Windows 7, the folder tree in the Explorer navigation pane by default no longer highlights the item in the view pane. This change was based on user testing and feedback, but if, like me, you prefer things the old way, you can play with two new check boxes on the Folder Options dialog. You can get to Folder Options in a variety of ways:

• From the Control Panel, go to Appearance and Personalization → Folder Options. (Or just type Folder Options into the Control Panel search box or the Start menu search box to go straight there.)
• From the Explorer menu bar, select Tools → Folder options.
• From the Explorer command bar, select Organize → Folder and search options.
• Or you can exercise your super élite status and just right-click on a blank space in the navigation pane.

However you wind up there, the item you want to turn on is Automatically expand to current folder (or Expand to current folder if you use the super élite method).

In the Customer Service and Support part of Microsoft, you will often see the term funnel. Here are some citations:

Effectively and efficiently solve issues by driving levers across the entire funnel.

Putting the Fun in Funnel.

Strengthening the front of the funnel.

The funnel is a way of viewing customer support engagements. For some reason, the funnel diagram is always drawn on its side with the mouth (the fat part) on the left and the stem (the narrow part) on the right. The width of the funnel represents the volume of customers at that stage of the support process, and the progress through the funnel represents how much time the customer has spent working on a solution.

At the mouth of the funnel are the customers who turn to built-in product help, online help, forums, Knowledge Base articles, blog entries, training materials, and similar resources. A significant percentage of customers get the help they need via self-help, where the solution to their problem existed even before they asked the question; they just had to find it.

The funnel narrows, and the customer picks up the phone or otherwise initiates a support incident. A support technician helps the customer via email, live chat, phone call, whatever. Another percentage of customers get their problem solved at this stage. It took longer, but the problem did get solved.

At the stem of the funnel are the customers whose problems remain unresolved, and now things get bad. The problem takes days to resolve, multiple engineers get involved, and maybe even a site visit is called for.

There is a concerted effort to improve the support resources at the front of the funnel. Of course, there are efforts to improve the support process at all of the stages, but the front of the funnel is a particular area of focus, since that's where everybody starts out, and that's where most users get their solutions. The term front of the funnel is in such heavy use it even gets its own acronym: FoF. Is it pronounced foff? Beats me.

The first time you open a WinHelp file, you get this pen-writing-in-book animation while WinHelp does um something which it passes off as preparing Help file for first use or something similarly vague.

I remember a conversation about that animation. The Windows shell team suggested to the author of WinHelp that the program use the shell common animation control to display that animation. After all, it's a short animation and it met the requirements for the animation common control. But the author of WinHelp rejected the notion.

(Pre-emptive "I can't believe I had to write this": This conversation has been exaggerated for effect.)

"Your animation control is so huge and bloated. I can do it much smaller and faster myself. The RLE animation format generates frames by re-rendering the pixels that have changed, which means that at each frame of the animation, a new pen image would be recorded in the AVI file. The pen cycles through three different orientations at each location, there are ten locations on each row, and there are four rows. If I used an RLE animation, that'd be 3 × 10 × 4 = 120 copies of the pen bitmap. Instead, I have just three pen bitmaps, and I manually draw them at the appropriate location for each frame. Something like this:

// NOTE: For simplicity, I'm ignoring the "turn the page" animation
void DrawFrame(int frame)
{
  // Calculate our position in the animation
  int penframe = frame % 3; // 3 pen images per location
  int column = (frame / 3) % 10; // 10 columns per row
  int row = (frame / 30) % 4; // 4 rows
  int i;
  POINT pt;

  DrawBlankPage(0, 0); // start with a blank sheet of paper

  // Draw the "text" that the pen "wrote" in earlier rows
  for (i = 0; i < row; i++) {
    DrawTextScribble(i, 0, 9);
  }

  // Draw the partially-completed row that the pen is on now
  DrawTextScribble(row, 0, column);

  // Position the pen image so the pen tip hits the "draw" point
  GetTextScribblePoint(column, row, &pt);
  DrawPenBitmap(penBitmaps[penframe], pt.x - 1, pt.y - 5);
}

"See? In just a few lines of code, I have a complete animation. All I needed was the three pen images and a background bitmap showing a book opened to a blank page. This is way more efficient both in terms of memory and execution time than your stupid animation common control. You shell guys could learn a thing or two about programming."

"Okay, fine, don't need to get all defensive about it. We were just making a suggestion, that's all."

Time passes, and Windows 95 is sent off for translation into the however many languages it is localized for. A message comes in from some of the localization teams. It seems that some locales need to change the animation. For example, the Arabic version of Windows needs the pen to write on the left-hand pages, the pen motion should be right to left, and the pages need to flip from left to right. Perhaps the Japanese translators are okay with the pen motion, but they want the pages to flip from left to right.

The localization team contacted the WinHelp author. "We're trying to change the animation, but we can't find the AVI file in the resources. Can you advise us on how we should localize the animation?"

Unfortunately, the WinHelp author had to tell the localization team that the direction of pen motion, and the locations of the ink marks are hard-coded into the program. Since the product had already passed code lockdown, there was nothing that could be done. WinHelp shipped with a pen that moved in the wrong direction in some locales.

Moral of the story: There's more to software development than programming for performance. Today we learned about localizability.

Commenter winhelp (probably not his/her real name) wonders what happened to WinHelp.exe.

I don't know, but it turns out the answer was already known to the Internet. At the time the question was posted, the answer was already in the Wikipedia entry for Windows Help—it even had a citation!

The question does highlight another one of those no matter what you do, somebody will call you an idiot dilemmas. On the one side, we have "Windows is already so big, what's the harm in adding another megabyte to the size to add this feature that is rarely used, primarly by older applications, so that customers won't have to download it?" On the other side, we have "Windows is too big, why not get rid of the components that exist only for the benefit of older applications and make them optional downloads?"

What probably swung the pendulum to the remove it from the core product side is the fact that the Windows help file format is equivalent to an EXE. (I don't know this personally; I'm just reading the Wikipedia article.) If somebody can trick you into clicking on a rogue HLP file, they can run arbitrary code and take over your account. The underlying functionality is useful, because you can write help files with links like Click here to open the Options dialog, and clicking the link will actually open the Options dialog (by invoking some accompanying native code that calls whatever APIs are necessary to get that Options dialog to open).

WinHelp came from the days before the Internet, when HyperCard was the reigning champion for page-based information presentation. You didn't have to worry that double-clicking a file on a remote server might take over your computer because you couldn't contact remote servers in the first place! (And if you could, it was because you were on a local-area network where all the computers were operated by your co-workers or other people you trusted.)

As I recall, there are some help-file-based viruses out there, so the security aspect is not merely a theoretical discussion. Removing the attack surface from the default configuration reduces the value of the help file attack. (Historians may note that HyperCard also permitted execution of arbitrary native code attached to a HyperCard deck. There were also HyperCard viruses.)

But now that you mention WinHelp, I remember a story about the little pen-writing-in-book animation that appears when the help engine is "preparing Help file for first use" (whatever that means). I'll take that up tomorrow.

Finalization is the crazy wildcard in garbage collection. It operates "behind the GC", running after the GC has declared an object dead. Think about it: Finalizers run on objects that have no active references. How can this be a reference to an object that has no references? That's just crazy-talk!

Finalizers are a Ouija board, permitting dead objects to operate "from beyond the grave" and affect live objects. As a result, when finalizers are involved, there is a lot of creepy spooky juju going on, and you need to tread very carefully, or your soul will become cursed.

Let's step back and look at a different problem first. Consider this class which doesn't do anything interesting but works well enough for demonstration purposes:

class Sample1 {
 private StreamReader sr;
 public Sample1(string file) : sr(new StreamReader(file)) { }
 public void Close() { sr.Close(); }
 public string NextLine() { return sr.ReadLine(); }
}

What happens if one thread calls Sample1.NextLine() and another thread calls Sample1.Close()? If the NextLine() call wins the race, then you have a stream closed while it is in the middle of its ReadLine method. Probably not good. If the Close() call wins the race, then when the NextLine() call is made, you end up reading from a closed stream. Definitely not good. Finally, if the NextLine() call runs to completion before the Close(), then the line is successfully read before the stream is closed.

Having this race condition is clearly an unwanted state of affairs since the result is unpredictable.

Now let's change the Close() method to a finalizer.

class Sample2 {
 private StreamReader sr;
 public Sample2(string file) : sr(new StreamReader(file)) { }
 ~Sample2() { sr.Close(); }
 public string NextLine() { return sr.ReadLine(); }
}

Remember that we learned that an object becomes eligible for garbage collection when there are no active references to it, and that it can happen even while a method on the object is still active. Consider this function:

string FirstLine(string fileName) {
 Sample2 s = new Sample2(fileName);
 return s.NextLine();
}

We learned that the Sample2 object becomes eligible for collection during the execution of NextLine(). Suppose that the garbage collector runs and collects the object while NextLine is still running. This could happen if ReadLine takes a long time, say, because the hard drive needs to spin up or there is a network hiccup; or it could happen just because it's not your lucky day and the garbage collector ran at just the wrong moment. Since this object has a finalizer, the finalizer runs before the memory is discarded, and the finalizer closes the StreamReader.

Boom, we just hit the race condition we considered when we looked at Sample1: The stream was closed while it was being read from. The garbage collector is a rogue thread that closes the thread at a bad time. The problem occurs because the garbage collector doesn't know that the finalizer is going to make changes to other objects.

Classically speaking, there are three conditions which in combination lead to this problem:

1. Containment: An entity a retains a reference to another entity b.
2. Incomplete encapsulation: The entity b is visible to an entity outside a.
3. Propagation of destructive effect: Some operation performed on entity a has an effect on entity b which alters its proper usage (usually by rendering it useless).

The first condition (containment) is something you do without a second's thought. If you look at any class, there's a very high chance that it has, among its fields, a reference to another object.

The second condition (incomplete encapsulation) is also a common pattern. In particular, if b is an object with methods, it will be visible to itself.

The third condition (propagation of destructive effect) is the tricky one. If an operation on entity a has a damaging effect on entity b, the code must be careful not to damage it while it's still being used. This is something you usually take care of explicitly, since you're the one who wrote the code that calls the destructive method.

Unless the destructive method is a finalizer.

If the destructive method is a finalizer, then you do not have complete control over when it will run. And it is one of the fundamental laws of the universe that events will occur at the worst possible time.

Enter GC.KeepAlive(). The purpose of GC.KeepAlive() is to force the garbage collector to treat the object as still live, thereby preventing it from being collected, and thereby preventing the finalizer from running prematurely.

(Here's the money sentence.) You need to use GC.KeepAlive when the finalizer for an object has a destructive effect on a contained object.

The problem is that it's not always clear which objects have finalizers which have destructive effect on a contained object. There are some cases where you can suspect this is happening due to the nature of the object itself. For example, if the object manages something external to the CLR, then its finalizer will probably destroy the external object. But there can be other cases where the need for GC.KeepAlive is not obvious.

A much cleaner solution than using GC.KeepAlive is to use the IDisposable interface, formalized by the using keyword. Everybody knows that the using keyword ensures that the object being used is disposed at the end of the block. But it's also the case (and it is this behavior that is important today) that the using keyword also keeps the object alive until the end of the block. (Why? Because the object needs to be alive so that we can call Dispose on it!)

This is one of the reasons I don't like finalizers. Since they operate underneath the GC, they undermine many principles of garbage collected systems. (See also resurrection.) As we saw earlier, a correctly-written program cannot rely on side effects of a finalizer, so in theory all finalizers could be nop'd out without affecting correctness.

The garbage collector purist in me also doesn't like finalizers because they prevent the running time of a garbage collector to be proportional to the amount of live data, like say in a classic two-space collector. (There is also a small constant associated with the amount of dead data, which means that the overall complexity is proportional to the amount of total data.)

If I ruled the world, I would decree that the only thing you can do in a finalizer is perform some tests to ensure that all the associated external resources have already been explicitly released, and if not, raise a fatal exception: System.Exception.Resource­Leak.

Bonus reading

• The often non-difference between Close and Dispose.
• Marshal.ReleaseComObject considered dangerous.

More than one customer has asked a question like this:

I'm looking for a way to search for all instances of a particular type at runtime. My goal is to invoke a particular method on each of those instances. Note that I did not create these object myself or have any other access to them. Is this possible?

Imagine what the world would be like if it were possible.

For starters, just imagine the fun you could have if you could call typeof(Secure­String).Get­Instances(). Vegas road trip!

More generally, it breaks the semantics of App­Domain boundaries, since grabbing all instances of a type lets you get objects from another App­Domain, which fundamentally violates the point of App­Domains. (Okay, you could repair this by saying that the Get­Instances method only returns objects from the current App­Domain.)

This imaginary Get­Instances method might return objects which are awaiting finalization, which violates one of the fundamental assumptions of a finalizer, namely that there are no references to the object: If there were, then it wouldn't be finalized! (Okay, you could repair this by saying that the Get­Instances method does not return objects which are awaiting finalization.)

On top of that, you break the syncRoot pattern.

class Sample {
 private object syncRoot = new object();
 public void Method() {
  lock(syncRoot) { ... };
 }
}

If it were possible to get all objects of a particular class, then anybody could just reach in and grab your private sync­Root and call Monitor.Enter() on it. Congratuations, the private synchronization object you created is now a public one that anybody can screw with, defeating the whole purpose of having a private syncRoot. You can no longer reason about your syncRoot because you are no longer in full control of it. (Yes, this can already be done with reflection, but at least when reflecting, you know that you're grabbing somebody's private field called sync­Root, so you already recognize that you're doing something dubious. Whereas with Get­Instances, you don't know what each of the returned objects is being used for. Heck, you don't even know if it's being used! It might just be garbage lying around waiting to be collected.)

More generally, code is often written on the expectation that an object that you never give out a reference to is not accessible to others. Consider the following code fragment:

using (StreamWriter sr = new StreamWriter(fileName)) {
 sr.WriteLine("Hello");
}

If it were possible to get all objects of a particular class, you may find that your customers report that they are getting an Object­Disposed­Exception on the call to Write­Line. How is that possible? The disposal doesn't happen until the close-brace, right? Is there a bug in the CLR where it's disposing an object too soon?

Nope, what happened is that some other thread did exactly what the customer was asking for a way to do: It grabbed all existing Stream­Writer instances and invoked Stream­Writer.Close on them. It did this immediately after you constructed the Stream­Writer and before you did your sr.Write­Line(). Result: When your sr.Write­Line() executes, it finds that the stream was already closed, and therefore the write fails.

More generally, consider the graffiti you could inject into all output files by doing

foreach (StreamWriter sr in typeof(StreamWriter).GetInstances()) {
 sr.Write("Kilroy was here!");
}

or even crazier

foreach (StringBuilder rb in typeof(StringBuilder).GetInstances()) {
 sb.Insert(0, "DROP TABLE users; --");
}

Now no String­Builder is safe—the contents of any String­Builder can be corrupted at any time!

If you could obtain all instances of a type, the fundamental logic behind computer programming breaks down. It effectively becomes impossible to reason about code because anything could happen to your objects at any time.

If you need to be able to get all instances of a class, you need to add that functionality to the class itself. (GC­Handle or Weak­Reference will come in handy here.) Of course, if you do this, then you clearly opted into the "anything can happen to your object at any time outside your control" model and presumably your code operates accordingly. You made your bed; now you get to lie in it.

(And I haven't even touched on thread safety.)

Bonus reading: Questionable value of SyncRoot on Collections.

A customer asked the rather enigmatic question (with no context):

Is there a way to get the reference count of an object in .Net?

Thanks,
Bob Smith
Senior Developer
Contoso

The CLR does not maintain reference counts, so there is no reference count to "get". The garbage collector only cares about whether an object has zero references or at least one reference. It doesn't care if there is one, two, twelve, or five hundred—from the point of view of the garbage collector, one is as good as five hundred.

The customer replied,

I am aware of that, yet the mechanism is somehow implemented by the GC...

What I want to know is whether at a certain point there is more then one variable pointing to the same object.

As already noted, the GC does not implement the "count the number of references to this object" algorithm. It only implements the "Is it definitely safe to reclaim the memory for his object?" algorithm. A null garbage collector always answers "No." A tracing collector looks for references, but it only cares whether it found one, not how many it found.

The discussion of "variables pointing to the same objects" is somewhat confused, because you can have references to an object from things other than variables. Parameters to a method contain references, the implicit this is also a reference, and partially-evaluated expressions also contain references. (During execution of the line string s = o.ToString();, at the point immediately after o.ToString() returns and before the result is assigned to s, the string has an active reference but it isn't stored in any variable.) And as we saw earlier, merely storing a reference in a variable doesn't prevent the object from being collected.

It's clear that this person solved half of his problem, and just needs help with the other half, the half that doesn't make any sense. (I like how he immediately weakened his request from "I want the exact reference count" to "I want to know if it is greater than one." Because as we all know, the best way to solve a problem is to reduce it to an even harder problem.)

Another person used some psychic powers to figure out what the real problem is:

If I am reading properly into what you mean, you may want to check out the Weak­Reference class. This lets you determine whether an object has been collected. Note that you don't get access to a reference count; it's a zero/nonzero thing. If the Weak­Reference is empty, it means the object has been collected. You don't get a chance to act upon it (as you would if you were the last one holding a reference to it).

The customer explained that he tried Weak­Reference, but it didn't work. (By withholding this information, the customer made the mistake of not saying what he already tried and why it didn't work.)

Well this is exactly the problem: I instantiate an object and then create a Weak­Reference to it (global variable).

Then at some point the object is released (set to null, disposed, erased from the face of the earth, you name it) yet if I check the Is­Alive property it still returns true.

Only if I explicitly call to GC.Collect(0) or greater before the check it is disposed.

The customer still hasn't let go of the concept of reference counting, since he says that the object is "released". In a garbage-collected system, object are not released; rather, you simply stop referencing them. And disposing of an object still maintains a reference; disposing just invokes the IDisposable.Dispose method.

FileStream fs = new FileStream(fileName);
using (fs) {
 ...
}

At the end of this code fragment, the File­Stream has been disposed, but there is still a reference to it in the fs variable. Mind you, that reference isn't very useful, since there isn't much you can do with a disposed object, Even if you rewrite the fragment as

using (FileStream fs = new FileStream(fileName)) {
 ...
}

the variable fs still exists after the close-brace; it simply has gone out of scope (i.e., you can't access it any more). Scope is not the same as lifetime. Of course, the optimizer can step in and make the object eligible for collection once the value becomes inaccessible, but there is no requirement that this optimization be done.

The fact that the Is­Alive property says true even after all known references have been destroyed is also no surprise. The environment does not check whether an object's last reference has been made inaccessible every time a reference changes. One of the major performance benefits of garbage collected systems comes from the de-amortization of object lifetime determination. Instead of maintaining lifetime information about an object continuously (spending a penny each time a reference is created or destroyed), it saves up those pennies and splurges on a few dollars every so often. The calculated risk (which usually pays off) is that the rate of penny-saving makes up for the occasional splurges.

It does mean that between the splurges, the garbage collector does not know whether an object has outstanding references or not. It doesn't find out until it does a collection.

The null garbage collector takes this approach to an extreme by simply hoarding pennies and never spending them. It saves a lot of money but consumes a lot of memory. The other extreme (common in unmanaged environments) is to spend the pennies as soon as possible. It spends a lot of money but reduces memory usage to the absolute minimum. The designers of a garbage collector work to find the right balance between these two extremes, saving money overall while still keeping memory usage at a reasonable level.

The customer appears to have misinterpreted what the Is­Alive property means. The property doesn't say whether there are any references to the object. It says whether the object has been garbage collected. Since the garbage collector can run at any time, there is nothing meaningful you can conclude if Is­Alive returns true, since it can transition from alive to dead while you're talking about it. On the other hand, once it's dead, it stays dead; it is valid to take action when Is­Alive is false. (Note that there are two types of Weak­Reference; the difference is when they issue the death certificate.)

The name Is­Alive for the property could be viewed as misleading if you just look at the property name without reading the accompanying documentation. Perhaps a more accurate (but much clumsier) name would have been Has­Not­Been­Collected. The theory is, presumably, that if you're using an advanced class like Weak­Reference, which works "at the GC level", you need to understand the GC.

The behavior the customer is seeing is correct. The odds that the garbage collector has run between annihilating the last live reference and checking the Is­Alive property is pretty low, so when you ask whether the object has been collected, the answer will be No. Of course, forcing a collection will cause the garbage collector to run, and that's what does the collection and sets Is­Alive to false. Mind you, forcing the collection to take place messes up the careful penny-pinching the garbage collector has been performing. You forced it to pay for a collection before it had finished saving up for it, putting the garbage collector in debt. (Is there a garbage collector debt collector?) And the effect of a garbage collector going into debt is that your program runs slower than it would have if you had let the collector spend its money on its own terms.

Note also that forcing a generation-zero collection does not guarantee that the object in question will be collected: It may have been promoted into a higher generation. (Generational garbage collection takes advantage of typical real-world object lifetime profiles by spending only fifty cents on a partial collection rather than a whole dollar on a full collection. As a rough guide, the cost of a collection is proportional to the number of live object scanned, so the most efficient collections are those which find mostly dead objects.) Forcing an early generation-zero collection messes up the careful balance between cheap-but-partial collections and expensive-and-thorough collections, causing objects to get promoted into higher generations before they really deserve it.

Okay, that was a long discussion of a short email thread. Maybe tomorrow I'll do a better job of keeping things short.

Bonus chatter: In addition to the Weak­Reference class, there is also the GC­Handle structure.

Bonus reading: Maoni's WebLog goes into lots of detail on the internals of the CLR garbage collector. Doug Stewart created this handy index.