If you're looking to get into some p/invoke action, you'd be well-served to check out the pinvoke wiki to see if somebody else has done it too. If what you need isn't there, you may end up forced to write your own, and here are some gotchas I've seen people run into:

• C++ bool and Win32 BOOLEAN are not the same as C# bool (aka System.Boolean). In Win32, BOOL is a 4-byte type, and BOOLEAN is a 1-byte type. [See also MadQ's remarks about VARIANT_BOOL.] Meanwhile, C++ bool is not standardized by Win32, so the size will vary based on your compiler, but most compilers use a 1-byte value. And then C# is even weirder: The bool is a 1-byte type, but it marshals as a 4-byte type by default.
• Win32 char is not the same as C# char (aka System.Char). In C#, char is a Unicode character (two bytes), whereas in C/C++ under Win32 it is an ANSI character (one byte).
• Win32 long is not the same as C# long (aka System.Int64). In C#, long is 64-bit value, whereas in C/C++ under Win32 it is a 32-bit value.
• If memory is allocated and freed across the interop boundary, make sure both sides are using the same allocator. It is my understanding that the CLR uses CoTaskMemAlloc/CoTaskMemFree by default. If your Win32 function doesn't use CoTaskMemAlloc, you'll have to teach the CLR which allocator you really want.
• When laying out structures, you have to watch out for alignment.

That last one is particularly gnarly on 64-bit systems, where alignment requirements are less forgiving than on x86. The structure declarations on pinvoke.net tend to ignore 64-bit issues. For example, the declaration of the INPUT structure (as of this writing—it's a wiki so it's probably changed by the time you read this) reads as follows:

[StructLayout(LayoutKind.Explicit)]struct INPUT {
  [FieldOffset(0)] int type;
  [FieldOffset(4)] MOUSEINPUT mi;
  [FieldOffset(4)] KEYBDINPUT ki;
  [FieldOffset(4)] HARDWAREINPUT hi;
}

This structure layout is correct for 32-bit Windows, but it's incorrect for 64-bit Windows.

Let's take a look at that MOUSEINPUT structure, for starters.

typedef struct tagMOUSEINPUT {
    LONG    dx;
    LONG    dy;
    DWORD   mouseData;
    DWORD   dwFlags;
    DWORD   time;
    ULONG_PTR dwExtraInfo;
} MOUSEINPUT, *PMOUSEINPUT, FAR* LPMOUSEINPUT;

In 64-bit Windows, the LONG and DWORD members are four bytes, but the dwExtraInfo is a ULONG_PTR, which is eight bytes on a 64-bit machine. Since Windows assumes /Zp8 packing, the dwExtraInfo must be aligned on an 8-byte boundary, which forces four bytes of padding to be inserted after the time to get the dwExtraInfo to align properly. And in order for all this to work, the MOUSEINPUT structure itself must be 8-byte aligned.

Now let's look at that INPUT structure again. Since the MOUSEINPUT comes after the type, there also needs to be padding between the type and the MOUSEINPUT to get the MOUSEINPUT back to an 8-byte boundary. In other words, the offset of mi in the INPUT structure is 8 on 64-bit Windows, not 4.

Here's how I would've written it:

// This generates the anonymous union
[StructLayout(LayoutKind.Explicit)] struct INPUT_UNION {
  [FieldOffset(0)] MOUSEINPUT mi;
  [FieldOffset(0)] KEYBDINPUT ki;
  [FieldOffset(0)] HARDWAREINPUT hi;
};

[StructLayout(LayoutKind.Sequential)] struct INPUT {
  int type;
  INPUT_UNION u;
}

I introduce a helper structure to represent the anonymous union that is the second half of the Win32 INPUT structure. By doing it this way, I let somebody else worry about the alignment, and it'll be correct for both 32-bit and 64-bit Windows.

static public void Main()
{
  Console.WriteLine(Marshal.OffsetOf(typeof(INPUT), "u"));
}

On a 32-bit system, this prints 4, and on a 64-bit system, it prints 8. The downside is that you have to type an extra u. when you access the mi, ki or hi members.

input i;
i.u.mi.dx = 0;

(I haven't checked what the PInvoke Interop Assistant comes up with for the INPUT structure.)

In the initial designs for the Start menu, the list of most-frequently-used programs on the Start menu would be completely empty the first time you opened it. This was perfectly logical, since you hadn't run any programs at all yet, so nothing was frequently-used because nothing had been used at all! Perfectly logical and completely stupid-looking.

Imagine the disappointment of people who just bought a computer. They unpack it, plug everything in, turn it on, everything looks great. Then they open the Start menu to start using their computer and they get... a blank white space. "Ha-ha! This computer can't do anything! You should have bought a Mac!"

(In usability-speak, this is known as "the cliff": You're setting up a new computer, everything looks like it's going great, and then... you're staring at a blank screen and have no idea what to do next. The learning curve has turned into a precipice.)

The original design attempted to make this initially-blank Start menu less stark by adding text that said, roughly, "Hey, sure, this space is blank right now, but as you run programs, they will show up here, trust me."

Great work there. Now it's not stupid any more. Now it's stupid and ugly.

It took a few months to figure out how to solve this problem, and ultimately we decided upon what you see in Windows XP: For brand new users, we create some "artificial points" so that the initial Start menu has a sampling of fun programs on it. The number of artificial points is carefully chosen so that they are enough points to get the programs onto the Start menu's front page, but not so many points that they overwhelm the "real" points earned by programs users run themselves. (I believe the values were chosen so that a user needs to run a program only twice to get it onto the front page on the first day.)

Note that these "artificial points" are not given if the user was upgraded from Windows 2000. In that case, the points that the Windows 2000 Start menu used for Intellimenus were used to seed the Windows XP point system. In that way, the front page of the Start menu for an upgraded uses already reflects the programs that the user ran most often on Windows 2000.

In the initial release of Windows XP, the "artificial points" were assigned so that the first three of the six slots on the most-frequently-used programs list were chosen by Windows and the last three by the computer manufacturer. If your copy of Windows XP was purchased at retail instead of preinstalled by the computer manufacturer, or if the computer manufacturer declined to take advantage of the three slots offered to it (something that never happened in practice), then Windows took two of the three slots that had been offered to the computer manufacturer, leaving the last slot blank. That way, the very first program you ran showed up on your Start menu immediately.

In Windows XP Service Pack 1, the assignment of the six slots changed slightly. Two of the slots were assigned by Windows, one by the United States Department of Justice, and the last three by the computer manufacturer. (Again, if you bought your copy of Windows XP at retail, then two of the computer manufacturer slots were assigned by Windows and the last was left blank.)

A few random links that I've collected over the last six months.

• Catalog Choice helps you get off the mailing lists of all those catalog companies. The DMA's Mail Preference Service is a bigger hammer, addressing all unsolicited mail from member organizations. (via.)
• The Opposite of Backup and other stories of "backups that didn't" reminds me of somebody who has a policy of, once a week, asking for a randomly-selected file to be restored from backup. "It is amazing what you learn by doing this." I've never tried doing it here at work, though I've considered it on occasion.
• Microspotting tracks down The Golden Helmet.
• Seized alcohol used as fuel in Sweden.
• Battlefield deception by the U.S. Army's 23rd Special Troops, whose job it was to create fake armies to fool enemy reconnaisance. The story reminded me of Jasper Maskelyne, who performed similar tricks for the British, including moving Alexandria Harbor and making the Suez Canal disappear. (I note for completeness that the reports of Maskelyne's derring-do have been disputed.)
• Researchers order medical products advertised via spam.
• It's an XXL world after all. MiceAge reports (third story) on the reason why the Small World ride will be closed for renovations, and why Scandinavia is partly to blame for you having to endure the stupid song while you wait for the jam to be cleared.
• Next time you're stuck at Disneyland waiting for the Haunted House or Pirates of the Caribbean ride to reopen, it might be because somebody scattered human ashes all over the ride, and the staff is busy cleaning it up. (via.)
• The University of Washington surplus property public store sells off property from various university departments. The online inventory is kept up to date, and you never know what you're going to find.
• Picking Up Girls Made Easy! Man, if only I'd known about this instructional audio set earlier!
• Pictures and narration from a randomly-selected Hong Kong wedding. A glimpse into another culture. The tea ceremony, the suckling pig, the three-dress rule, the funny poem, the dummy cake... (Okay, not all of those elements are traditional.)
• Shawn Travers fill in some of the details on how to create a bootable Windows DVD.
• Hand over the document and nobody gets hurt: Watch a security researcher analyze some malware script.
• Mattias Lindberg shows us how to use the moon as a compass.

And then the obligatory plug for my column in TechNet Magazine, which, despite the fact that Microsoft's name is on the magazine cover, does not establish the official Microsoft position on anything.

• The Healing Powers of Safe Mode. (See also the earlier blog entry with bonus content.)
• Windows Explorer Doesn't Do Text. Another entry where I failed to come up with a catchy title.
• The Known DLLs Balancing Act.
• Calling Dr. Watson.
• The File System Paradox. (The editors got spellcheck-happy and "corrected" my Web 2.0 names from Raymnd and Chenster to Raymond and Chester, which kind of ruins the joke.)
• A Joystick Need Not Be a Microwave. The original title had the word "burrito" in it. Oh, and here's the blog entry from The Audio Fool that inspired the column. (There's also a follow-up.)

Let's take another look at the basic principle that determines which programs show up in the Start menu:

Each time you launch a program, it "earns a point", and the longer you don't launch a program, the more points it loses.

If you stare at this long enough, you might see a hole in this principle: What about a program that you launch once and keep running all the time? According to the rule, this program would "earn a point" when you first launched it, and then it would gradually lose points⊜ even though you clearly use this program frequently. (Here, "frequently" is an understatement for "all the stinking time!")

Thus was born another fine-tuning rule: For each consecutive day◊ a program was kept continuously running, it "earned a point" as if you had launched it yourself.

This little "feeding the program points under the table" was enough background radiation to keep the program afloat in the points race, but not so much as to overwhelm the programs that you actually launch frequently. After all, if you keep the program running all the time, the Start menu didn't have to give the program high placement. Most of the times you open the Start menu, you don't need to launch that program; it's already running. The program just needed to be kept from dying out completely.

Footnotes

⊜In extreme cases, it might even drop all the way to zero, at which point the program would act like you'd never run it at all!

◊"Day" here is shorthand for a more complicated definition (taking into account idle time), the details of which are not relevant.

We already saw that programs in the pin list are pruned from the most-frequently-used programs list because they would be redundant. Another fine-tuning rule was introduced after the initial explorations with the new Windows XP Start menu: Programs with specific "noise" names are removed from consideration.

Many "noise" programs were showing up as frequently used because they happened to be shortcuts to common helper programs like Notepad or Wordpad to display a "Read Me" document. These shortcuts needed to be filtered out so that they couldn't be nominated as, say, the Notepad representative. The list of English "poison words" is given in Knowledge Base article 282066.

(Incidentally, a program can also register itself as not eligible for inclusion in the front page of the Start menu by creating a NoStartPage value in its application registration.)

We'll see in the epilogue that Windows Vista uses an improved method for avoiding the "unwanted representative" problem.

Last time, we traced an IP_ON_HEAP failure to a shell extension that used an older version of ATL which was not DEP-friendly. But that led to a follow-up question:

Why aren't we seeing this same crash in the main program as in the shell extension? That program uses the same version of ATL, but it doesn't crash.

The reason is given in this chart. Notice that the default configuration is OptIn, which means that DEP is off for all processes by default, but is on for all Windows system components. That same part of the page describes how you can change to OptOut so that the default is to turn on DEP for all processes except for the ones you put on the exception list. There's more information on this excerpt from the "Changes to Functionality in Microsoft Windows XP Service Pack 2" document.

The program that comes with the shell extension is not part of Windows, so DEP is disabled by default. But Explorer is part of Windows, so DEP is enabled for Explorer by default. That's why only Explorer encounters this problem.

(This little saga does illustrate the double-edged sword of extensibility. If you make your system extensible, you allow other people to add features to it. On the other hand, you also allow other people to add bugs to it.)

The saga of the DEP exception is not over, however, because it turns out I've been lying to you. More information tomorrow.

Welcome to CLR Week 2009. As always, we start with a warm-up.

The String.Format method doesn't throw a FormatException if you pass too many parameters, but it does if you pass too few. Why the asymmetry?

Well, this is the type of asymmetry you see in the world a lot. You need a ticket for each person that attends a concert. If you have too few tickets, they won't let you in. If you have too many, well, that's a bit wasteful, but you can still get in; the extras are ignored. If you create an array with 10 elements and use only the first five, nobody is going to raise an ArrayBiggerThanNecessary exception. Similarly, the String.Format message doesn't mind if you pass too many parameters; it just ignores the extras. There's nothing harmful about it, just a bit wasteful.

Besides, you probably don't want this to be an error:

if (verbose) {
  format = "{0} is not {1} (because of {2})";
} else {
  format = "{0} not {1}";
}
String.Format(format, "Zero", "One", "Two");

Think of the format string as a SELECT clause from the dataset provided by the remaining parameters. If your table has fields ID and NAME and you select just the ID, there's nothing wrong with that. But if you ask for DATE, then you have an error.

Recall that a corrupted program sometimes results in a "Program too big to fit in memory" error. In response, Dog complained that while that may have been a reasonable response back in the 1980's, in today's world, there's plenty of memory around for the MS-DOS emulator to add that extra check and return a better error code.

Well yeah, but if you change the externally visible behavior, then you've failed as an emulator. The whole point of an emulator is to mimic another world, and any deviations from that other world can come back to bite you.

MS-DOS is perhaps one of the strongest examples of requiring absolute unyielding backward compatibility. Hundreds if not thousands of programs scanned memory looking for specific byte sequences inside MS-DOS so it could patch them or hunted around inside MS-DOS's internal state variables so it could modify them. If you move one thing out of place, those programs stop working.

MS-DOS contains chunks of "junk DNA", code fragments which do nothing but waste space, but which exist so that programs which go scanning through memory looking for specific byte sequences will find them. (This principle is not dead; there's even some junk DNA in Explorer.)

Given the extreme compatibility required for MS-DOS emulation, I'm not surprised that the original error behavior was preserved. There is certainly some program out there that stops working if attempting to execute a COM-style image larger than 64KB returns any error other than 8. (Besides, if you wanted it to return some other error code, you had precious few to choose from.)

A few random links that I've collected from other Microsoft bloggers.

• The Microsoft fallacy by Nadyne Mielke.
• The history of cards.dll as told to Adam Nathan.
• Hur hamnade den svenska versionen av Internet Explorer 7 med felaktig menytext för kommandot "Nytt Fönster"? (How did the Swedish version of Internet Explorer 7 wind up with incorrect menu text for the "New Window" command?) You'll have to be able to read Swedish to understand the explanation.
• Group Policy Setting Reference for Windows Vista via the Microsoft Consulting Services infrastructure team in Ireland (wow what a mouthful).
• What does it mean when I'm asked whether I'm on a Home, Work, or Public network? Matt McSpirit explains.
• If you want to learn more about Timewarp (which has saved me on more than one occasion), you can check out a Channel9 presentation with the Timewarp team.
• Add .url as a searchable file type and you can search your Favorites.
• EldarM describes The five worst problems of home-grade routers: Home networks that gave Windows Home Server the heebie-jeebies.
• Heather Leigh has tips for increasing the odds of getting your resume in front of the hiring manager.
• The Filing Cabinet (a blog on file services and storage) provides tips on using offline files with network attached storage and instructions on how to update your offline files cache when the server name changes.
• Tim Newton explains the relationship among sessions, desktops, and window stations.
• "If you don't get a naked dwarf in my office by this afternoon, I'm coming over there to get one for myself." Taken out of context, that's a pretty strange statement to make in a business meeting. James O'Neill explains the circumstances.
• Do black pixels really consume less energy? Danny Thorpe pulls out his power meter and finds out.
• Robert Hensing introduces us to hybrid sleep.
• Chris Jackson opines on software personalization.
• Arlindo Alves shows how to use group policy to block the use of USB storage devices aside from the ones you explicitly permit.
• Vishal Singh from Windows Application Experience (a group which includes the Application Compatibility team) teaches us about virtualized files, a mitigation for applications that assume administrative privileges. Additional application compatibility resources can be found on TechNet.
• The Terminal Server team blog shows off the new multimonitor remote desktop client.
• Kieran Snyder reminds us that word breaking is more than just looking for spaces. Jonas Barklund takes a break from correcting my Swedish grammar to give a concrete example of the difficulties with word breaking in Japanese.

Prerequisites: Basic understanding of assembly language.

When you debug through some managed code at the assembly level, you may find that there are an awful lot of nop instructions scattered throughout your method. What are they doing there; isn't the JIT smart enough to remove them? Isn't this going to slow down execution of my program?

It is my understanding that¹ this nop instructions are inserted by the JIT because you're running the program under the debugger. They are emitted specifically so that the debugger can set breakpoints in locations that you normally wouldn't be able to. (For example, they might represent a line of code that got optimized out or merged with another line of code.)

Don't worry. If there's no debugger, the JIT won't generate the dummy nops.

Nitpicker's Corner

¹As with all statements of alleged fact, this statement is an interpretation of events based on observation and thought and does not establish a statement of the official position of the CLR JIT compiler team or Microsoft Corporation, and that interpretation may ultimately prove incorrect.