The same question came up two times in 26 minutes (on the same discussion list, no less), so I figured I’d answer it once here as that seems a reasonable indicator that others may have the same question.
What is the deal with shimming on Windows Vista 64-bit?
Well, it turns out it’s a bit of a mixed story, so let’s go through it.
First, Compatibility Administrator itself runs just fine on x64. It’s a 32-bit app, so it’s running on WOW64, but that’s OK. You can run the tool to create custom shim databases to run on either x86 or x64.
Now, let’s talk about the shim engine. It’s installed on x64, and it supports shimming both 32-bit and 64-bit binaries. So far, so good.
But once we start talking about the custom shim databases you can create today using Compatibility Administrator, the story is not complete today.
You see, the custom shim databases you create (whether you create them on x86 or x64) will only shim 32-bit applications. So, while the platform supports shimming 64-bit applications, the tools don’t give you the ability to do so.
So, if you have a native 64-bit application that isn’t working on Windows Vista, then you are unable to use shims today in order to mitigate the problems you discover.
I haven’t come across any problems with native code doing this, but I have come across problems with managed code. There just isn’t much native code that is compiled for 64-bit (and what is available tends to be fairly recent). Managed code, however, defaults to compiling for “any CPU” – which means that the JIT compiler will compile native 64-bit code for managed code when it’s running on 64-bit. So, you can fix it on x86, but not on x64, even though the same app runs on both.
Well, that’s not very cool. That just might discourage you from moving to x64 if you have a broken managed code application, eh?
Well, there are two solutions for that. Obviously you could recompile the application and change the compiler flags to target 32-bit only (but if you’re doing that, you may as well fix the bugs!). If you can’t recompile, however, you’re not stuck. Just use corflags.exe to set the 32BIT flag, and you’ll be on your way.
By the way, if you really want to get under the covers, you can. The underlying cause is that the XML we generate doesn’t have the (optional) OS_PLATFORM attribute. Try it yourself – if you look in Process Monitor, you’ll see that compatadmin creates an XML file in appdata\local\temp. Of course, it deletes this file when it’s done, so you’ll want to catch it before we’re done with it – a quick glance at the stack tells you that you’ve got msxml3.dll in the stack when you’re creating this, so if you bm on msxml3!* and start stepping until the call to CloseFile, you can go into the temp directory and pull out the XML to see for yourself.