So, the other day I was investigating a visual issue in an application. On Windows 2000, the background of the form was white. On Windows Vista, in Windows Classic mode, the background of the form was white. But, for whatever reason, in either Windows Vista Aero or Windows Vista Basic, the background of the form was black. And it was a little hard to read that way.

The first thing I did was try to find something out about the application, but in this case is was an in-house application. The best I could do was to poke around the application and gather some facts so I could form my hypothesis on what could be going wrong, and how I could fix it.

A quick glance at the output of lm in the debugger revealed that msvbvm50.dll was loaded. OK, so we have a Windows 2000 application built on Visual Basic 5.0. This strongly suggested that the application wasn't leveraging visual styles and the functions in uxtheme.dll to pick up colors (which also meant that the FakeLunaTheme shim probably wouldn't do any good, and in fact it didn't - I tried anyway).

So then I started going down the road of setting breakpoints on functions such as GetSysColor to see what is going on, and then I was attempting to mentally translate the code from ASM to VB5. Oh, and I was doing this with public symbols too, which made it even more fun.

Then, I got a clever idea. What if I didn't have to do the translation in my head? I mean, Lutz Roeder's awesome Reflector for .NET is unbelievably useful for prying secrets from managed code, I wondered if anybody had ever done the same for VB5/6 code? It seemed like it could be possible...?

Some quick web searching later, and I came up with VBReFormer. Ah - and they have a free edition! Let's give that a go.

Literally 2 seconds later, I was at the form, looking at properties, and the answer was right in front of my face. The developer has set the form background color to be the system color for Title Bar Text. (Seriously.) And, alas, we don't have a shim for that one. But we could just change the title bar text, the easiest way being to switch to Windows Classic mode when using this app. Or, just put up with the fact that the black background didn't look as nice (at least the text was blue, so it wasn't unreadable as it would have been if it were black text).

And this is yet another example of what I am always preaching. There is a certain geek pride in going after a deep understanding of what's going on. But there are no prizes for spending the most time in a debugger. In fact, it's a great way to make time vanish if you aren't careful. Use the tool to get the answer to your question - prove or disprove a hypothesis - and then get out. If there is an easier tool, use it. If you think "wow, I bet this tool would be awesome if it existed" then poke around - it very well may.

Most of all, don't be afraid to cheat. Nobody is counting the number of times you typed bp in a debugger, or how quickly you can ferret out the calling convention to sort out arguments with public symbols. But they do tend to notice when you fix things.

Oh, and kudos to the VBReFormer team for saving me a boatload of time. I don't know if there is a better tool, I'm not an expert in that space and this is the first and only one I looked at. But I was thrilled that I didn't have to hand over that portion of my life to a debugger.

What is probably the most common question I get regarding application compatibility?

"Can you just give me a list of all of the applications that work on Windows Vista?"

Sounds easy, huh? Well, it's not quite that easy.

First of all, there is the obvious problem of claiming that any software works at all. If anybody can come up with a full proof way of proving that any piece of software at all is bug-free, then that person is going to be very rich indeed. Even if we fully test software as we would use it, that doesn't mean we use it how you would use it. If I use some CAD software to design houses again and again and it works flawlessly under a huge amount of testing, what happens if the bugs exist when you use it to design something very small (like a screw) or something with a lot more vertices, such as a piston engine? We just don't know.

You would be surprised at how many "compatibility problems" I have investigated where, after uncovering the root of the problem, I can say definitively that the software could have never worked on any operating system. Sure enough, we run it on their current operating system, and it doesn't work there either.

Since you can't effectively just have people imagine how to use something and test it that way, our idea was to enhance any testing and bug submissions that we investigate with a community feature, which we added to the Application Compatibility Toolkit. But it turns out that this was harder than it seems. Why?

Well, first of all, people want this data when they're getting started. So, while we would exchange your votes for other peoples' votes, most people would perform this exchange before they had voted for anything. So, you'd exchange your "I don't know yet" vote for what we had out there. And how many people would go back at the end to send up their votes after they were done testing? Well, quite a few actually, but not as many as everyone would like. So that's one challenge.

But the bigger problem is that the majority of enterprises don't really much care if it works or not if, when something does go wrong (remember, works doesn't mean bug free), they call the vendor and the vendor will just hang up on them if you're using Windows Vista. They want "works" and "supported".

(This is another conversation I have a lot. "Can you figure out how to get this to work." "I see that this software directly controls your company's finances - if it takes some shims to work, are you really going to want to run it that way?" "Oh, no, it has to be supported." "Then why are we bothering to fix up this version?")

So we had another idea that we thought was quite clever. We already had infrastructure in place to allow our partners to certify an application as Certified for Windows. So, why not just leverage that. But, instead of making you pass a set of quality and reliability criteria that needs to be verified by a third party vendor (oh yeah, and that costs money), why not just let the partner self-certify that it works and that they'll support it on Windows Vista? Let's make it so easy that everybody of course would do it.

Except, of course, they didn't. We did make it super easy, but we were a middle man that not every partner saw as necessary in the equation. They already have a relationship with you, why do they have to tell us? And it probably smelled a little "official" what with the standardized logo and all. So, despite being a pretty good idea, it didn't get the uptake it needed to be super useful either.

Two good ideas, and still not a whole lot of data. Hrph.

So, we had another idea. If not enough people are coming to us, we'll go to them. So we hired some folks to go ferreting around vendor web sites looking for statements of support. Hey, we're the ones who have the shiny new OS, we're happy to do the legwork. And that's what we're still doing. Taking the existing Works With and Certified For data, and for the rest, scouring our partners' web sites looking for support statements. If they have one saying the support Windows Visa, we'll tell you about it, and give you a link so you can go and see it yourself!

The list.

And hence http://www.appreadiness.com/default.aspx was born.

Except it's not the entire list. The community data isn't on there. Nor can it really be, since we have a privacy policy and statement within ACT. You get this data only when you have that application. So, our next step is to get this data into ACT so you can see all 3 things at one swoop, correlated to the inventory data of what you're running. What do other people say? What does the vendor say to us? And what does the vendor say to you on their web site.

(We have also added a new Community section to appreadiness.com to try and capture some of this.)

But, if you're not using ACT, you can get 2 out of 3, and we're still working on it. We're working really hard to push out any data we can to help you out. There's no reason for everyone to do the same web searches - we'll do it once, and let you get it from there.

One caveat I would add - there are no stipulations on what can go on this list regarding the configuration they support. The developers could say it's supported on Windows Vista only if running as full admin with UAC disabled. They could say it's supported on Windows Vista only if you run with a certain security policy. And, if your standard image doesn't allow you to run this way (say you are targeting a standard user desktop, which a lot of people are now that we have so much great support for standard user mitigations in Windows Vista) then perhaps you don't want it supported *that* way - you'll want to read the support statement we link to in order to really understand it.

How are we doing? Is this the list you want? When we get this fully tied in with ACT, are we getting closer to what you need? Is this list enough? How could we do better? In an ideal world, exactly one person would have to investigate compatibility, and the rest of the world could leverage that knowledge. I don't think we'll ever get to the ideal, but I think we're a lot closer now to keeping this at a minimum and letting you just verify the scenarios that are important to you as well as assessing the level of vendor support.

Because we want to give you "the list." Even if it's hard work.

I'm working with a customer on their migration from Windows 2000 to Windows Vista. And, as you might expect, they have built up quite a collection of software that explicitly checks to see if it is running on Windows 2000. Not Windows 2000 or higher, but Windows 2000.

So, we're using a lot of Win2000SP3VersionLie.

Along the way, we ran into a bit of a snag. We had an application that was failing when we reported the version number as Windows Vista, but when we applied the version lie, the application was crashing. Specifically, it was happening when we clicked File -> New.

Since it was crashing, I pulled out the debugger to see what was going on. I suspected that we had just gotten past the version check and hit another compatibility issue, but as it turned out, the version lie was the problem. The stack trace?

0:000> k
ChildEBP RetAddr 
WARNING: Stack unwind information not available. Following frames may be wrong.
0012b9c8 68f08728 GrooveUtil!_GrooveTraceKeyDefined+0x8ea
0012ba00 68f082b8 GrooveUtil!_GrooveTraceKeyDefined+0x7db
0012ba34 68f09cac GrooveUtil!_GrooveTraceKeyDefined+0x36b
0012bacc 68f0908c GrooveUtil!GCError::TerminateGrooveProcess+0x296
0012bae8 68f09fe8 GrooveUtil!GCError::GCError+0x1f
0012bb20 68ef104d GrooveUtil!GCError::int_RawError+0x26
0012bb60 68ef3eef GrooveUtil+0x104d
0012bb74 68f1bf38 GrooveUtil!GCChar::IsSpaceChar+0x7c7
0012bb90 68ef1c9e GrooveUtil!GCOSVersionInformation::IsWindowsVistaOrLater+0x6c
0012bb98 68ef1f21 GrooveUtil!DllUnregisterServer+0x68
0012bba8 68ef1d95 GrooveUtil!GCUtilDLL::GetSystemInfo+0x272
0012bbd4 68f6546e GrooveUtil!GCUtilDLL::GetSystemInfo+0xe6
0012bbf8 757a1742 GrooveUtil!GCoUnknownEnum2::CreateInstance+0x131c1
0012bc00 68f5433c MSVCR80!_initterm+0xf
00000000 00000000 GrooveUtil!GCoUnknownEnum2::CreateInstance+0x208f

...wait a minute - what is Groove doing here?

Groove has a shell extension that pops up whenever you use Windows Explorer. (You never know when you want to right click and select Groove Folder Synchronization - even when you're just using a common dialog to pick a new file for a completely unrelated application.)

And it turns out that Groove doesn't like to be lied to, and that lying about being Windows 2000 makes it AV.

What is the solution? Simply exclude GrooveUtil.DLL from the version lie shim. In Compatibility administrator, with the Windows 2000 version lie selected, press the Parameters button. In Module Name, enter GrooveUtil.DLL, press the Exclude radio button, and then click add. Save it, and then install the new custom shim database.

No more crash.

So, keep in mind - when you version lie to an app, you version lie to all of the DLLs in the app - even ones that it may not have direct control over. Like Groove.

(For what it's worth, the XP SP2 version lie doesn't cause a Groove crash.)

Oh, and for those developers contemplating writing code to check and see if the version number we report really is the version number, and looking for alternate ways to verify the return value, please don't. Some day, you may need a version lie, and you'll just make it harder for us to get it to you. If you can avoid an AV, great, but there's no reason to go nuts with those test matrices. We'll just exclude you...

When addressing difficult application compatibility issues, there are some things you just can't fix. Some applications will never work on Windows Vista, shims or no shims.

For example, I was debugging one application that happened to be written in Visual Basic 4.0 16-bit. (Ah, the days when you would pick the bitness in Visual Basic. I had almost forgotten about that!) I was watching it error out with Error 70 - Access Denied and trying to sort out what the problem was. A little help from Process Monitor revealed that there were no file or registry keys that were the cause of the problem. I was next thinking it might be related to COM, when a colleague (whose name rhymes with Orange) suggested looking for windows hooks. Sure enough, a bp on user32!setwindowshookexa turned up a hit, I investigated the arguments, and it was setting up a hook of type WH_JOURNALPLAYBACK.

Uh oh.

Now it all made sense. The bug was opened detailing that you have to press the Enter key after entering your data instead of pushing the submit button.

Yep - it's SendKeys. Rather than just calling the method, the developers pushed the button using SendKeys to get the button to call the method. (Really, I don't understand why anybody would do that.)

And that's something we can't fix. We could give it UIAccess, but we really needed the other mitigations to keep the application working, so that didn't work at all.

But I digress - the purpose of this post isn't to gripe about some SendKeys silliness that somebody wrote in the previous century. Rather, I wanted to discuss one small bit of the next thing we talk about when we come across an application that doesn't work.

"Well, your options are to use Virtual PC / Server, or Terminal Services."

Actually, let me back up. Lots of people also throw in Microsoft SoftGrid. Somewhere along the line, the way in which we have overloaded the notion of application compatibility has caused some folks to conflate the different meanings of compatibility. SoftGrid is great for app to app compatibility. It's not so hot for app to OS compatibility. There are some things it will do. However, to my knowledge, there isn't anything it could do that I couldn't similarly achieve using shims. (Prove me wrong.) So, if I tell you it can't be shimmed, this usually isn't a great option to bring up, even though it is exceptionally cool at the things it does do well. (It's not a product shortcoming when it doesn't do everything - I'm writing this using Windows Live Writer, which is downright dreadful for doing accounting for large enterprises, but it's a great blog post editor.)

But I digress yet again. (I do that sometimes.)

Let's zero in on Terminal Services. Specifically, some of the issues you may or may not encounter.

If an application doesn't work on Windows Vista, then it's not super likely to work on Windows Server 2008 either, terminal services or no. However, if it did work on Windows XP, then it just may work on Windows Server 2003. However, it may need a little bit of help. Here, you can use the Version Lie shims to lie about being a client version of the operating system. Yes, shims work just fine on Windows Server. Now, we don't typically apply them to server applications (even though the infrastructure is on server) - I mean, do you want a shimmed up old application to help you deliver 5 nines uptime? Because that may not work very well. But for shimming up client applications to run in terminal services, it can work great.

It can also help if it runs fine on Vista but not Windows Server 2008 because it needs that version lie.

All of that storytelling just to say "Hey, don't forget, you can shim things to work on Windows Server too, if you're trying to do terminal services." What can I say - I'm not feeling particularly terse tonight.

(Update: added the "s" to the SetWindowsHookExA API.)