Chris Jackson's Semantic Consonance : Shims

Understanding the AdditiveRunAsHighest Flag on Windows 7

Chris Jackson — Thu, 08 Oct 2009 23:32:35 GMT

This post corrects an error from a previous post.

Back when I was explaining the changes in the operating system layers for Windows 7, I incorrectly described the AdditiveRunAsHighest. Since RunAsHighest is already confusing enough, and AdditiveRunAsHighest is even more confusing (even I got it wrong), I want to make sure I actually get it right (and apologize for not having done so before).

AdditiveRunAsHighest requests that the application receives the RunAsHighest flag if and only if nobody else has requested a higher level of elevation.

That means we will overrule a manifest if that manifest is asInvoker, but we will not overrule a manifest if that manifest is requireAdministrator. It also means we will overrule a layer if that layer requests RunAsInvoker, but we will not overrule a layer if that layer requests RunAsAdministrator.

In short, it means that this flag will only be used to increase your level of elevation (to highestAvailable) and will never be used to decrease it (from requireAdministrator).

The reason why this flag exists? Setups. If we think you are a setup, then the setup detection logic applies the VistaSetup layer, which has (as you might imagine) RunAsAdministrator. We don’t want setups to fail for every standard user out there – they still need to prompt. So, this layer will not cause standard users to stop seeing prompts for setups (or manifested apps for that matter). If we had used the existing RunAsHighest flag, then it would have broken setup detection for standard users – and we love our standard users.

Why Do Some Operating System Modes Include AdditiveRunAsHighest While Others Do Not?

Chris Jackson — Fri, 21 Aug 2009 00:54:20 GMT

A little while back, those of us who explain application compatibility for a living and try to help people get their arms around it ran up against those who implement it in the product on the scale of … the whole earth.

Those of us who explain things for a living really prefer (really, really prefer) when the system is internally consistent, because that makes it easier to explain.

Those who build systems really work to make the investments they can, with finite resources, to fix the greatest percentage of applications they can.

And sometimes those goals are not in alignment.

Case in point: remember back in April when i was talking about how we added the new AdditiveRunAsHighest shim to our operating system layers? Well, it turns out that we didn’t touch quite all of them. The following layers have AdditiveRunAsHighest applied:

Win95
Win98
WinXP
WinXPSP1
WinXPSP2
WinXPSP3

The following layers do not have it applied:

NT4SP5
Win2000
Win2000SP2
Win2000SP3
WinSrv03
WinSrv03SP1
VistaRTM
VistaSP1
VistaSP2
WinSrv08SP1

What’s the rhyme or reason here? Well, you can start to tease out some logical ones. First, most people don’t use compatibility modes on server apps. (They may use them on a server, such as when you use terminal services apps, but if you have a high-throughput server application, shimming it up and running it unsupported is probably not high on your list of acceptable mitigations. So, remove Server 2003 and Server 2008. Next, if it worked on Windows Vista, it has already seen UAC, so we don’t really need to have it there. But … what about NT4 and Windows 2000? If those were included, couldn’t we just say, “ever client operating system prior to Windows Vista includes this shim”?

Yes, we could.

And wouldn’t the same arguments that made sense for the other ones make sense here?

Yes, they would.

So, what gives?

Well, in the game of probability (fix the most apps), internal consistency was not the focus. Fixing the largest number of apps was. And there simply were not enough apps that needed the Windows NT 4 and Windows 2000 modes to bring it on the radar that they needed them too. And those of us who do explain these things for a living didn’t notice it until July, when the bug bar was way too high to get this fix in (since it doesn’t actually block you from getting things done, it just makes it harder to explain why the system behaves the way it does).

So, instead of a nice, logical rule, you have a list to memorize. 95, 98, and XP get AdditiveRunAsHighest. Everything else does not. Sorry for making you memorize a list – I’d much rather you were memorizing rules that you could logically explain, but so it goes.

Stock Viewer Shim Demo Application - Now Available in Japanese!

Chris Jackson — Wed, 24 Jun 2009 06:14:00 GMT

I have had the Stock Viewer Shim Demo application available for over a year now, and I'm delighted at how much impact it has had. I see people using this all the time! But previously it was available only in English. Well, no more - it has now been (mostly) localized to Japanese! Enjoy.

PCA Changes for Windows 7: How To Tell Us You are Not an Installer, Take 2 (because we changed the rules on you)

Chris Jackson — Fri, 19 Jun 2009 02:18:48 GMT

I have an ongoing conversation with a customer whose application is now popping up a Program Compatibility Assistant dialog box, where it didn’t used to before (on either Windows Vista or Windows 7 beta). What’s going on?

Well, when I cracked the resources for the binary, you can spot right away that it’s going to trigger installer detection, given the presence of the word “Install” in multiple places. You can also turn on shim diagnostics and see that it’s wiring up with GenericInstaller.

But why wasn’t it popping PCA on Windows Vista? Is this a new PCA scenario? I explored that, and discovered that it was Scenario #1 – an app that we detect as a setup doesn’t leave anything behind in Add/Remove Programs. That suggests possible failure.

So, existing scenario, new PCA prompt, and the final bit of evidence: the application doesn’t prompt for UAC Elevation on either Windows Vista or Windows 7. You should now be able to connect all of the dots.

The application has a UAC manifest.

On Windows Vista, the presence of a UAC manifest disables PCA. On Windows 7, that’s not good enough. You have to have one of the new-fangled Windows 7 GUIDs in your manifest to avoid PCA annoyance.

So, if you’re following along, here’s what happened:

Customer has an app that looks suspiciously enough like an installer that the operating system is trying to intervene and make things work
Developer notices that, doesn’t want/need the elevation, wonders how to make it go away, and discovers our documentation saying how you can turn off setup detection by including a UAC manifest – being good law-abiding citizens, they do exactly that (at their own expense)
Unbeknownst to the developer, we came along behind their back and changed the rules to make our previous guidance null and void, and once again we’re annoying their users with dialog boxes that make the developer look bad
Now the developer, if they wanted to make their users happy, would have to stop what they are doing, update their currently shipping app to include a manifest for an operating system that isn’t even released yet, so that Windows 7 can be more successful by having a more compatible and friendly OS (at their own expense)
Repeat for every subsequent version of Windows, as the Windows 7 GUID would no longer indicate Windows 8 compatibility either when it comes along

Does anyone else see anything wrong with this picture?

I’m trying to see what we can do about this, because I don’t think we’re being fair to developers right now. I believe that an asInvoker manifest is a pretty darned good indication that you’re not an installer, both from the perspective of GenericInstaller and PCA. I have absolutely no idea why somebody would disagree.

In the interim, rather than gripe about this (if you only know how many not-so-polite words and pejorative phrases I have already deleted from this blog post, you’d likely be very disappointed in me), I figured I’d tell you how you can fix it until whenever / if ever we address this:

Shim your application with SpecificNonInstaller.

I know of no other way to tell setup detection that “hey, you’re wrong” that will survive into the future. Because UAC Manifests only work on Windows Vista, Windows 7 Manifests will presumably only work on Windows 7, and who knows what you’ll need in the future?

What I don’t understand about PCA is, why haven’t we learned our lesson that asking the user a question every time we’re confused isn’t the right approach? <sigh />

ComputerWorld Article on Shims

Chris Jackson — Sat, 23 May 2009 05:08:05 GMT

ComputerWorld did a story on shims featuring my session from TechEd North America – check it out: http://www.computerworld.com/action/article.do?command=viewArticleBasic&articleId=9133382

Why Custom Actions get a Windows Vista Version Lie on Windows 7

Chris Jackson — Thu, 07 May 2009 01:00:28 GMT

Those of you who write application installers using Windows Installer may have noticed a bit of a change in the behavior of version checking in Windows 7 – if you happen to be doing your version checking from a custom action. Let’s have a look.

If you’d like to follow along, here’s what you need to do. Open up Compatibility Administrator. Expand the Applications tree (this could take a minute), and then start typing Windows Installer to quickly scroll down to this entry.

First, you’ll notice a few things which we expect. We’re shimming up msiexec.exe (meaning we’re shimming up every MSI package in the world) with WRPDllRegister, WRPMitigation, and WRPRegDeleteKey. These are all of the Windows Resource Protection shims – so if you have an installer that tries to overwrite kernel32 with some really ancient version, for example, it’ll get fixed up instead of failing.

There’s NonElevatedIDriver – that’s an InstallShield fix, for MSIs created by InstallShield. So far, so good.

Oh.

And there’s VistaRTMVersionLie.

Whazza – huh? (Double takes are completely appropriate here.)

That’s right, we’re shimming up the Windows Installer application to give it a Windows Vista version lie. What does this affect? Any DLL custom action you implement that does a check to GetVersion(Ex) is now going to think it’s getting Windows Vista instead of Windows 7. Do not pass go, do not collect $200, you’ll think you’re on Windows Vista.

However, any application that uses the MSI tables and takes a look at the VersionNT property will be told the correct version of Windows. So, on Windows 7, VersionNT == 601. Just like you’d expect.

From a pragmatic perspective, this improves the overall compatibility of installers on Windows 7 to a huge degree. We did this to systematically fix apps that refuse to install on anything greater than Windows Vista. And it succeeds at this spectacularly.

From a "bonus side effect” perspective (if you like to think that way) – it does serve the effect of pushing people towards using the declarative method of checking versions in the MSI tables, which is our recommended way of checking for specific versions.

Not only is it recommended, it’s the only sensible API we have for checking a version. The others are really hard to get right. How would you check to see if a version is greater than or equal to a specific version? Well, GetVersion returns something really strange until you yank it apart (Windows 98, for example, is 0xC0000A04). How obvious is this?

dwMajorVersion = (DWORD)(LOBYTE(LOWORD(dwVersion)));
dwMinorVersion = (DWORD)(HIBYTE(LOWORD(dwVersion)));

OK, so that’s a little silly, and GetVersionEx has a structure that simplifies this. But even with the OSVERSIONINFO structures, you can do things wrong. For example, say you want to check and see if somebody is using at least Windows XP. You could easily do that as:

if (osvi.dwMajorVersion == 5 && osvi.dwMinorVerison == 1) {…

And of course that will do the trick while Windows XP is the only thing around that it works on, but as soon as we release a new version of Windows, it breaks. But let’s say you’re trying to get it right, and you ordered one of those fancy computers whose keyboard actually has a “greater than” key – you could still lose the logic. I’ve seen this many times:

if (osvi.dwMajorVersion >= 5 && osvi.dwMinorVersion >= 1) {…

That’ll work on Windows XP, break on Windows Vista, and work on Windows 7. It’s the wrong logic. Here’s what you’d have to do:

bIsWindowsXPorLater = ( (osvi.dwMajorVersion > 5) || ( (osvi.dwMajorVersion == 5) && (osvi.dwMinorVersion >= 1) ));

And now we’re complex again.

BUT – with the MSI tables, you can actually use the > key with ease. VersionNT >= 501 will work for Windows XP or greater – for anything we release into the future.

It’s declarative, so you can see the check. It’s easy to get it right. It’s got a lot going for it. Nevertheless, I’ve still seen a lot of people doing things like:

VersionNT = 500 OR VersionNT=501 OR VersionNT=502 OR VersionNT=600

I wonder what they’re going to do for Windows 7?

Now, of course, from the ivory tower we’re always saying “don’t check versions” and we don’t hear the cries that you have to. In the end, it’s a business decision, and I don’t begrudge anybody for saying they can’t afford to test and support something indefinitely on every permutation of system. That’s hard to do. I know you’re going to check for versions. But, there’s an arms race between app compat and the people who want to do the checks. There has even been a suggestion that we *never again* change the version of Windows. I think that’d be downright silly. There are valid business reasons to know what you’re running on.

For example, I’ve got a conversation with a customer going on right now where they work on Windows XP, fail on Windows Vista RTM, and work on Windows Vista SP1. Stuff like that happens. (And, since they were checking from a custom action, they were refusing to install on Windows 7, because they thought they were going onto Windows Vista RTM, where they were known to fail. Ouch.) Ideally, this would use the MSI tables to exclude known bad versions, rather than the unknown.

What I’d like to see is a way to declare supported versions in addition to known bad versions. A known bad version should be blocked – let’s be sensible here. But an unknown version, well if you let it install smoothly, then depending on your relationship with the customer would that indicate a suggestion of support? If it could give some sort of warning “note: this operating system is not supported” but still run, then app compat doesn’t need to come along and remove the launch condition or custom action or verlie the entire framework just to keep apps running.

I get what you’re going for, but I also get that people want their apps to work unless there’s a technical reason why they don’t work. Even if they’re unsupported. Because, for some people, that’s OK. Particularly consumers, many of whom don’t have a lot of money right now to be buying new stuff. Just dazzle them with amazing new features, and they’ll buy it, but people don’t like buying stuff just because you said “don’t work on the fancy new OS that your new laptop came with.”

And so it goes.

Changes to the Operating System Layers (Compatibility Modes) in Windows 7

Chris Jackson — Wed, 29 Apr 2009 02:19:47 GMT

It’s visible in the beta, but I haven’t heard a lot of people talking about this externally. Regardless, I wanted to shed some light on what happened, and add a bit of the human perspective behind the decision.

If you inspect the operating system layers (called Compatibility Modes in Compatibility Administrator), you’ll find that they contain an important new entry: AdditiveRunAsHighest. Let’s explore this a bit, because it’s pretty important what it does.

RunAsHighest

First and foremost, AdditiveRunAsHighest is RunAsHighest. (We’ll get to the Additive part in a bit.) It’s an incredibly confusing elevation flag for many folks. It basically means that, if you have the ability to elevate to a higher token, then please do. Otherwise, just stay where you are. That means, if you’re a member of the Administrators group with UAC turned on, and you’re not currently running elevated, you’ll see a prompt. If you are a standard user, you will not. If you have the SeLoadDriver privilege in your token but would otherwise be a standard user, we’ll still split your token, and if you provide the same credentials, we’ll elevate to a token that contains that privilege. (You can’t think too black and white about elevation – not everyone is cleanly either an admin or a standard user.)

I say that it’s confusing because most people don’t fully grok that. Most people, rather, believe that MMC.exe requires being an admin, and that’s why it prompts. It doesn’t require it, you just happen to have a more privileged token available so it’s going to try to use it. If you were running as a standard user, you wouldn’t see a prompt!

OK, so we’re off to a good start – the new shim is related to the most confusing elevation flag we’ve got. How have we made that more interesting?

AdditiveRunAsHighest

If you’re an IT Pro, you’ll like this: you always get to win. You see, a developer can specify, in the XML manifest for an application, the run level for that application. But you, as the IT Pro, get to overrule that. If you think the developer made a bad call with their specified run level, just specify what you think, and you’ll win.

But AdditiveRunAsHighest means that you only care to vote if the developer didn’t. So, if you find a manifest specifying run level, you’re voting to take what they asked for. If not, then you are asking for RunAsHighest.

(I guess I find that a little curious because, if you know enough about Windows Vista to add a runlevel to the manifest, it’s not entirely clear why you would need an XP or earlier shim, but so it goes.)

Putting it all together

What this basically means is that, if you were running as a standard user, and you didn’t work, the RunAsHighest will leave you as a standard user, and unless file/registry virt fixes you, you’re still broken. No regression. But, if you were previously a full admin, and now you’re a protected admin, you’ll prompt to get back to full admin, and away you go.

If only it were that simple. But, realistically, nobody is super happy with this outcome. This doesn’t represent what we wanted to do, it represents what we could do in the parameters we were given (which included, of course, the parameter of, “we have to ship an operating system”).

Elevation fixes ~20% of broken applications that used to work on XP

There are truths, half-truths, and statistics. This is a statistic. That’s the bright side. The down side is that the other 80% of applications aren’t fixed by elevation (actually it’s more like 50% when you incorporate this PLUS other fixes in the layer, so that’s a little unfair of me), yet they are prompting you anyway! But, in the absence of other alternatives, it was decided that people are more annoyed by apps not working than they are by prompts. Oh yeah, we know you’re still annoyed by prompts – we just think you’re more annoyed by broken apps.

And, when it came down to it, even if the number of apps fixed were 10%, it would have been a no-brainer for the team. Broken apps are a serious downer.

What does it mean for me?

Well, if you’ve built up a custom shim database that contains operating system emulation layers, their behavior is going to change. You may want to edit them. I typically recommend that you chose only the specific shims that you need, rather than taking a whole layer, and I’d just go that route.

If you’re fiddling with the compatibility tab, it means you’re going to get more prompts.

Several Program Compatibility Assistant scenarios lead to the XPSP2 layer, which will mean more prompts even if you don’t get geeky on us.

It also could mean that more apps work, if you are a consumer with little knowledge of elevation, tokens, administrators, or any of that garbage, and just want your apps to work. That’s who we’re really targeting with this. Someone advanced enough to find the compatibility tab (or lucky enough to have PCA suggest fixing something) but not advanced enough to be reading this and looking to become a shim ninja.

What we would rather do

Like I said, this was a trade-off. And, honestly, a few folks were pretty up in arms about this (and I count myself among them) until we finally got people to stop suggesting that this was a solution people actually were quite fond of and shouldn’t we just drink the kool-aid and believe that it truly was good for us, and instead fess up that they fought it themselves but just couldn’t get it done in the ideal way so had to find the best compromise for the non-ideal choices that lay before them.

We’d rather have a quick, automated way to put in a targeted fix that didn’t mean “keep running as admin” and instead did something less reckless. But that’s hard to do. (Aaron Margosis seems like he’s coming pretty darned close, though – just give him time.)

If we’re going to elevate, we’d like to have a way to kill the prompts somehow. (Aaron won’t be so fond of that one.)

In the end, it’s all a balancing act. Unlike milk, software doesn’t go bad. You shouldn’t have to go buy a new version of something until it offers features that make you happy (or, these days, until you can afford it). But how do we move the software ecosystem forward, without making you pay the price?

I keep saying this, I know, but it keeps being true: app compat is hard.

What does the MoveIniToRegistry Shim Do?

Chris Jackson — Fri, 05 Dec 2008 23:11:30 GMT

I’m still catching up with requests to talk about stuff – here’s one that came in back in June (and just came again today from somebody else):

“MoveIniToRegistry clearly requires parameters, but there's no documentation defining what these are. (Presumably this fix will apply an IniFileMapping?)”

I haven’t doc’d this shim yet, so here’s the quick and dirty docs on it.

First of all, it doesn’t apply an IniFileMapping. That’d be kind of cool, but that’s something you’d want to do to an installer ideally, since you only need to do it once.

Instead, it intercepts calls to CreateFileA, OpenFile, WriteFile, and CloseHandle. You feed the shim with the name of the files you’re going for, and it ensures that writes happen to both the original file AND to the registry location.

That’s pretty important. The return value from the API is *not* changed by applying this shim. Rather, you still write to the original location, but you also write to a registry location.

For an example, let’s look at Barbie Sticker Designer. Now, this is a program I consider business critical, and I spend the majority of my working days using it. I find it to be a lot less stressful to use than Outlook. How is this shim configured? If we look at it using Compatibility Administrator, you can see the command line used:

%windir%\System.ini * SCRNSAVE.EXE HKEY_CURRENT_USER “Control Panel\Desktop” SCRNSAVE.EXE REG_SZ

The generic argument list is:

IniFile [IniSection] IniKeyName RegBaseKey RegKeyPath RegValue RegValueType

Hopefully that makes the purpose of this shim more clear. It’s still going to write to system.ini (and that writing will drive the return value of the API you are calling), but it’s also going to write the screensaver information to the registry, which is where the system actually looks these days. So, it’s not fixing a permissions issue, it’s fixing an implementation detail (we moved where screensaver configuration is). If we’ve moved something else from an INI to a registry key and don’t already have a fix for it, then you could use this to fix it up as well.

If you were hoping for an easy way to apply an IniFileMapping, well, sorry about that. That’s the problem with these shims – they sometimes have these incredibly tempting sounding names, yet in the end what they actually solve isn’t always what you expected them to solve. I’m trying to get things documented in the order of what’s most useful to you, but keep calling things out to me because I may not always know, and we’ll find out!

Shimming Applications on Windows Vista 64-Bit

Chris Jackson — Mon, 01 Dec 2008 19:47:38 GMT

The same question came up two times in 26 minutes (on the same discussion list, no less), so I figured I’d answer it once here as that seems a reasonable indicator that others may have the same question.

What is the deal with shimming on Windows Vista 64-bit?

Well, it turns out it’s a bit of a mixed story, so let’s go through it.

First, Compatibility Administrator itself runs just fine on x64. It’s a 32-bit app, so it’s running on WOW64, but that’s OK. You can run the tool to create custom shim databases to run on either x86 or x64.

Now, let’s talk about the shim engine. It’s installed on x64, and it supports shimming both 32-bit and 64-bit binaries. So far, so good.

But once we start talking about the custom shim databases you can create today using Compatibility Administrator, the story is not complete today.

You see, the custom shim databases you create (whether you create them on x86 or x64) will only shim 32-bit applications. So, while the platform supports shimming 64-bit applications, the tools don’t give you the ability to do so.

So, if you have a native 64-bit application that isn’t working on Windows Vista, then you are unable to use shims today in order to mitigate the problems you discover.

I haven’t come across any problems with native code doing this, but I have come across problems with managed code. There just isn’t much native code that is compiled for 64-bit (and what is available tends to be fairly recent). Managed code, however, defaults to compiling for “any CPU” – which means that the JIT compiler will compile native 64-bit code for managed code when it’s running on 64-bit. So, you can fix it on x86, but not on x64, even though the same app runs on both.

Well, that’s not very cool. That just might discourage you from moving to x64 if you have a broken managed code application, eh?

Well, there are two solutions for that. Obviously you could recompile the application and change the compiler flags to target 32-bit only (but if you’re doing that, you may as well fix the bugs!). If you can’t recompile, however, you’re not stuck. Just use corflags.exe to set the 32BIT flag, and you’ll be on your way.

By the way, if you really want to get under the covers, you can. The underlying cause is that the XML we generate doesn’t have the (optional) OS_PLATFORM attribute. Try it yourself – if you look in Process Monitor, you’ll see that compatadmin creates an XML file in appdata\local\temp. Of course, it deletes this file when it’s done, so you’ll want to catch it before we’re done with it – a quick glance at the stack tells you that you’ve got msxml3.dll in the stack when you’re creating this, so if you bm on msxml3!* and start stepping until the call to CloseFile, you can go into the temp directory and pull out the XML to see for yourself.

CorrectFilePaths Has to Point to a Directory Which Exists

Chris Jackson — Wed, 01 Oct 2008 01:43:40 GMT

A question came up via comments. (I was going to say that it came up recently, but another glance reveals that it came up in, oh, June. I don’t think I can fairly call that recent…)

“…the fix seems only to work if the directory structure exists…”

This is true, and worth noting. If you point the fix to a directory which doesn’t exist, the shim won’t create the directory, it will just fail, in much the same way the application would fail if it were trying to create an application in a directory which didn’t exist (which, to Windows, appears to be exactly what happened). So, if you’re planning to shim an application and want to create a directory to store things, you should consider creating this directory at install time.

CompatAdmin How Do I Shim Thee? Let Me Count The Ways...

Chris Jackson — Wed, 10 Sep 2008 21:13:37 GMT

When you come across issues debugging applications, there are typically several ways to solve them. Today, I'm going to pick on our own stuff and throw a few different shims at it. Interestingly enough, what I'm going to be shimming up will be the tool I use to create shims: Compatibility Administrator. That's right - I'm going to shim up my shim tool.

What's the bug?

---------------------------
Compatibility Administrator
---------------------------
You do not have administrative rights. Some features might be disabled.
---------------------------
OK
---------------------------

Yeah - I don't enjoy warnings like that, and I certainly don't need them every time.

Solution #1: shim up CompatAdmin.exe with RunAsAdmin. That makes the MessageBox go away because it actually is an admin.

What else could we do? Well, we could figure out which functionality need admin rights. Sitting there using the product non-elevated, I counted two broken features. First, I can't right click and disable a shim. Second, I can't install a shim database.

I don't really need to disable things often, and it seemed to me that installing is something that could be factored out. In fact, a quick glance at Process Monitor tells me that installing a shim database already is factored out! CompatAdmin.exe just calls sdbinst.exe. Is it calling it using CreateProcess (which would require ElevateCreateProcess) or ShellExecute(Ex)? ShellExecute. Nice. So, in theory the think I need to do actually should work just fine, except the software is stopping me from doing it!

---------------------------
Compatibility Administrator
---------------------------
You need administrative rights to perform this operation.
---------------------------
OK
---------------------------

A big fat LUA Bug in our own stuff. It stops me from doing something that works because it doesn't believe it does. Nice work. Let's lie to it.

Solution #2: shim up CompatAdmin.exe with ForceAdminAccess. The dialogs go away, and I can still do what I actually want to do - right click install works just fine, and prompts me when I do that. I then end up running less code elevated, and being more secure. I can read existing databases without elevations. What a huge win! (Well, except not being able to turn shims off, which may be interesting from time to time, but for the vast majority of what I do, this is a great solution.)

Are we done yet, or is there another way to skin this problem?

Well, these are messageboxes. We can suppress them one last way.

Solution #3: shim up CompatAdmin.exe with IgnoreMessageBox (parameter You do not have administrative rights. Some features might be disabled.,Compatibility Administrator). We'll get that to stop annoying me! It still shows the MessageBox when I try to install a database (which is not helpful) but if I suppress that one also, then it just does nothing and says nothing, which is arguably not a very good outcome. But at least it's a touch less chatty. I'll chalk this up as the worst solution.

Personally, I've been using ForceAdminAccess lately. But I thought this was a fun app to show the options available, particularly since you need to shim CompatAdmin somehow to make it useful. (Yeah, I know - that's kind of ridiculous and we should fix it. The bug has been open for over a year - I just poked the team again.)

How Long Can the Command Line Argument Be for a Shim on Windows Vista?

Chris Jackson — Thu, 14 Aug 2008 01:02:29 GMT

I had a comment come up on an earlier post:

"FYI, I've hit a limitation with CorrectFilePaths - the maximum amount of data that can be entered in the parameter field is 512 bytes - not enough to fix more than 2 files under "Program Files" unfortunately..."

Now, I knew the limitation existed, but I had never spent the time to understand the exact limitation, or where it existed. Is the limitation in Compatibility Administrator when it generated the XML for the SDB compiler? If so, we could always be sneaky and edit the XML before the compiler was run. Is the limitation in the SDB compiler? It's a lot more work, but if so we could always be sneaky and reverse the SDB format to add additional information. Or is the limitation in the shim engine itself, which makes being sneaky quite challenging indeed?

And the answer is that the limitation is in the shim engine itself. We can see that with a bit of reverse engineering.

The API that the shim engine calls happens to be publicly documented, so that makes our job easier. It's SdbReadStringTag. Here's where we actually pull the command line:

0x6C105914: 6800040000             PUSH        0x400
0x6C105919: 56                     PUSH        ESI
0x6C10591A: 50                     PUSH        EAX
0x6C10591B: FF7508                 PUSH        DWORD PTR [hSDB <Void *>]; ARG:EBP+0x8
0x6C10591E: 66891E                 MOV         WORD PTR [ESI],BX_IS_ZERO
0x6C105921: E865260000             CALL        __imp__SdbReadStringTagRef@16; (0x6C107F8B) ; SdbReadStringTagRef

Since this is a __stdcall function, arguments are passed right to left. The first PUSH, therefore, corresponds to __in DWORD cchBufferSize, and this is what limits the command line.

So, we know that the command line is limited to 0x400 characters (1,024), and the limitation is implemented in the shim engine itself.

Shims that require longer command lines, such as CorrectFilePaths, you therefore need to be careful with. If you can shorten the command line by only taking the portion of it you need, that is helpful. If you can leverage file and registry virtualization for fixes, that leaves you more of your 1,024 characters to work with as well!

Putting It All Together: Using My Ramblings to Solve Real Problems

Chris Jackson — Tue, 01 Jul 2008 21:35:59 GMT

I send out a lot of links to my articles in response to questions that come up, but the other day I had a chance to use a pile of them to solve a fairly complicated problem end to end. So, I figured I'd share how we can piece together all of this knowledge to solve a more sophisticated problem.

Let's begin at the beginning. The CorrectFilePaths shim wasn't working. We were trying to redirect c:\somedata.txt to %userappdata%\somedata.txt, but it wasn't working. Why not? We weren't sure.

So, we started out by turning on shim debug spew to see if the shim was being wired up. Indeed it was - we could see it picking up the shim when we launched the process. We could also look at the process in Process Monitor and see in the stack that the call to write c:\somedata.txt was passing through AcLayers!NS_CorrectFilePaths::APIHook_CreateFileA. So, we knew that the shim wasn't improperly applied.

Our next hypothesis, then, was that the shim's command line was not configured properly. I tried copying and pasting from Process Monitor in case there was a copy error, but no such luck. So, it was time to go in with a debugger and see what was happening.

Now, where do we set the debugger breakpoint? Well, why not when we enter the shim? So, I set a bp on AcLayers!NS_CorrectFilePaths::APIHook_CreateFileA and took a look at the arguments we were passing to this function. What did I see?

\somedata.txt

Indeed, it never specified the C drive - just the root of whatever drive you happened to be running on. And, since CorrectFilePaths just uses a literal string match, the c: was causing the match to fail.

Replace that argument with \somedata.txt;%userappdata\somedata.txt and everything worked fine!

So, using a bit of knowledge of Process Monitor, a bit of knowledge on the Debugging Tools for Windows, and knowledge from this blog, we could actually solve a real-world problem. And that's what I like to see. Let me know if you are finding gaps that nobody is talking about yet, and we'll see if we can get them filled here!

Documentation Update for the Application Compatibility Toolkit

Chris Jackson — Wed, 18 Jun 2008 21:17:51 GMT

Last week, we updated the documentation for the Application Compatibility Toolkit. It's kind of hidden, though - if you go to the Application Compatibility Toolkit download page, you will find a new item in the list of files:

ACT50_Doc_Update_Installation_Instructions.zip (1.9 MB)

It's got a bit of a silly name (because it's not just instructions - it's the update also) and a silly package (or no package, really - you drag and drop the new chm into a protected Program Files folder, so you'll get UAC prompted), but it is goodness.

Liz Ross is the technical writer who creates the docs for ACT. She's got some other goodies as well, and this is the latest manifestation of the workflow we've put together. I'm busy translating shim docs from source code to geek, and she's busy translating them from geek to human.

Have a look. I'm particularly interested in getting shim documentation out faster, and hopefully updating without waiting for the next release of ACT can help you be more effective resolving application compatibility issues.

We'd love to hear your feedback on the updated docs!

Resolve Issues with Windows Resource Protection using the WRPMitigation Shim

Chris Jackson — Sat, 31 May 2008 00:57:22 GMT

Every now and again, I bump up against a setup application (it's almost always a setup application) that tries to drop older versions of protected operating system files. It's fairly easy to mitigate, but I thought I would go through some of the mechanics, and some of the places where the mitigation can break down.

Let's take a walk down memory lane...

Introduced in Windows 2000 and brought over to Windows ME ... I'll pause for everyone to recover from the shivers in your spine ... we offered a feature called System File Protection. This feature is designed to protect the integrity of the operating system. This feature had a couple of issues, which I talk about here: http://blogs.msdn.com/cjacks/archive/2007/04/20/windows-resource-protection-wrp-and-activex-control-installation-on-windows-vista.aspx.

So now, things are better, except of course the applications that break, which is why we didn't just modify the ACLs in the first place. So, one by one, we just start applying our WRP shims to applications that need it, and this got us back where we needed to be for application compatibility. We automatically apply this shim if we detect that you are a setup. We apply this shim to regsvr32.exe. We throw it everyplace that we think we're going to catch people trying to write to protected operating system files.

So, how does it work?

Well, first we always try to run the original API. If that fails, then we check to see if you are a WRP protected file. We do that for performance - if what you are trying to do already works, we don't need to fix you, nor do we need to run code to determine if we ought to fix you. We just let you go on doing your thing.

But if the operation didn't work, then we'll check to see if the file is WRP protected. If so, then we'll pretend that things worked.

How do we pretend?

Well, if you're trying to delete, we can just return success. Move? Success. Change attributes? Success.

But what if you're trying to write?

In that case, we don't get very far by returning success, because we need to return a handle that is valid or the application could AV. So, we just create a temp file, and return a handle to this temp file.

Right now, I'm running an application attempting to update kernel32.dll, and I find this in %temp%: WRP112.tmp.

That's where I'll be writing when I write. My application continues to work, and we resolve the issue.

But let's back up a minute - what happens if I am trying to drop my file in a non-standard location and register it from there? Here, we can run into issues. Clearly, the application is intentionally trying to circumvent protection mechanisms. They drop the dll somewhere other than the location of the protected dll (which succeeds) and then call regsvr32.dll. (Some people think they're really tricky - I once saw an app drop some version of shell32.dll from Windows 2000, but they called it shell32.ico, then called regsvr32 on it.)

Our check to see if it's a protected operating system file says "no" because it isn't owned by Trusted Installer, and we don't fix it up.

With people trying to be tricky like this, I usually end up modifying the package to remove this drop. But I suppose I could CorrectFilePaths to hit my protected location...