Pat's Application Compatibility blog

Tales of Application Compatibility Weirdness – Demystifying UAC Virtualization

patricka — Mon, 14 Dec 2009 20:03:28 GMT

We had a customer in the lab recently and they were quite certain their application was Windows 7 ready. To their surprise, it had several issues in certain situations. In this post, I’ll talk about one of the situations they encountered as well as other stumbling blocks you might run into with UAC Virtualization.

Why Doesn’t my Program Work in Program Files?

For this particular application, the default install directory was a custom directory on the system drive (e.g. C:\MyApp). The application performed as expected in this scenario. However, Microsoft recommends that applications be installed in Program Files. This is secured by default to reduce elevation of privilege attacks and is a known location for enterprise inventory tools. Also, installing to program files is a client logo requirement.

We changed the install location to Program Files and the application eventually failed. Why?

User Account Control = Standard User by Default

Program Files is considered a “per machine” install location. It is protected by Mandatory Integrity Control and only allows users read and execute rights. Under UAC, processes will run as a limited standard user and are denied write access to Program Files. The application was designed to store various data files in the install directory. It also stored data in the registry under under the protected HKLM\Software key as well. Here’s where it starts to get confusing… Most of the files and registry keys appeared to be able to write but others were not.

Why?

UAC Virtualization to the Rescue (sort of)…

There is a mitigation that is included in Vista and Windows 7 called UAC Virtualization. This mitigation is intended for this exact scenario -- legacy applications that write to protected per-machine locations. If a legacy (non-manifested) application tries to write to a protected directory such as Program Files or certain keys in the HKLM hive, the “access denied” error is captured and the write is redirected to the user’s UAC virtual store. This allows legacy applications that store state in protected directories and registry locations to continue to work in most cases.

So, why did it fail?

Fuzzy Gray Area

UAC virtualization’s goal is to virtualize data. It doesn’t virtualize various executable files (exe, bat, dll, etc.). It doesn’t virtualize files that the user is explicitly denied write access. This is what occurred for the application and why it failed.

Mystery solved…

…but Wait. There’s More Weirdness!!!

The scenario of a few files not being virtualized is uncommon. Here are a few more common issues that can really make you scratch your head.

Weirdness Due to Multiple Users

UAC Virtualization works by catching the access denied error writing to a “per machine” location and redirecting it to a “per user” location. This solves the problem of writing the data but this can change the behavior of your application. For example, if a game application is designed to store a high score file to Program Files, every user will get his/her own copy of the high score file. This isn’t the correct behavior for a high score file. There should be one file for all users.

Weirdness Due to Running an Application Elevated

If you right click an application shortcut and select “Run as Administrator”, this runs the application with the full admin token. The application now has write access to protected locations. Therefore, UAC virtualization is disabled and it will read and write from the “per machine” location instead of the virtualized “per user” location. Now, you have a situation where the application is using completely different files or may not work at all.

Please, make it stop!

You may be saying “This is madness.” Remember, this is a mitigation that greatly improved the ability to have legacy applications that weren’t designed to run as standard user to continue to work. Without this mitigation, lots of existing applications that store state in “per machine” protected locations would fail with access denied errors.

Design for Standard User

Applications that target Vista or above should be designed to be run by standard users. This means that the application only writes data to “per user” locations in the file system or registry. Standard user was introduced in Windows XP. Vista introduced “standard user by default” with UAC. Running as standard user is here to stay but virtualization will eventually be removed in future OS releases.

Ideally, you want to design your application to only write to per user locations. Sometimes, that’s not possible or it doesn’t make sense. Here are a few suggestions on how you might change your application if it needs to store per machine data.

For files, you may want to write to the The programdata directory. It is intended for per machine configuration data. Check out Chris’ post on options for writing configuration data.

For the registry, you may want to create your own key HKLM\Software\MyCompany\MyApp and then set the appropriate permissions at install time. Read more about registry virtualization here.

Manifest Your Application

Once you have your application ready to run as a standard user, you will want to disable the mitigation. This is done by adding an application manifest with the “trustInfo” section to your application. For more information on manifesting your application, look at my post on manifests.

Answers to Several Application Manifest Mysteries and Questions

patricka — Thu, 10 Dec 2009 04:12:00 GMT

Recently, someone asked me this question in e-mail -- “How do I get my application to run elevated when someone launches it?” Instinctively, I responded with a “You need to manifest it.” I got a nice long e-mail in return. ;-) I quickly realized that application manifests appear simple but are quite confusing. Manifests are becoming very relevant as they control much more application behavior in Windows 7 and future OS’s. Any application that targets Vista or greater, should contain an application manifest.

Application Manifest Basics

An application manifest controls execution behavior of a binary and is defined in XML.
An application may or may not have an application manifest.
Application manifests can be embedded into an executable or can be an external file.
Application manifests have been around since XP. We keep extending them to control more behavior and tend to create new MSDN articles: XP introduced side by side assemblies, Vista added UAC behavior, Windows 7 added switchback and DPIAware. New sections are ignored on previous OS’s.
In XP, external manifests have priority and over embedded manifests. In Vista and beyond, embedded manifests have priority over external manifests. For example, if you have an embedded manifest and and external manifest for an application – On XP, the external one will be used; on Vista and later, the embedded one will be used.
On Vista and Windows 7, a manifested application with a trustInfo section will control what token will be used to start the process (standard user or admin). Also, all legacy mitigations will be disabled. This includes UAC Virtualization, Installer Detection, and the Program Compatibility Assistant.
Windows Client Logo requires applications to be manifested.

Checking for an Embedded Manifest

You might be wondering if your exe’s have embedded manifest and what they look like. My favorite tool to dump a manifest is sigcheck. Use the –m switch to dump the manifest. E.g. sigcheck.exe –m myapp.exe

Example output of a manifest with only side by side behavior:

This executable is a native C++ program.

D:\dev\legacyUAC\Debug>sigcheck.exe -m legacyUAC.exe

sigcheck v1.60 - sigcheck
Copyright (C) 2004-2009 Mark Russinovich
Sysinternals - www.sysinternals.com

D:\dev\legacyUAC\Debug\legacyUAC.exe:
        Verified:       Unsigned
        File date:      7:43 PM 10/26/2009
        Strong Name:    Unsigned
        Publisher:      n/a
        Description:    n/a
        Product:        n/a
        Version:        n/a
        File version:   n/a
        Manifest:
∩╗┐<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestVersion="1.0">
<dependency>
    <dependentAssembly>
      <assemblyIdentity type="win32" name="Microsoft.VC90.DebugCRT" version="9.0
.21022.8" processorArchitecture="x86" publicKeyToken="1fc8b3b9a1e18e3b"></assemblyIdentity>
    </dependentAssembly>
</dependency>
</assembly>

Example output of a manifest with UAC trustInfo section:

This executable is a .NET C# program.

D:\dev\legacyDPI\legacyDPI\bin\Debug>sigcheck.exe -m legacyDPI.exe

sigcheck v1.60 - sigcheck
Copyright (C) 2004-2009 Mark Russinovich
Sysinternals - www.sysinternals.com

D:\dev\legacyDPI\legacyDPI\bin\Debug\legacyDPI.exe:
        Verified:       Unsigned
        File date:      3:06 PM 11/12/2009
        Strong Name:    Unsigned
        Publisher:      Microsoft
        Description:    legacyDPI
        Product:        legacyDPI
        Version:        1.0.0.0
        File version:   1.0.0.0
        Manifest:
∩╗┐<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestVersion="1.0">
<assemblyIdentity version="1.0.0.0" name="MyApplication.app"/>
<trustInfo xmlns="urn:schemas-microsoft-com:asm.v2">
    <security>
      <requestedPrivileges xmlns="urn:schemas-microsoft-com:asm.v3">
        <requestedExecutionLevel level="asInvoker" uiAccess="false"/>
      </requestedPrivileges>
    </security>
</trustInfo>
</assembly>

If you are wondering what those three weird characters at the beginning of the manifest are, it is the byte order mark. I talk about it in this post.

External Manifests

An external manifest is a text file that contains the manifest XML. The file must exist in the same directory as the executable and have the same name with a “.manifest” extension. For example, if the name of your executable is MyApp.exe, your manifest file would be named “MyApp.exe.manifest”.

Remember, in Vista and beyond, embedded manifests have priority over external manifests. Therefore, if the executable already has an embedded manifest, the external manifest will be ignored.

Embedding a Manifest

There are two ways you can embed a manifest. You can have the compiler embed it or you can embed it with the manifest tool (mt.exe) included in the Platform SDK.

Embedding at Compile / Build Time

Visual Studio 2008 has functionality to embed manifest for native and managed projects.

For Visual Studio 2005, Catherine Heller has a great post on how to embed manifests in your build process.

Embedding using mt.exe

The manifest tool is a command line tool that lets you add or merge manifests to an executable. For usage, check the MSDN article for the Manifest Tool. Here’s an example for adding a manifest to an executable:

mt.exe -manifest MyApp.exe.manifest -outputresource:MyApp.exe;#1

You will be modifying the binary when you embed the manifest. If your exe was signed, this will invalidate the signature and it will need to be re-signed.

Planning for the Future

Windows 7 introduced “switchback” into the manifest by adding a <compatibility> section to the manifest. This allows you to state what OS you support. So, if you insert the tag that you support Windows 7, you will get a handful of new behaviors. Otherwise, your app will “switchback” to the Vista behavior. The plan is to use this section to make better choices for compatibility behaviors in future OS’s. When adding manifests to applications, consider adding the supportedOS tag.

Questions from the Developer Booth at TechEd Europe 2009

patricka — Tue, 24 Nov 2009 01:15:22 GMT

I had the opportunity to go to Berlin for TechEd Europe 2009. I delivered a presentation on Application Compatibility and the rest of the time I hung out in the “Test your app on Win 7” and “Win 7 Developer drop in center”. I got some great questions from attendees. I thought I’d blog some of the general Q&A. Thank you to all that stopped by the booth. I loved Berlin. I hope I get to go back soon.

Q: Do I need to worry about anything in Windows 7 for web apps?

A: There are some IE8 changes that may affect web applications. The Internet Explorer 8 Readiness Toolkit is a very good reference. The developer checklist is a good place to start.

Q: What do I need to worry about for existing installers on Windows 7?

A: The common problems that are encountered with existing installers are the same issues encountered in Vista:

Version check criteria
MSI custom actions

Ideally, you should not check for version if possible. Version checking tends to block applications that may install and run just fine on a newer OS. If you need to check version, you should display a warning rather than exiting the installation. Version checking can also cause issues after an upgrade with uninstall. If you check version, the user may not be able to uninstall your application after an upgrade to the new OS.

MSI custom actions can run in a standard user context or the administrator. Sometimes the custom action is running in the wrong context. Chris Jackson has a great post on the common issue with deferred custom actions.

Q: Where can I find info on developing on Windows 7?

A: The Windows 7 Developer Guide gives a good overview of new developer functionality. If you are looking for presentations and code samples to get you started, check out the Windows 7 Developer Jumpstart Kit.

Q: I’m a .NET Developer. What’s the guidance for developing on Windows 7? Should I use the Code Pack or wait for .NET 4.0?

A: The short answer is: Use the Code Pack today but think of it as free code to include in your application that wraps the native Windows 7 API’s. In .NET 4.0, you will see some of the new Windows developer features available in WPF. Yochay Kiriaty has recently outlined guidance on this topic.

Q: .NET 1.1 SP1 support for Windows 7?

A: As with all older products, be sure to check the support lifecycle for the product first. There isn’t an official statement of .NET 1.1 SP1 compatibility for Windows 7. However, because compatibility was a key goal between Vista and Windows 7, efforts were made to ensure .NET 1.1 SP1 works on Windows 7.

Q: Does <insert app name here> work on Windows 7?

A: The Compatibility Center is the best reference to get information on what applications have been reported to be compatible with Windows 7.

Windows 7 Application Compatibility Testing Guidance document

patricka — Mon, 05 Oct 2009 21:18:11 GMT

If you have an existing application, you may be wondering if it’s going to work on Windows 7. If it works on Vista, you have a good shot at it working on Windows 7.

As a developer or a tester, you may be wondering if you have tested your application to uncover potential issues on the new OS. We’ve worked with a lot of ISV’s over the past few years and we’ve come up with a document that focuses on the common issue areas of application compatibility with testing scenarios. The Windows 7 Application Compatibility Testing Guidance document can be downloaded from MSDN Code Gallery.

Displaying the UAC consent dialog on the desktop

patricka — Tue, 14 Jul 2009 21:39:40 GMT

I had written a draft of this post a while ago (last year). I use Live Writer as my blog editor. Between Live Writer and Intellimirror,this draft has survived multiple reloads of interim builds of Windows 7. So, I think in some ways, it deserves to be resurrected and posted. :-) Here’s the post with a Windows 7 update:

Frequently, we need to give presentations over Live Meeting. When we demonstrate UAC, the consent dialog is displayed on the secured desktop. Therefore, the Live Meeting participants don't see the UAC consent dialog. To solve this, we use a policy setting to allow the dialog to be displayed on the user desktop.

The reason I bring this up is that this same trick can be used if you have an automated functionality test that needs to click the consent dialog.

You can adjust this with a policy. The local security policy (Run secpol.msc) is Local Policies | Security Options | User Account Control: Switch to the secure desktop when prompting for elevation. Set this policy to "disabled" to allow the consent dialog to be presented on the user desktop.

On Windows 7, this setting is exposed through the UAC slider bar UI. Just slide the knob to the next notch down from the default:

Chris talks all about the slider in his post.

Of course, running with this policy disabled opens up some risks and isn't recommended for normal operation. For more information on the secure desktop and UAC consent can be found in the UACBlog in the User Account Control Prompts on the Secure Desktop post.

Developing Compatible Applications for Windows 7 webcast

patricka — Wed, 24 Jun 2009 21:29:00 GMT

I know I could talk all day about application compatibility topics and not cover everything that can be discussed. Here’s a one hour webcast I did that focuses on the top issues and most confusing topics for Application Compatibility. I do some of my favorite demos showing UAC architecture, data redirection, services isolation, and mandatory integrity control. The target audience is for developers and testers.

Here’s the webcast: Developing Compatible Applications for Windows 7

My main goal in this presentation is to talk about moving application from XP or Vista to Windows 7 and demystify some of common problems. I also wanted to “plug” the Application Compatibility Cookbooks. These documents are some of the best resources for understanding what breaking changes have occurred and how to mitigate them.

Make sure to reference both cookbooks. Most of the breaking changes are in the original XP to Vista cookbook.

Moving from XP-> Vista/2008 –> Win7: “Application Compatibility Cookbook”

Moving from Vista -> Win 7: “Windows 7 Application Quality Cookbook”

OEM Ready Test Case 1 Incorrectly Fails with a UTF-8 manifest

patricka — Wed, 10 Sep 2008 23:08:53 GMT

I haven't posted in a while. I've been helping out with a couple other programs we have going on in the labs these days. OEM Ready is a subset of Certified for Windows Vista targeted at applications that ship on new PC's.

If you are using the automation in the OEM Ready Certification Test Tool, you can get a false failure if your application has a UTF-8 embedded manifest that has the byte order mark (BOM) included. Visual Studio 2008 includes the BOM if you use it to embed the manifest.

The error in the log looks something like this:

2008-09-06T01:56:32.4030000Z	Information	INFO: Embedded manifest exists: ï»¿<?xml version="1.0" encoding="UTF-8" standalone="yes"?> <assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestVersion="1.0"> <trustInfo xmlns="urn:schemas-microsoft-com:asm.v3"> <security> <requestedPrivileges> <requestedExecutionLevel level="asInvoker" uiAccess="false"></requestedExecutionLevel> </requestedPrivileges> </security> </trustInfo> <dependency> <dependentAssembly> <assemblyIdentity type="win32" name="Microsoft.VC90.DebugCRT" version="9.0.21022.8" processorArchitecture="x86" publicKeyToken="1fc8b3b9a1e18e3b"></assemblyIdentity> </dependentAssembly> </dependency> </assembly>
2008-09-06T01:56:32.4040000Z	Warning	EXCEPTION: Data at the root level is invalid. Line 1, position 1../n at System.Xml.XmlTextReaderImpl.Throw(Exception e) at System.Xml.XmlTextReaderImpl.Throw(String res, String arg) at System.Xml.XmlTextReaderImpl.ParseRootLevelWhitespace() at System.Xml.XmlTextReaderImpl.ParseDocumentContent() at System.Xml.XmlTextReaderImpl.Read() at System.Xml.XPath.XPathDocument.LoadFromReader(XmlReader reader, XmlSpace space) at System.Xml.XPath.XPathDocument..ctor(TextReader textReader) at Microsoft.LogoTest.Tests.Common.ManifestChecker.VerifyManifest(String manifest)
2008-09-06T01:56:32.4040000Z	Warning	WARNING: Intermediate result: C:\Program Files\Microsoft\OEM Ready Demo\OEMReadyDemo.exe FAILED the test. It does not have manifest that is compliant with UAC guidelines.
2008-09-06T01:56:32.4050000Z	Error	ASSERT: The test failed because one or more of the executable did not comply with the UAC guidelines
2008-09-06T01:56:32.4100000Z	Information	INFO: Test result: Failed

Sigcheck is used by the test tool to dump the manifest. Sigcheck outputs the BOM as garbage characters (ï»¿). This throws off the test tool and it gives an incorrect result. This is a known bug with the tools and will be fixed. For now, if you run into this problem, you can submit this test case as a pass.

Pat

Registering ATL Performance Counters on Vista and Server 2008

patricka — Wed, 09 Apr 2008 19:33:24 GMT

If you use the ATL Server performance counter libraries, you may run into a problem on Vista and WS08. The ATL libraries don't play well with the updates to Windows Resource Protection. WRP enforces restrictions on the registry to prevent corruption of performance counters. ATL breaks these rules and tries to update the registry directly.

There is a workaround to the issue that is outlined in Q933241.

To avoid this problem, the recommended way to add counters is through the lodctr tool or the API LoadPerfCounterTextStrings. See Adding Performance Counters.

If you are really ambitious, ATL Server code has been released as shared source. I haven't looked at the code or the project but there may be an opportunity to change the way ATL loads perf counters in the future: http://blogs.msdn.com/vcblog/archive/2007/01/19/atl-server-visual-c-shared-source-software.aspx

Pat

Windows Server 2008 Application Compatibility presentation

patricka — Tue, 08 Apr 2008 01:01:23 GMT

In the compatibility labs, we always give an big overview presentation about Windows application compatibility. There's a lot of overlap between Vista and WS08 AppCompat but we call out some specific areas that mostly affect WS08. We've recorded the WS08 specific section of the presentation and posted it to Channel 9. I hope you find this useful in preparing your applications for WS08.

Detecting what Server Roles are installed on Windows Server 2008

patricka — Wed, 05 Mar 2008 09:35:00 GMT

I think there is always a nugget of information that inspires a blog. This is the idea that inspired my blog.

By default, Windows Server 2008 has no server roles installed. This isn't intended to make you do more work (it's a feature). The goal is to be secure by default. After the OS installed, you add the roles and features that that are needed.

So, from an application compatibility point of view, your application may require a role that isn't installed by default. There are two ways to programmatically query what roles and features are installed.

Using ServerManagerCmd

UPDATE: ServerManagerCmd is being deprecated. It is recommended to use PowerShell cmdlets for Server Manager starting with R2 forward.

There's a cool command line utility that has been added to Window Server 2008 called ServerManagerCmd. ServerManagerCmd allows you to add, remove, and query roles and features.

If you execute the following command:

> servermanagercmd.exe -query roles.xml

An XML file of all the roles and features installed will be created. You could then parse that file to determine what roles and features are installed. I was hoping to find something in the scripting repository to do this but couldn't find anything. Powershell would be a good option but remember... no features are installed by default. Powershell and .NET 3.0 would need to be installed. You could do that with servermanagercmd as well.

For example:

> servermanagercmd.exe -install Powershell

ServerManagerCmd is great for scripting but what if you want to do determine installed roles in your application's installer? ...or avoid lots of parsing in a script? ..or needing to install .NET and Powershell? WMI is probably a better choice.

Using WMI

With Windows Server 2008, there is a new class in the WMI namespace \Root\cimv2 called Win32_ServerFeature. This will allow you to use WMI to iterate through what roles and features are installed. WMI is a very good option for determining in your installer if a prerequisite role is installed.

Where do I start?

patricka — Thu, 28 Feb 2008 10:08:17 GMT

When you work on a given subject for a while, you tend to jump right in and forget to start at the beginning. In the Application Compatibility lab, we kick off the week with an overview presentation and cover the basics of common compatibility issues. Those topics are probably a good place to start with this blog.

The first thing to mention is the Application Compatibility Cookbook. This is the primer of Application Compatibility. I was going to call it the K&R of of Application Compatibility but I don't want to date myself ;-). I highly recommend reading the cookbook or at least keep it handy for a reference.

Now that you've looked at the cookbook (Go look at it now if you haven't), you are probably saying, "Whew, that's a lot of stuff. Can you just tell me the top issues with Application Compatibility?" So, let's start with a list of the top areas of issues:

User Account Control (UAC):

UAC is intended to run applications as standard user by default. Even if you are a member of the administrators group, applications that are not marked to run with administrator privileges will run with reduced "standard user" rights. Applications may not be designed to run as a standard user and may not have rights or privileges to certain resources.

Next steps if you have an issue:

Installation

Installation issues usually revolve around UAC. Almost all installers need to run with admin rights because they usually write to Program Files or HKLM in the registry. MSI issues usually revolve around custom actions being run in the wrong context (user or admin).

Next steps if you have an issue:

MSI Installer issues - See rflaming's blog
Custom installer issues with UAC - See installer topics under UAC in the Cookbook

Service Isolation

On Vista and Windows Server 2008, only services run in session 0. Desktop applications run in sessions 1+n. This blocks Windows messages between services and desktop apps. This also blocks services from displaying UI on the desktop.

Next steps if you have and issue:

Session 0 Isolation in the cookbook

Internet Explorer Protected Mode

IE in protected mode is sometimes called "Low Rights IE". When protected mode is on, IE runs at a low integrity level. This isolates IE from the rest of the processes and resources on the system. ActiveX controls and browser objects may not have appropriate rights running in protected mode.

Next steps if you have an issue:

IE protected mode in the cookbook
Understanding and Working in Protected Mode Internet Explorer

That's it for now. One final note, the area for all things regarding Application Compatibility can be found on Technet.

Pat

Welcome to my blog

patricka — Wed, 27 Feb 2008 22:12:05 GMT

I'm Pat Altimore an Application Developer Consultant who spends most of my time working in the Application Compatibility Labs at Microsoft. I'm planning on posting stuff of interest that we find in the lab. I hope the posts here will be helpful to developers that are getting their apps ready for Vista or Windows Server 2008 (and beyond).

There are a lot of great blogs by other AppCompat consultants out there. Please check out: MaartenB, CJacks, and Aaron_Margosis.