Hi, I am Wei Wang, a Lead Software Design Engineer in Test on the User Account Control (UAC) team within the Windows Security organization. As many of you experienced, User Account Control in Windows Vista fundamentally changes the way users interact with Windows, therefore, it has a great impact on how testing is performed on the Windows platform and the numerous supported applications. The Windows test organization went through a great deal of work to augment its test infrastructure to be UAC compliant. Many test teams spent a significant amount of time to redesign how their tests are structured in order to fully exercise their components in a UAC environment.
There are three major buckets of change with respect to the differences between XP and Win2K3 to Vista.
1) Moving away from testing as the built-in administrator account
2) Test executable separation (test setup/clean-up vs. runtime test cases)
3) Testing as user of different user groups (Users, Administrators, Network Operator, etc...)
Historically the majority of testing was performed as the built-in administrator account because it was a lot easier to test when you are running with full administrator privilege. Because most users run as a member of the local administrators group, it is effectively the same whether you test as the built-in administrator or a member of administrators group. However, in Windows Vista with UAC enabled (as it is by default), this is no longer true since all users run with standard user rights by default even when being a member of the local administrators group. Therefore, with this change, testing as the built-in administrator is not the same as testing as a member of the local administrators group. This is a dramatic change from both the user and test perspective. Instead, all tests should be executed under the default standard user account or in some cases as a user who is a member of local administrators group --user in Admin Approval Mode.
Throughout our internal and external test evangelism effort, we (the UAC team) encountered many cases where setup and clean-up code, which often perform administrative tasks, are compiled into the same executable with the code that exercises all the end user test cases. With the changes in UAC, a single test binary composed of setup/cleanup and runtime functionality no longer work --the setup and clean-up code will by default fail to run due to the lack of administrative rights. There are two workarounds to this issue. The first one is to move the setup and clean-up code into a separate executable and to mark this executable with a manifest defining a run level equal to “requiredAdministrator”. Please refer to the UAC developer whitepaper on how to mark applications with the applicable run level. This solution will closely match the end user experience. However, if for some reason, separating your test into separate binaries is not an acceptable solution; the second workaround is to mark the test executable as “requiredAdministrator” and run the setup and clean-up code with full administrator rights and to impersonate down to a standard user access token during the rest of the test case execution. The problem with this testing method is that your impersonating token will not be identical to the filtered token on the system; therefore you can potentially hit issues when running as standard user without impersonation. Additionally the impersonated token will not have the virtualized bit set, and thus registry and file virtualizations will not occur.
Another major test change is really not caused by UAC, but manifests with UAC. Some test teams continue to test as administrator as they did in the past. The standard user account is either not tested, or only tested in a few test scenarios. Historically this was understandable as most customers ran as administrators on XP and Win2K3. However, this is no longer the case as all users, irrespective of privilege, will logon as a standard user. It is essential to verify your APIs, components, and applications interact correctly and reasonably as standard user. The basic rule is to review your scenarios, define the standard user and administrative behavior both positive and negative, then exercise each scenario under both user accounts and verify the outcome. Following this simple methodology will give you complete functional test coverage. Additionally you can reference the UAC Shell Guidance <coming in Beta2> for our recommended best practices for UI components to verify if your user experiences are consistent with our designs.
User Account Control fundamentally changes how testing is performed. We understand that this is a significant shift in how we’ve tested in the past, but UAC is an important effort to reduce the default attack surface of the Windows operating system. We hope you will see the benefit of UAC and therefore invest in changing your test infrastructure to be UAC compliant. This is only an overview of UAC’s impact on testing and we will follow up with a series of more in-depth articles on tools, test design, implementation tips, and additional guidelines to facilitate testing with UAC.