Windows Automation API 3.0

Some of you may be wondering why we refer to the Microsoft Accessibility programming platform as "Windows Automation API 3.0". We named it that way because the SDK isn’t really just about MSAA or UI Automation anymore. It's a combination of both old and new technologies. For instance, the Dynamic Annotation API is no longer unique to MSAA Servers – it can be used by these applications to add support or annotate UI Automation properties. The IAccessibleEx interface bridges two worlds together in a very organic way. Even from UI Automation, new LegacyIAccessible control pattern and the new Client API (i.e., ElementFromIAccessible) function enables jumping between MSAA accessible object tree and UI Automation element tree. We set the version of Windows Automation API to 3.0 in order to respect the previous MSAA version number as well as the generations of new UI Automation implementations in Windows.

When we talk about technologies specifics, we continue to use 'UI Automation' or 'MSAA'. Windows Automation API is used when we talk about collections of technologies or the Windows specific frameworks.

SDK Tools

The Windows 7 RC SDK features a new Inspect Object tool (Inspect) for the Windows Automation API SDK. It may look familiar to some of you who’ve been using the old Microsoft Active Accessibilty (MSAA) tool, however the bulk of the code is actually re-written from scratch. The new Inspect tool features the essence of UI Automation included in Windows 7.

Core Tools

Microsoft Windows SDK for Windows 7 (Release Candidate) includes all of the tools. The latest revisions of Windows Automation API SDK tools are:

• Inspect Object (Inspect.exe) features accessible UI inspection by hit-testing or event tracking either by MSAA or by UI Automation. It uses the latest UI Automation Client COM API and it has most coverage of the ad-hoc UI testing by Windows Automation API. Important: New UIA mode of the Inspect.exe has dependency on new Windows Automation API framework – you have to have pre-release Windows 7 environment to use the new mode as of today.
• Accessible Event Watcher (AccEvent.exe) tracks WinEvents for MSAA. The current version is limited to WinEvents, however we are planning to extend it to support UI Automation events in the future. Since WinEvent isn’t limited just for MSAA, you can also use the tool for general purpose of tracking WinEvents sent across the desktop. (FYI: You can actually see some UIA events that are based on IAccessibleEx or some (not all) native UIA implementations. Those events are actually visible by AccEvent by choosing "(unknown)" and "Event" in the "Events" and "Event Information" options of AccEvent. You can find those WinEvent based UI Automation event IDs from UIAutomationClient.h by searching "UIA_EventIds" or "UIA_PropertyIds" (for corresponding "property changed events" for UI Automation).)
• UI Spy (UISpy.exe) is as if Inspect and AccEvent combined with neat object tree presentation for UI Automation. The tool is written in managed UI Automation client API – the tool doesn’t feature new UI Automation control patterns and properties introduced with Windows 7 because the managed API hasn’t updated with the OS release.

Inspect

It's worth highlighting some of the new tricks for UI Automation support using Inspect. You can now inspect brand new UI surfaces that support UI Automation natively. Following are a list of things you can try out with Windows 7 RC and Internet Explorer 8.

• New UI Automation AriaRole and AriaProperties properties for web contents that supports ARIA and displayed by the Internet Explorer. You can find IE8 specifics of the implementation under their SDK. You can find the original specification from UI Automation Community Promise, too.
• Most of UI Automation Control Pattern methods are accessible though the “Action” menu in the UIA mode. From simple Invoke.Invoke function to ItemContainer.FindItemByPropertycontrol; ad-hoc investigation can be achieved easily and quickly.

You can find all keyboard shortcuts and other key features of Inspect Object tool at http://msdn.microsoft.com/en-us/library/dd318521(VS.85).aspx.

The SDK and platform are still under development, therefore things may still change, but we hope you find the tools useful in the meantime. Keep in mind that the MSDN online Library is still being updated with new changes, but these should get you started in the right direction.

Old versions of these tools are located in the Microsoft Download Center as MSAA 2.0 SDK tools, however these are retiring with the release of new versions (with the exception of AccExplorer).

UIAVerify and AccChecker

While they are not technically SDK tools, there are also a couple of other tools that you can leverage (one called 'UI Accessibility Checker' and the other 'UI Automation Verify',) but it would take us an entire post to describe them! If you'd like to learn a bit more about what they do, check out  http://www.codeplex.com/AccCheck and http://www.codeplex.com/UIAutomationVerify. In short, UI Automation Verify is designed for integrating your UI Automation Provider testing into your test pass and UI Accessibility Checker features a set of accessibility test scenarios and criteria for general UI accessibility testing.

Common tips and suggestions

There are few common tips and useful suggestions to keep in mind whenever any of these accessibility test tools are used.

• Launch tools as elevated (with Windows Vista, Windows Server 2008, or Windows 7):
  With increased security of the desktop by Windows, accessibility tools should be either signed with UIAccess flag in the manifest (and installed on either under %programfiles% or under %windir%) or you could get blocked from certain types of UI surface. Quick workaround is to launch the tool elevated, by either right click (shift+f10) the executable and select “Run as administrator” or choose compatibility tab “Run this program as an administrator” in file properties of the executable (e.g., Inspect.exe).
• Do not mix hit testing and focus tracking (or any other mix):
  While Inspect is set to follow both mouse pointer and focus events (for convenience of quickly locating UI on screen), doing mixed texting can be very confusing even with ad-hoc investigations. You should stick to either just mouse for keyboard to navigate the UI, and make sure you choose right combination for the object tracking; “focus tracking with keyboard navigation” OR “mouse cursor tracking with hit-testing.” You can use tools’ keyboard shortcut (e.g., CTRL+SHIFT+F6) to navigate the object structure without disturbing the focus or mouse pointer location if necessary.
• Do mix tools per scenarios and purposes
  While many people confused Inspect as test tool for focus tracking – it simply is not. Inspect is certainly handy for doing sanity check of focus location on desktop, but timing issues of focus event(s) can fool you very badly. Simply, Inspect isn’t simply design to achieve the goal of “any event tracking” – AccEvent or UISpy’s UIA event logger is designed for the very purpose. So do mix or use right set of tools when your test scenario is looking for particular scenarios. Again, Inspect isn’t tool for focus tracking test.

*UISpy and UIAVerify are based on managed UI Automation Client API which has not been fully updated with the Windows 7 version of UI Automation implementation. The tool may not find properties, patterns or events that are not yet supported by the managed API.

• SDK tools and references
  http://msdn.microsoft.com/en-us/library/dd373661(VS.85).aspx
• UISpy
  http://msdn.microsoft.com/en-us/library/ms727247.aspx

• Sumit’s blog “Where is UISpy.exe?”
  http://blogs.msdn.com/windowssdk/archive/2008/02/18/where-is-uispy-exe.aspx
• UI Automation Verify
  http://www.codeplex.com/UIAutomationVerify
• UI Accessibility Checker
  http://www.codeplex.com/AccCheck

Just a note - We are aware of the broken table-of-contents (TOC) with MSDN Library “UI Automation for Win32 Applications” section and apologize for any inconvenience. Once the TOC is fixed, we should have all the latest SDK contents featuring the new UI Automation Specifications implemented by Windows 7. In addition, the navigation is supposed to be replaced by “Windows Automation API: UI Automation” containing additional content.