MSAA, UIA brief explanation

MSAA, UIA brief explanation

  • Comments 2

MSAA, UIA brief explanation

 

中文链接:

http://eparg.spaces.live.com/blog/cns!59BFC22C0E7E1A76!4008.entry

 

 

What is MSAA:

http://msdn.microsoft.com/en-us/library/ms971310.aspx

 

MSAA is the same as IAccessible Interface.

 

The interface is originally designed for disabled people. While in most cases, it is used as an implementation method of UI Automation. However, this interface is not rich enough to fit into all aspects of UI Automation, as it only provides the name, type, positions of the target elements. It does not expose other functions like ClassName, Orientation, nor the operation supports like Window Move, Selection, to say nothing of Event Support.

 

As a result, Microsoft believes there is a good reason to provide dedicated interfaces for UI Automations:

 

IAccessibleEx

IRawElementProviderSimple:

 

http://msdn.microsoft.com/en-us/library/dd561900(VS.85).aspx

 

In MSDN category, the two interfaces belong to Windows Automation API, not MSAA.

 

The first interface IAccessibleEx, sounds like MSAA’s IAccessible, but it is just a proxy for the Automation Client to obtain the IRawElementProviderSimple. The implementation of pattern and property are in IRawElementProviderSimple. IAccessibleEx lives for only two reasons:

 

1.      Provide the MSAA client a familiar way to use the new UI Automation API. The MSAA client is able to query for IAccessibleEx interface and consume the new feature without using the new style of UIA API. The client code can still work in COM style.

2.      Provide the UI control programmer the MSAA way to implement and expose new UI Automation feature for the control. The programmer just needs to implement the IAccessibleEx interface for the control in COM style, and the pattern and property UI Automation features are accessible from new UIA APIs.

 

http://msdn.microsoft.com/en-us/library/dd561882(VS.85).aspx

 

IAccessibleEx is just a bridge. In Win7, the implementation for the bridge is in UIAutomationCore.dll.

 

To use UI Automation, the client could choose to use UIA API directly, or, use the old style, which is obtaining IAccessible, and QueryService (not QueryInterface because the system needs to do some hook) for IAccessibleEx, and then just use it or continue to query for IRawElementProviderSimple.

 

After getting the IRawElementProviderSimple, it is very easy to consume the patterns and providers:

 

http://msdn.microsoft.com/en-us/library/ms746691.aspx

 

Please note, the AutomationProvider could be implemented in either UI Server or the Client side. To implement in UI Server, the control could derive from IRawElementProviderSimple and return the client request like this:

 

if ((m.Msg == WM_GETOBJECT) && (m.LParam.ToInt32() ==
                AutomationInteropProvider.RootObjectId))
            {               
                m.Result = AutomationInteropProvider.ReturnRawElementProvider(
                        this.Handle, m.WParam, m.LParam,
                        (IRawElementProviderSimple)this);             
                return;
            }

 

The client implementation benefits the old Win32 control as we do not need to modify any existing implementations for the control. For client side provider, it is also called as Provider/Automaton Proxy. In client implementation, the ProviderOptions property should return ProviderOptions.ClientSideProvider.

 

For Win32 and WinForm default implementations, they are in client side. The assembly is UIAutomationClientSIdeProvider.DLL. The ProxyHwnd.GetElementProperty’s implementation is very typical:

 

Getting UIAotmationID by sending WM_GETCONTROLNAME message

Getting Name property by reading MSAA’s Name property first, calling GetWindowLong and GetWindowText API the second and trying PostMessage for last.

 

I just found an old Automation Paper created in 2004:

http://msdn.microsoft.com/en-us/library/ms996405.aspx

 

The author Brian McMaster, who is a Test Architect now, educated people to use WM_GETCONTROLNAME message to get control ID, take care of the hierarchy of UI control, and watch out for the case that window owner and window parent are different, in the year when UIA was absence.

 

With above understanding, it is quite easy to implement the server side for WinForm:

 

1.      Override WndProc by calling AutomationInteropProvider.ReturnRawElementProvider to return WM_GETOBJECT message.

2.      Implement IRawElementProviderSimple and return ProviderOptions.ServerSideProvider.

3.      Implement the providers like ISelectionProvider.

 

The full code is available in the links above.

 

For UIAutomation client, besides the IAccessibleEx I talked above, there are two kinds of UIAutomation API, managed and unmanaged.

 

The managed API is in System.Windows.Automation namespace. The class name is AutomationElement. The functions are very direct to use.

The unmanaged API is a group of functions whose name starts with UIA.

 

In current design, the managed API uses unmanaged API internally, while the provider used in the unmanaged API are implemented in managed code. The relationship is a bit complex. And there are some features work fine in unmanaged API but may fail in managed API because of bug. In Win8, we may merge the implementation to reduce the maintenance cost.

 

The user is not allowed to add (extend) new property and pattern in UIA. You cannot register your new pattern provider or property. You cannot even use the functions from the interface directly:

 

http://msdn.microsoft.com/en-us/library/system.windows.automation.provider.automationinteropprovider.hostproviderfromhandle.aspx

 

"The interface returned by this method can only be passed back to UI Automation. Attempting to call a method on the interface will raise an exception"

 

If you really want to do it, follow the steps to get IAccessibleEx, and do your own QueryInterface.

 

Above explanation is for Win32 and WinForm. For WPF, it uses AutomationPeer to implement UIAutomation.

 

The communication between client and server for native UIAutomation implementation (AutomationPeer and ReturnRawElementProvider) is named pipe. The client provider with MSAA is by DCOM and Windows Message.

 

Additional links:

 

UI Automation Community Promise Specification

http://www.microsoft.com/downloads/details.aspx?familyid=A1FE1066-BF4F-44FC-834B-676B311E83A2&displaylang=en

 

UI Automation of a WPF Custom Control

http://msdn.microsoft.com/en-us/library/cc165614.aspx

 

The followings two docs are created by Microsoft, but I cannot find the link from MSDN. They explains UIA’s design goal, IAccessibleEx and IRawElementProviderSimple’s relationships very well.

 

[added on 9/24/2009]
Clientside provider information doc:
 
 
sxe ld UIAutomationClientsideProviders*
[/added]
 

http://www.accessinteropalliance.org/docs/Introducing_IAccessibleEx.doc

http://www.accessinteropalliance.org/docs/What_is_UIAutomation.doc

Leave a Comment
  • Please add 5 and 7 and type the answer here:
  • Post
  • PingBack from http://blog.a-foton.ru/index.php/2009/03/28/msaa-uia-brief-explanation/

  • Hi guys,

    This question is related to CUIT, but is strongly related to MSAA and UIA since I'm not sure which of them (or both?) I'm supposed to support to get CUIT to identify my control.

    Here's the story: I'm struggling to get Coded UI to work with a WinForm that hosts a custom control (which contains standard winform controls - buttons, text fields etc.)

    The winform is owned by a WPF application, which coded UI has no problem locating, and I managed to automate the menu: File > New > Work Item which launches the winform as a new window (owned by the WPF application, as a modal dialog).

    However, it fails to find the winform. I've implemented all the ControlAccessibleObject methods, and I can see in UI Accessibility Checker that my UIA tree is complete. However, my MSAA tree is empty, showing only 'root' > '(nothing)', both with role 'Invalid'.

    I should also mention that Coded UI uses UIA technology for the WPF form, but insists on using MSAA to try and locate the winform dialog. I'd be happy to tell it to use UIA instead, but the field is read only...

    How do I fix this? Isn't having a completely implemented UIA tree enough? Thanks!

Page 1 of 1 (2 items)