UITest framework – Winforms Plugin - Part 1

Article
04/13/2010

Introduction

UITest framework supports Windows Forms (in short Winforms) applications developed using .Net 2.0/3.5/4.0 framework. Ituses MSAA (Microsoft Active Accessibility) which is a part of Windows Automation API 2.0 to drive the automation on Winforms/Win32 controls.

Microsoft Active Accessibility helps users to get programmatic access to user interface elements from outside the application. The primary condition for UITest to work on any application is proper accessibility implementation from the application.

Accessibility Basics

The main idea behind Accessibility is to provide the functionality to access UI elements programmatically to get information about or manipulate these elements. UI elements that support this functionality are called accessible. In most cases this means that a UI element supports the IAccessible interface. You can also say that an accessible UI element is represented by the IAccessible interface in the world of Active Accessibility. So if a control is accessible if would provide an object of IAccessible which will in turn be used by UITest to fetch the important properties from it e.g., Name, Role etc. All the standard winforms controls provide accessible support by default and user defined custom control can also be tested using UITest if it has overridden IAccessible interface.

Recommendations for Accessibility Support for Winforms controls

· Unique Name among sibling: It helps in finding the control uniquely during playback

· Correct ControlType: It helps in identifying the control properly and recording the correct action on it.

· Parent/Child Relationship: If the parent child relationship is defined correctly for the control it will help recorder/playback to record/search for the control.

· Firing of correct events: Recorder records actions based on the event fired from the control so if the controls is able to fire the correct event the recording would be correct.

Refer the following link for detailed idea on MSAA and accessibility: https://msdn.microsoft.com/en-us/magazine/cc301312.aspx

Recording for Winforms controls

While recording any interaction through keyboard/mouse by the user under the desktop will be captured in the form of action and the control so that during playback the action can be performed on the same control. There are different kinds of information we store in order to search and do the action on the control.

· QueryID of the control in the form of some set of properties that help to find the control during playback

· Aggregated UITestAction

QueryID of a Winforms control

Technology of the control

This is the basic requirement of the recorder. The recorder first needs to determine what technology does the application belongs to so that it can use the appropriate accessibility API’s. UITest framework architecture enables this feature with a plug-in model where a plug-in can be registered for a specific technology. For MSAA plug-in it identifies a Winforms window based on the class name of the window containing “WindowsForms10.Window” as mentioned above. However UITest uses MSAA as a default plug-in also if the window does not qualify search logic of any plug-in. So if there is a Win32 application window, MSAA plug-in will be used to record and playback the actions.

QueryID of the control

To search for a control during playback we store some of its properties in the form of QueryID. To put it in a simple term this is the address of a control. It stores all required information of the hierarchy of the control so that it can be traced back during playback. A typical Winforms QueryID will look like this

“<Top Level Window’s Query Element>;<Immediate Parent’s Query Element>;<Control’s Query Element>”

Following is the brief description of each of the component:

TopLevelWindow and its search properties

RnP tool first records the information about top level window (TLW) of the control the user acted on so that it can be searched during playback. This information is vital because without this the control has to be searched on each window opened in the user’s machine. We try to get the following properties for the TLW:

Property	Description
ClassName	There is a predefined class name format for all the winforms controls as follows: [WinformsVersion].[Win32ClassName]. [DomainQualifier]. app. [AppDomain.CurrentDomain.Id]. So in case of window we generate the class name like WindowsForms10.Window with contains property just to avoid the failures when the other value changes.
Name	We generate the ControlText (i.e., title) of the window as the name for the top level window. So during recording if the title of the window gets changed the new one will also be recorded under the same top level window to avoid the search failure during playback.
ControlType	It will be ‘Window’ for the top level window always.

Parent of the Control

While recording when the user acts on a control the recorder navigates to up till the desktop and generate either its immediate or some intermediate parent in the query id so that the control can be searched during playback under this.

Following are the two type of parent can be generated for a Winforms control:

1. Win32 parent: In any Winforms UIApp there are two different trees one containing MSAA controls and another Win32 controls. The search in Win32 tree is pretty fast and more reliable.So the recorder would try to generate a Win32 parent for the control most of the time e.g., Window (which comes as the invisible parent of the control). Following are some of the properties used in QueryID to identify the control :

· ControlID

· ControlName

· ClassName

· ControlType (window in most of the cases)

2. MSAA Parent: When it is not possible to get a Win32 parent we generate the MSAA parent as the parent in the QueryID. Following are the properties recorded for this

· Name

· ControlType

· Value (for some specific controls e.g., cell in the data grid)

Control

To search for a control under the parent (mentioned above) we store some of its properties to identify it uniquely. Following are some of those:

· Name: The name is recorded in most of the cases to identify the control, this is basically the Acc_Name returned by the IAccessible interface for the control.

· Value: Some time there are many controls with the same name hence we generate value as the search property. Eg. cells in datagrid view have same name but value can be a unique way to find inside a row.

· ControlType: This helps in identifying the type of the control.

So the query id for a control inside a Winforms application would be of form A.B.C. Following is the simple example of QueryID for a control

A => Top level window

|_B => Default window around the button C with ControlName and ControlType as 'Window' as the search property

|_C => Button with Name and other properties

Additional Search Properties

Apart from the control specific properties, some other properties may be required to identify the control, e.g., if the control is nameless or the parent has to be expended before looking for a control e.g, tree item. Following are some of those.

Search configuration of the control

We have some predefined search configuration which helps to narrow down the search space (search only in visible controls) or to do perform some prerequisite before actually starting the search (expand parent tree node before searching for the child node). Following are the different search configs:

SearchConfig Parameter	Description
AlwaysSearch	UITest uses a cache while doing actions on an Application by adding Always Search in the search config user can force UITest to not to use the cached value for the control.
DisambiguateSearch	If the parent and the controls properties are same there are chances that UITest would start doing action on the parent itself. This config can be used to ask the playback to act on its child rather than the parent itself.
ExpandWhileSearching	Expand thecontrol before looking for the other control inside it. E.g., TreeView
NextSibling	Search in the siblings inside the container. Sometime if the control is nameless we may iterate from the named control inside the same container to reach the control.
VisibleOnly	Search only in the visible control. It helps to reduce the search space.

NextTo

This is generated for the nameless control. It contains a integer value and another control info. Offset is basically the distance from the named control among its sibling. So during playback the actual control is found using named control and with the offset value.

Instance

NextTo is generated for a limited no of distance i.e., if there is no named control within a specific range from the nameless control, the instance property is generated which is its position among all similar controls having same search properties.

UITestAction

The manual actions performed by the user is filtered and aggregated to a well-defined action using Aggregators. Following are some of the basic actions on common controls that do not need the aggregation/filtering.

Control Name	Possible user action	Recorded Action
Button/Hyperlink/Label/TextBox	Left click	MouseAction(SingleClick) with Left as the mouse key
	Right click	MouseAction(SingleClick) with Right as the mouse key
	Double click	MouseAction(DoubleClick) with Left as the mouse key

Aggregators

UITest uses some predefined set of rules to filter out/combine a group of actions to a well-defined action on the control. For example if a user

types a string say 'xyz' in the edit box

deletes 'z' using backspace

Types 'abc' again.

The simplest raw recording would have been

SendKeys('xyz')

SendKeys("BackSpace")

SendKeys("abc")

Though this will work in normal way but this sequence of action is not well defined for a better testing. Instead of having raw actions in the recording strip UITest will record single SetValue('xyabc") action on the edit box. The advantage is the state of the textbox will always be correct after the action irrespective of the initial state of edit box.

Another common instance is selecting a value in the combo box which compromises following steps

Click on the expand button

Select a list item

UITest would record this as SetValue on the combobox. Following are some of the common controls and aggregated actions which will get recorded.

Control Name	Possible user action	Recorded Action	Comment
TextBox, RichTextBox	Type xyz in the control	SetValue(“wyz”)	All the intermediate actions will be aggregated and the final value will be recorded as SetValue action
ComboBox / NumericUpDown	Select some item say Item1 in the control	SetValue(“Item1”)	Intermediate actions e.g., clicking on expand button selecting item will be eaten up, the only action recorded will be SetValue on the control
CheckBox / Radiobutton	Selection/DeSelection	SetState(Checked/Normal/ Intermediate)	Depending upon the final state of the control the SetState action will be recorded, for example in checkbox if user does two clicks the final state of check box would be recorded as SetState
DateTimePicker / MonthCalendar	Selection of a date	SetValue(“dd-MMM-YYYY”)	Clicks on the date or other controls would be eaten up.
MenuItem	Click on a leaf/intermediate menu item e.g., In notepad click on File menu item then on New	Mouse.Click(File.New)	The first click on File menu item will be eaten up and another click on the child will be stitched with the parent menu item i.e., File.
TreeView	Expand a node	SetState(expanded) on the tree item	SetState(expanded) on the node. Same like menu item here also the intermediate clicks on the ancestor tree items will be eaten up
	Selection of a leaf tree item	MouseClick(treeitem)	The final click would be recorded on the tree item which can be searched while expanding its ancestors using QueryID during playback
DataGridView	Enter data in cell	SetValue(data) in cell	Though the user types value in edit control but it is be recorded on cell as the edit box appears only when user clicks on the cell
	Select a checkbox inside the cell	SetState(Checked/Normal) on cell	It will also be recorded on the cell only to make sure the search passes during playback
	Select an item in the combobox inside cell	SetValue(item) on cell	The intermediate actions on the combo box will be aggregated out
	Navigation using keyboard (Tab/Up/Down /Left/Right) keys across cells	SendKeys(Keyboard_Key)	We record this on the cell

Playback for Winforms controls

To play a recording strip following are the major steps taken by UITest framework:

Search

Using the top level window and technology information, playback uses the right accessibility API to locate the control. Search has lot features to be successful even if the application is going through changes. Search algorithm for Winforms is complex as many applications do not have proper accessibility support.

There can be different mechanisms to search for a control inside desktop using different kind of UI Trees. For example UITest uses one of the following while searching for a control

1. Win32 Search

2. MSAA Search

Win32 search is fast but can only be used for the windows. UITest would fall into the appropriate searching mechanism depending upon the control in hand. Following are the steps while searching for a control

1. Search for the top level window (use smart match if the search fails for the exact value)

2. Search for the intermediate/immediate parent of the control (Win32/MSAA search based on the control’s properties)

3. Search for the control inside its parent

Following is the detailed explanation of our search mechanism:

TopLevelWindow Search

When the playback starts first it tries to find top level window with the following logic:

1. Pass 1: Search only in all visible windows inside desktop (no timeout) with SmartMatchOption turned off

2. Pass 2: Search in all visible + minimized windows with a timeout of 15% of total search time out value (e.g, 2 min by default). In this pass also the SmartMatch option is set to false

3. Pass 3: SmartMatch is enabled in this and with a timeout value of (SearchTimeout - 15%ofSearchTimeout)

SmartMatch

Some time it is possible that the title of the window is not same as it was during recording. For example when an application has multiple pages inside it the title would have the page name along with the application name on it. So there are chances that during playback the title is not same as it was during recording. e.g., Internet Explorer. Smartmatch helps in these cases by enabling the search to pass by selecting a window that matches the title based on predefined heuristics.

Control Level Search

When A is found, we look for B inside A and then C afterwards. For control level search there are 4 retries without any timeout. If the control is not found in these 4 retries, there will be a SkipIntermediate Search where any C will be searched under A irrespective of B.

WaitForReady

Sometime in an application there might be some controls which take time either to load or to get enabled. For example in file upload application the save button will be disabled till you file is loaded fully. So if a user has a recording strip for the following actions:

1. Launch the application

2. Select a file

3. Click on the save button

In above scenarios there are fair chances that if the file is large enough the save button would not be enabled and playback can just go and click on the save button. WaitForReady feature helps UITest to resilient in this scenario by verifying whether the control/app is busy or not before actually doing the action. Playback engine uses ThreadBased WFR with following kind of waiting mechanisms:

· UI thread

· All threads

· Disabled

Following are the two properties related to WFR which can be set by the user:

WaitForReady Level

FASTFORWARD

In FASTFORWARD scenario WFR for the foreground is set by default. This is not configurable.

CUIT

A user can change the WFR settings using following APIs in CodedUITest.

Playback.PlaybackSettings.WaitForReadyLevel defined under (Microsoft.VisualStudio.TestTools.UITesting) can be used to get/set the WFR options.

The possible WFR options are defined in an Enum named WaitForReadyLevel in Microsoft.VisualStudio.TestTools.UITest.Extension.

public enum WaitForReadyLevel

{

AllThreads,

Disabled,

UIThreadOnly,

}

WaitForReady Timeout

The default timeout for WFR is 1 min in both FASTFORWARD and CUIT scenario. One can change this value as follows:

FASTFORWARD

This is not exposed through test settings but can be re-configured in mtlm.exe.config file.

CUIT

This can be get/set using the following property :

Playback.PlaybackSettings.WaitForReadyTimeout

Ensure Visible

UITest has very good support to playback the action on the control even if the control is not in viewport. Consider the following scenarios

· Application is visible but the control on which Action is to be performed is not viewable.

Application is outside the screen area

· Application is obscured by some other Application

In the above Scenarios UITest will ensure that the control is made visible and then action is performed on it. This is absolutely essential as UITest does Actual Mouse/Keyboard actions and the control must be visible for that. Because of the advanced support for ensure visible. All actions done on the Scrollbar during recording are aggregated out getting us a very clean recording.. Following are some of ways in which UITest tries to bring the control into view port:

1. DoDefault action on the control using MSAA APIs

2. By setting focus on the control

3. Using keyboard navigation keys (up/down/left/right)

4. Using scrollbars (if exist in the container)

Action

Each action that was recorded has related playback logic. The logic depends on the combination of control type and the action type recorded. Once the action is done playback tries to verify whether did it really happen on the same control or not with the help of UI synchronization.

UISync

In automated UITesting it is important to verify whether a particular action has really reached the appropriate control or not. For example SimpleMouseClick on a button may click somewhere else in the app. RnP uses UISync to verify this with the help of windows messages.

Following are some of the instances where we disable UISync verification

· Different process arch (x86/x64) i.e., playback is a 32bit process but if the AuT is running as 64bit process.

· For modifier keys (Alt, ctrl etc.)

· SendKeys where it do not go to the target element rather some of its child e.g., combobox

· For disabled controls as they don’t respond to any of the window messages. There is cons of this say during playback the button got disabled hence the click on the button would pass as UISync is disabled but the next action may fail depending upon the AuT’s state. So user has to take care of this either by looking at the snapshot or logs after playback failure.

Summary

In this part 1 of the blog we have seen how Winforms plugin records and playback actions on Winforms/Win32 application. In Part 2 of the blog we will the known issues and how the user can troubleshoot issues in record and Playback on Winforms applications.

Author – Deepak Singhal

SDET - CodedUITest

Share via