Learn to use Visual Studio, Visual Studio Online, Application Insights and Team
Foundation Server to decrease rework, increase transparency into your application and increase the rate at which
you can ship high quality software throughout the application lifecycle
As part of Coded UI Test’s support for windows store apps introduced in Microsoft Visual Studio 2012 as detailed in this blog , there is now playback support for touch gestures on Windows Store Apps. These gestures are only a simulation and hence one would not need to have a touch monitor to use these API’s. Currently these API’s can only be used under the CUIT for Windows store project.
All the members in this family of Coded UI Test gesture API’s have two modes of operation so to speak:
In the former case the way a gesture searches for a control is similar to how a CodedUITest Mouse action does today. In the section that follows, we walkthrough each of the gesture API’s provided with examples. Please refer to the end of the blog post for a sample project demonstrating this support.
Gesture classification and walkthrough
The list of touch API’s can be broadly classified into two categories:
Before we proceed with the different gesture API’s provided and their usage it is important to understand the parameters that can be passed to them and their units.
Tap / Double Tap:
A tap simulates a left mouse button click.
// A simple tap on a button control. Gesture.Tap(button); // A tap at the bottom left part of the screen to bring up the start screen. A control-less gesture. Gesture.Tap(new Point(5, UITestControl.Desktop.BoundingRectangle.Height - 10));
A double tap on the other hand simulates a double click which can come into play in cases where one would need to select a word or depending on the app, magnify content.
// A double tap at a point inside a textbox control to highlight a word. Gesture.DoubleTap(textbox, new Point(10,10));
Press and Hold:
A press and hold gesture involves a single finger tap which extends for a given duration, the effect of which is similar to a right mouse click. This gesture is mostly used to bring up tool tips or context menus. For instance the following code brings up a context menu on an image control.
// A press and hold on an image control for a default duration of 1200 ms. Gesture.PressAndHold(imageControl);
A slide is used primarily for panning interactions but can also be used for drawing, moving or writing.
The values of the start and end points correspond to the coordinates on the screen or relative to a specified control. For instance a panning interaction from (200,300) to (400,300) will result in a slide from (200,300) for 200 pixels. In the following code sample the two slide actions have the same effect but each represents a mode of operation – Control and Control-less.
Point absoluteStartPoint = new Point(control.BoundingRectangle.Right -10, control.BoundingRectangle.Top + control.BoundingRectangle.Height/2); Point absoluteEndPoint = new Point(control.BoundingRectangle.Left + 10, control.BoundingRectangle.Top + control.BoundingRectangle.Height/2); Point relativeStartPoint = new Point(control.BoundingRectangle.Width - 10, control.BoundingRectangle.Height / 2); Point relativeEndPoint = new Point(10, control.BoundingRectangle.Height / 2); // A slide on the control for the default duration. Gesture.Slide(control, relativeStartPoint, relativeEndPoint); // A slide on absolute screen coordinates for the default duration (Control-less mode) Gesture.Slide(absoluteStartPoint, absoluteEndPoint);
A swipe is a short slide with the primary intention of selection. Swipe is always associated with a control and hence this is the only gesture that cannot be operated in a control-less mode.
// A swipe downwards on the control for a length of 40 pixels. Gesture.Swipe(selectableImageControl, UITestGestureDirection.Down, 40);
A flick gesture can be associated with a rapid slide in a scenario where one would quickly want to go through a long list of items. This gesture is characterised by its high speed.
Note: Although flick, slide and swipe seem similar they differ in their purpose of usage and are hence designed to be comfortably used for their appropriate objective.
Flick is fast if the gesture is to cover a large distance in a short duration and is slow if it is to cover a short distance for a long period of time. The below code snippet illustrates this on the start screen. Notice that as the duration of the gesture increase flick approaches slide.
Point relativeStartPoint = new Point(startScreen.BoundingRectangle.Width-100,startScreen.BoundingRectangle.Height/2); // A fast flick on the start screen. Gesture.Flick(startScreen, relativeStartPoint, (uint)startScreen.BoundingRectangle.Width / 2, UITestGestureDirection.Left, 2); // A relatively slower flick. Gesture.Flick(startScreen, relativeStartPoint, (uint)startScreen.BoundingRectangle.Width / 2, UITestGestureDirection.Left, 15);
Pinch/Stretch is essentially used for one of the following types of interactions: optical zoom, resizing or semantic zoom.
Note: Pinch never overshoots beyond the point of contact of the two touch points.
As an example the pinch gesture below performs a simple zoom out on an image control.
Point relativePoint1 = new Point(imageToZoom.BoundingRectangle.Width/4, imageToZoom.BoundingRectangle.Height/4); Point relativePoint2 = new Point(3 * imageToZoom.BoundingRectangle.Width/4, 3* imageToZoom.BoundingRectangle.Height/4); // A pinch on a control with each touch contact moving a 100 pixels towards each other. Gesture.Pinch(imageToZoom, relativePoint1, relativePoint2,100);
This gesture rotates a control in either clockwise or counter-clockwise direction.
Point relativePoint = new Point(imageToTurn.BoundingRectangle.Width - 10, 10); //Rotate an image about a focal point for 90 degrees clockwise. Gesture.Turn(imageToTurn, new Point(10, 10), relativePoint, 90);
Note: For a more generic scenario where one would want to rotate a control around its mid-point only the control and the angle of turn needs to be specified and the gesture logic takes care of the rest.
// Rotate the control around its midpoint for 180 degrees clockwise. Gesture.Turn(imageToTurn, 180);
This enumeration helps in easily specifying the 4 basic direction in which a gesture can be performed and has the following members:
Flick and swipe have overloads which accept this enumeration as a direction.
// To bring up the charms bar. Gesture.Slide(new Point(Screen.PrimaryScreen.Bounds.Width - 1, Screen.PrimaryScreen.Bounds.Height / 2), new Point(Screen.PrimaryScreen.Bounds.Width - Screen.PrimaryScreen.Bounds.Width / 4, Screen.PrimaryScreen.Bounds.Height / 2)); // To bring up the app bar. Gesture.Flick(new Point(Screen.PrimaryScreen.Bounds.Width / 2, Screen.PrimaryScreen.Bounds.Height - 1), (uint)Screen.PrimaryScreen.Bounds.Height / 4, UITestGestureDirection.Up, 5);
Is anyone actually using these touch based gestures for codedui? I have a feeling this is just a research project or is this more?
Support for touch gestures is part of the VS 2013 product. This is not a research project. Are you running into any issues that makes you believe otherwise?
When will this be available for WPF? This is what we would need...
Will it possible to upgrade an existing Coded UI Test to a Gesture supported test, i.e. will something like Mouse.Click be synonymous with Gesture.Tap?
In addition to my last comment, would the same be true for Mouse.DoubleClick (to Gesture.DoubleTap - though this may not be desirable in many cases)?
In Cross browser testing,Coded UI is not able to automate the window security pop up window which require UserName and Password for authentication to connect to server.
Due to which Coded UI framework is not recognizing the Upper level window hence all operation scenario are failed to perform their action.
if you have any suggestion please share.
@Michael: This is in our backlog. I'm curious to know what scenario you are trying out though.
@Dominic: Gesture support is currently only possible for Automated UI testing on store apps. Are you planning on migrating coded UI tests for store from mouse.click to gesture.tap? They are actually synonymous.
@sanjay: Can you please post this issue at visualstudiogallery.msdn.microsoft.com/11cfc881-f8c9-4f96-b303-a2780156628d with more details about the error you are seeing etc.
Is this input somehow generating traffic on the I2C bus?