Automatically execute Coded UI test on different Lab VMs

  • Article

Imagine this scenario: you are working in setting up the Lab Management features of TFS 2010. The main purpose is to execute automated tests. The customer asked to have a lab environment able to test different clients OS, namely Windows XP and Windows 7, each equipped with a number of browsers.

Sure, you could have multiple environments each with a single but different client, but this will require much more resources; the environment has already three servers: domain controller, SQL Server and a Web server. Two environments needs 2*(3+1)=8 VMs, while collapsing in one you have 3+2=5 VMs.

You are using the end-to-end workflow, i.e. LabDefaultTemplate.xaml, to deploy the application and test it. The template wizard has two combo boxes: one for Test Configuration and one for the Test Settings.

2011-05-31_134520_no_arrows

So you define a test setting, using the proposed defaults.

2011-05-30_140104

But then you realize that both client VMs are in the Web Client role: tests will be distributed among them!

The solution is to:

  • using different roles for the two clients
  • create different Test Configurations for each client role

As you see below the role are editable

 2011-05-31_133811

In my case I gave distinct names to the client VMs, say “Web Client XP” and “Web Client 7”; then defined two Test Settings: in each setting, I selected a distinct VM where to run the test (see below picture).

clip_image001

Finally, I define two builds based on the E2E template, each pointing to the proper (automated) test settings, and linked to the matching Test Configuration (see below).

clip_image002clip_image003

So far so good, but having two build definitions is a bit inelegant. Later I found this blog post https://blogs.msdn.com/b/lab_management/archive/2011/05/26/configuration-matrix-testing-using-visual-studio-lab-management.aspx, that suggest how to add a second test run with different settings to the E2E workflow.

Happy building!