Steve Lange @ Work

Steve Lange's thoughts on application lifecycle management, Visual Studio, and Team Foundation Server

Ordered Tests in TFS Build

Ordered Tests in TFS Build

Rate This
  • Comments 4

In an earlier article I discussed how to use and Ordered Test to control the execution order of Coded UI Tests (the same can be applied to other test types as well).  I received a few follow-up questions about how to do this in TFS Build so tests run in a particular order as part of a build.

Here’s one way that’s remarkably easy.

In my example, I have a project called JustTesting, which contains just a test project with 3 unit tests (which will always pass, BTW).

image

I put those tests into an ordered test:

image

In Solution Items, I open up my JustTesting.vsmdi file, create a new test list (called Ordered Tests), and add my ordered test to it.

image

Once that’s done, I check everything into TFS (my Team Project’s name is “Sample CMMI”.

Next, I set up a build definition (in Team Explorer, right-click Builds, and select “New Build Definition”).  Set whatever options you want (name, trigger, workspace, build defaults) but stop at “Process”.

In the section named “2. Basic”, you’ll see that by default the Automated Tests field is set to (something like): “Run tests in assemblies matching **\*test*.dll using settings from $/Sample CMMI/JustTesting/Local.testsettings”. 

image

Click on the ellipsis on the right of that to open the Automated Tests dialog:

image

Remove the entry you see (or leave it if you wish to include that test definition), and then click “Add”.

In the Add/Edit Test dialog, select the optoin for “Test metadata file (.vsmdi)”.  Use the browse button to find and select your desired .vsmdi file.  In my example, JustTesting.vsmdi.

Uncheck “Run all tests in this VSMDI file”, then check the box next to your test list containing the ordered test.  In my example, the test list is called “Ordered Tests”.  Your dialog should look something like this:

image

Click OK and you’re Automated Tests dialog should look like:

image

Click OK again, then save your build definition.

Queue a new build using this definition.  Once complete, look at the build report to see your test results.

image

image

It’s a few steps, but nothing ridiculous.  And I didn’t have to hack any XML files or do any custom coding.

Hope this helps!

Leave a Comment
  • Please add 7 and 8 and type the answer here:
  • Post
  • Hi, I have an ordered test that i would like to execute each time the project is built. Your solution would solve the purpose, but i was wondering, if you could also demonstrate, how to complete this process by manually editing the TFSbuild.proj file

    It would be very sweet of you, if you could do that...

    Thanks and regards

    Gagan

  • I don't see the Process Option you mentioned. Can you please let me know, where I can find that option. (I can see General, Trigger, Workspace, Build Defaults, Project File and Retention Policy Options)

  • Hi,

    Let me know whether you have checked the results in MTM or this result would be displayed aftr the TFS build execution?

    Thanks and Regards,

    Guru

  • The results of an ordered test are not viewed (nor is an ordered test run) via MTM.  MTM is designed for more manual testing and test case management.  Ordered tests are executed either in Visual Studio or via mstest.

Page 1 of 1 (4 items)