Integrating existing testing systems with Microsoft Test Manager, Team Foundation Server and Visual Studio

I have been working with different partners in the testing space for the last couple of months and a frequently asked question is:

“Where do we start integrating our test tools with the Microsoft application life cycle platform?”

And the answer is relatively straight simple

“You need to integrate yourself into the development workflow…we use Team Foundation Server for that ALM workflow engine”

Unfortunately simple doesn’t always mean comprehensible…so I typically go on to say:

“…So that you can your tests run as part of the build process –and let the development team know if the tests passed/failed”

…At this point the light comes on and the question usually becomes:

“how do I do that?”

A generalized pattern for a tool targeting a manual test team might be:

1. File bugs from your IDE into TFS – if you can extend the existing IDE
2. Create Test Cases in TFS
3. Create a Tests type in Visual Studio that knows how to execute your existing tests
4. Set the newly created Visual Studio Tests to be executed during the automation testing

 

Below i list a couple of ways of achieving these steps:

Step1. File bugs from your IDE into TFS

This is very straightforward and I have covered it in the post:

Notepad for Workitems: how to programmatically create workitems in Visual Basic for Team Foundation Server 2010

Step2. Create Test Cases in TFS

This is just pretty much the same thing as the TFS Notepad – but instead of a bug it is a test case.

 

        static ITestManagementTeamProject GetProject(string serverUrl, string project)

        {

           TeamFoundationServer tfs = new TeamFoundationServer(serverUrl);

           ITestManagementService tms = tfs.GetService<ITestManagementService>();

          return tms.GetTeamProject(project);

        }

 

        private void Button_Click_1(object sender, RoutedEventArgs e)

        {

            string serverurl = "http://localhost:8080/tfs";

            string project = "Training1";

            ITestManagementTeamProject proj = GetProject(serverurl, project);

            ITestCase tc = proj.TestCases.Create();

            tc.Title = "Training1";

            tc.Save();

        }

Step3. Create a Tests type in Visual Studio knows how to execute your existing tests

This step is probably the most work as you will need to create a wrapper for the existing test execution engine so testers using the Microsoft test tools can interact and get the results of the test being run.

In the example below I am just shelling out to and checking for return…Hoping anything done in the wild will be a little more sophisticated than this!

Imports System.Text

Imports Microsoft.VisualStudio.TestTools.UnitTesting

 

<TestClass()> Public Class UnitTest1

 

    <TestMethod()> Public Sub TestMethod1()

        Dim p As New System.Diagnostics.Process

        ' // Redirect the output stream of the child process.

        p.StartInfo.UseShellExecute = False

        p.StartInfo.RedirectStandardOutput = True

        p.StartInfo.FileName = "c:\mobiletester\mobiledevicetester.exe"

        '// The line above just shows how you could launch ANYTHHING

        p.Start()

        '// Do not wait for the child process to exit before

        '// reading to the end of its redirected stream.

        '// Read the output stream first and then wait.

        Dim output As String = p.StandardOutput.ReadToEnd()

        p.WaitForExit()

        If CType(output.ToString, Integer) = 2 Then

            Assert.Fail("Test Failed")

            'At this point consider automatically filing a bug

        Else

        End If

    End Sub

 

End Class

After the Test is created you will want to make it available as a template. 

This way testers and Automation engineers won’t have to manually add a bunch of references or paste in blocks of code…luckily we make this pretty straight forward.

 

Note: Anna Russo has done a better sample using the generic test; please see:

http://visualstudiogallery.msdn.microsoft.com/ef313348-8fd1-463b-9404-7472db99fda5

 

 

 

 

Step4. Set the newly created Visual Studio Tests to be executed during the automation testing

In step 2 you created a Test Case- but how do you get it to execute your Visual Studio created in step 3?  

Turns out there are a couple of ways to do this; using Visual Studio, import the test cases using TCM finally and probably the method our partners and ISV will want to use programmatically.

Here are the directions for each:

1.  How to: Associate an Automated Test with a Test Case 

2. tcm: Importing Automated Tests into Test Cases

3. How To: Associate automation programmatically

 

 

For its worth, in Microsoft Test Manager – there is a tab called “Associated Automation” but it is intended to show testers which tests were set to run during automation and therefore is read only.