Integrating existing testing systems with Microsoft Test Manager, Team Foundation Server and Visual Studio

Integrating existing testing systems with Microsoft Test Manager, Team Foundation Server and Visual Studio

Rate This
  • Comments 4

I have been working with different partners in the testing space for the last couple of months and a frequently asked question is:

“Where do we start integrating our test tools with the Microsoft application life cycle platform?”

And the answer is relatively straight simple

“You need to integrate yourself into the development workflow…we use Team Foundation Server for that ALM workflow engine”

Unfortunately simple doesn’t always mean comprehensible…so I typically go on to say:

“…So that you can your tests run as part of the build process –and let the development team know if the tests passed/failed”

…At this point the light comes on and the question usually becomes:

“how do I do that?”

A generalized pattern for a tool targeting a manual test team might be:

  1. File bugs from your IDE into TFS – if you can extend the existing IDE
  2. Create Test Cases in TFS
  3. Create a Tests type in Visual Studio that knows how to execute your existing tests
  4. Set the newly created Visual Studio Tests to be executed during the automation testing

 

Below i list a couple of ways of achieving these steps:

Step1. File bugs from your IDE into TFS

This is very straightforward and I have covered it in the post:

Notepad for Workitems: how to programmatically create workitems in Visual Basic for Team Foundation Server 2010

Step2. Create Test Cases in TFS

This is just pretty much the same thing as the TFS Notepad – but instead of a bug it is a test case.

 

        static ITestManagementTeamProject GetProject(string serverUrl, string project)

        {

           TeamFoundationServer tfs = new TeamFoundationServer(serverUrl);

           ITestManagementService tms = tfs.GetService<ITestManagementService>();

          return tms.GetTeamProject(project);

        }

 

        private void Button_Click_1(object sender, RoutedEventArgs e)

        {

            string serverurl = "http://localhost:8080/tfs";

            string project = "Training1";

            ITestManagementTeamProject proj = GetProject(serverurl, project);

            ITestCase tc = proj.TestCases.Create();

            tc.Title = "Training1";

            tc.Save();

        }

Step3. Create a Tests type in Visual Studio knows how to execute your existing tests

This step is probably the most work as you will need to create a wrapper for the existing test execution engine so testers using the Microsoft test tools can interact and get the results of the test being run.

In the example below I am just shelling out to and checking for return…Hoping anything done in the wild will be a little more sophisticated than this!

Imports System.Text

Imports Microsoft.VisualStudio.TestTools.UnitTesting

 

<TestClass()> Public Class UnitTest1

 

    <TestMethod()> Public Sub TestMethod1()

        Dim p As New System.Diagnostics.Process

        ' // Redirect the output stream of the child process.

        p.StartInfo.UseShellExecute = False

        p.StartInfo.RedirectStandardOutput = True

        p.StartInfo.FileName = "c:\mobiletester\mobiledevicetester.exe"

        '// The line above just shows how you could launch ANYTHHING

        p.Start()

        '// Do not wait for the child process to exit before

        '// reading to the end of its redirected stream.

        '// Read the output stream first and then wait.

        Dim output As String = p.StandardOutput.ReadToEnd()

        p.WaitForExit()

        If CType(output.ToString, Integer) = 2 Then

            Assert.Fail("Test Failed")

            'At this point consider automatically filing a bug

        Else

        End If

    End Sub

 

End Class

After the Test is created you will want to make it available as a template. 

This way testers and Automation engineers won’t have to manually add a bunch of references or paste in blocks of code…luckily we make this pretty straight forward.

 

Note: Anna Russo has done a better sample using the generic test; please see:

http://visualstudiogallery.msdn.microsoft.com/ef313348-8fd1-463b-9404-7472db99fda5

 

 

image

image

 

image

 

Step4. Set the newly created Visual Studio Tests to be executed during the automation testing

In step 2 you created a Test Case- but how do you get it to execute your Visual Studio created in step 3?  

Turns out there are a couple of ways to do this; using Visual Studio, import the test cases using TCM finally and probably the method our partners and ISV will want to use programmatically.

Here are the directions for each:

1.  How to: Associate an Automated Test with a Test Case 

2. tcm: Importing Automated Tests into Test Cases

3. How To: Associate automation programmatically

 image

 

For its worth, in Microsoft Test Manager – there is a tab called “Associated Automation” but it is intended to show testers which tests were set to run during automation and therefore is read only.

image

Leave a Comment
  • Please add 5 and 3 and type the answer here:
  • Post
  • Why not use a generic test in step 3? That even allows you to write some xml (summaryresults) which is also supported by test manager.

    Generic test

    msdn.microsoft.com/.../ms182626.aspx

    Summary results

    msdn.microsoft.com/.../ms243174.aspx

  • Great suggestion!

    Let me add an example of this too!

    Thanks,

    Chuck

  • Hi

    Something that is missing in TFS is the concept of a "positional automated test" to position up data etc.

    For example, before you can delete a user, that user needs to exist:

    Postional test - Create user.

    Real test - delete user.

    I know you can have the create user and delete user in the same test, but on more complex scenarios this is not possible.

    BTW we have integrated our automated test framework into TFS, but do it slightly different to the above, this is what we have:

    1. Create test cases in TFS for the "Real" tests. These are linked to the automation with a custom field in the test case template.

    2. Execute the automated tests outside of TFS / MTM using our current process.

    3. Import the results from your automation run into MTM - we use a custom built tool for this which gets the automation id and updates the test result in MTM.

    This works for us as test cases, requirements, bugs etc are all linked in TFS.

  • I'm very intersted in TFS extensibility points - for example, how can I integrate a separate build system like TeamCity into the document flow of TFS-ALM?  

    Do you have any pointers to that kind of content?  

    Thank you,

    -Steve

Page 1 of 1 (4 items)