TCM.exe – A General Introduction

Dennis Stone — Mon, 09 Nov 2009 16:09:10 GMT

Along with the new tools we’ve added in VSTS 2010 for testers comes the TCM.exe command-line application. This command-line isn’t intended to be an alternative for everything you can do in MTLM but rather to fill some gaps and enable some of the less common scenarios that still may be important for some testers.

You can get a general overview of TCM.exe over on MSDN in this article.

Some examples of things you can do with TCM.exe include:

Create an automated test run that contains test points across multiple test suites.
Create an automated test run without specifying a Team Build (you can specify the build directory where automated test binaries are picked up directly).
Publish results from a trx (MSTest / VS run results file) against your test plan.
Import VSTT Unit Tests (or even other VSTT test types) from your test binary into your team project. Creates a Test Case work item from each Unit Test (given some user specified constraints on what to import) and creates the association with the automated unit test for you. You can also sync this set of imported tests with a specified Test Suite.

Test Impact post by Daryush

Dennis Stone — Mon, 09 Nov 2009 15:35:52 GMT

There’s a nice post on Test Impact from within Visual Studio (geared towards Developers rather than testers) here.

Check it out if you’re interested in learning about a different side of Test Impact!

Test Impact Walk-through

Dennis Stone — Mon, 29 Jun 2009 21:04:00 GMT

One of the key areas of focus for Visual Studio Team System 2010 is to enhance the experience for testers. There’s already lots of overview information on many of the new features out there so I’m going to dive right in to one of my favorite new features.

In short, the Test Impact Analysis feature gives you a list of recommended tests based on the changes that were made in a build. As you run your tests through Microsoft Test and Lab Manager (code named Camano) you will have the option to run the Test Impact Data Collector which is one of the many built-in Data Collectors available while running tests. This particular data collector will associate the code paths you execute in your application with the test you are running.

When Team Build generates a new build it will analyze what code has been changed since the last build was done and then provide you with a list of the impacted tests. These are the tests that touch the same pieces of code that were changed since the last build.

For a more thorough explanation of the Test Impact Data Collector read this blog post by Amit Chatterjee.

The first thing you need is an Application Under Test, for this walk-through I’ll use a simple calculator application. I’ll have a C# Class Library with some simple Add, Subtract, Multiply, and Divide functions and a C# Windows Forms project that will provide some input fields for the numbers and some buttons that call the library function. I’ll skip the creation of this sample application, you an use the attached sample application, mock up something similar or follow along with your own test application.

Add your test application to Source Control
Create a new Build Definition for your application
1. Make sure “Analyze Test Impacts” in the Process tab is set to “True”:
Queue a new build
Start “Microsoft Test and Lab Manager” (aka Camano)
1. If the dialog below comes up click “Set context”:
2. On the “Set Plan Context” page click “New”
3. Enter a name for your Test Plan and click Save & Close
4. Go to Testing Center –> Plan –> Contents
5. Click “New” in the toolbar to add a new Test case
6. Add a Title for the Test case, mine is “Add – Basic”, I’ll repeat these steps and add “Subtract – Basic”, “Multiply – Basic”, and “Divide – Basic” as well. Here’s what my Test Plan Contents will look like when I’ve done this.
7. Now it’s time to switch to the Lab Center so we can set up our Test Settings
8. Switch to the Test Settings tab and click “New”
9. Enter a Name and click “Next”
10. To keep things simple uncheck all collectors except the “Test Impact Collector” and click “Save and Close”
11. Go back to the Testing Center.
12. Switch to the Track tab and go to the “Assign Build” activity
13. The build you created earlier will be in the “Available builds” list, select it and click “Assign to plan”.
14. Now it’s time to run our tests. Switch to Testing Center –> Test –> Run Tests
15. Select all of your tests and click “Run with Options”
16. In the Run Options dialog select your Test Settings and verify that your newest build is selected. Click Run.
17. Test Runner will now appear
18. Click Start test and record, then launch your Application Under Test. This ordering is important, you must start the test before launching the test application or the impact data will not be collected.
19. Perform each test now. For my Add test I’ll set 2 numbers in my text boxes and click Add, for Subtract test I’ll set 2 numbers and click Subtract, etc. etc. Run your test then click “End Test”, then start the next test. There’s no need to re-launch your application between every test. When you’re done click “Save and Close” to return to Manual Test and Lab Manager (MTLM). It should look something like this:
20. To verify the Test Impact Collector worked correctly you can open up one of your test results and look at it’s attachments, there should be an impact xml file listed there. If there isn’t, re-check your steps above.
21. Now, switch to Visual Studio and make a change to your application under test. I’m going to make a change to my “Add” method and check it in.
22. Queue a new build and wait for it to complete.
23. Switch back to MTLM
24. Go to Testing Center –> Track –> Assign Build and assign your new build to your plan. (See Step #12 if you need help)
25. Now switch to Testing Center –> Track –> Recommended Tests
26. Notice the “Add – Basic” test is now listed.

That’s it for this post, I look forward to any questions or comments you may have. If you have any questions on some of the steps above try looking them up on the MSDN site. Topics such as creating builds, queuing builds, etc. should be well covered there.

Dennis Stone's Blog : VSTS Testing

TCM.exe – A General Introduction

Test Impact post by Daryush

Test Impact Walk-through