Learn to use Visual Studio, Visual Studio Online, Application Insights and Team
Foundation Server to decrease rework, increase transparency into your application and increase the rate at which
you can ship high quality software throughout the application lifecycle
After the release of Visual studio 2010, Microsoft Test Manager gained a lot of popularity and we heard a whole lot of feedback about the tool, both good and bad. We consolidated it and decided to fix the top items causing dissatisfaction to the MTM users, the top 3 being performance issues in MTM, multiline support for test steps and test data reduction. Below are the details of these and other improvements we did in Microsoft Test Manager in the Visual Studio 11 Beta release.
Significant performance gains in MTM
Improving the performance was a major ask from the Test Manager users and we have done a lot of investigations and work to make it better. The performance of scenarios (for both LAN and WAN connections) like connecting to a plan, displaying the tests inside a suite, starting testing inside test runner, saving bugs in test runner, listing of runs on the analyze runs activity, creation of lab environments has improved to a great extent.
The new version of test manager has a whole new activity for exploratory testing. This is very useful when you don’t have a full fledged test plan before starting the test plan - it lets you create rich actionable bugs without existing test plan and also enables you to create a regression test suite that you could use in later phases of your testing. Your exploratory testing could either be associated with a requirement work item type like a user story, or it could be a general exploratory testing session without context of any work item. It lets you file rich bugs with action steps, video logs, your comments, screenshots, file attachments and a lot more. You could also create test cases with pre populated action steps, that get added to your test plan.
Multi line support in test steps
This feature lets you add multiple line in the same test step with the use of ALT+ENTER key combinations, just like Excel. This can be particularly useful to consolidate all the related actions in a single line and proves very handy along with the rich text feature for test steps, described below. This was also part of an update released in September 2011.
Test Data reduction
While publishing the results of automated test runs to TFS, MTM would upload all the binaries and the deployment items of the run which was bloating up the TFS Database and causing performance issues in opening the published run results, typically in cases of large deployment items. Now by default, it uploads only the trx and data collector attachments, not the deployment items or test binaries, except for code coverage enabled runs (as part of build definition). This setting can be changed via a new Boolean attribute in test settings - 'UploadDeploymentItems'. This is also available as a QFE here.
Making a copy of a suite along with test cases across plans
We now have the ability to clone test suites across plans. This can be very helpful in letting you work on multiple releases in parallel. When you clone a test suite, the new suite will contain copies of all the test cases in the source suite. The new test cases that get created will not retain the historical data like test runs, bugs, results, etc. This feature is available through the command line utility tcm.exe.
Rich text support in test steps
The Test Manager now comes with a rich text toolbar at the top of the test steps grid. This means support for all types of rich text formatting of the test steps' text, like bold, italics, underline, font types, font sizes, font colors, etc.
Test steps grid usability issues
There were significant improvements made to the usability of the test steps' grid. We added a splitter to adjust the space between the test steps and the parameters grid, so if you are not adding parameters, you can use its' space for your test steps. We also fixed multiple issues related to accessibility, navigation, selection, copy paste, etc.
Ability to access test case work item in Test Runner
The test case title in the test runner in now a hyperlink which open a read only view of the test case work item form wherein you can view complete details about the test case, anytime of your test execution. Also the description field in the test runner, which earlier used to support plain text only, can support rich text now.
Ability to mark test case result in MTM without launching Test Runner
Another much asked feature in the Microsoft Test Manager was the ability to mark single/multiple test case results, without the need to launch the Test Runner. The 'Run Tests' activity under the 'Test' tab has options now to mark a test case result as passed or failed, in addition to the earlier options of blocking the test case and resetting it to active state.
Results tab in Plan Activity
In the Plan tab, there is an option to view Results which gives a good view of the test plan result status. You can choose the test configuration and the test suite for which you want to see the results. Also, you can choose to see the results sorted by testers or test suites
Thanks for sharing useful information,
Is there any enhancement in recording using MTM?
What about lab in MTM is there any new major changes?
Good Stuff, Impressive! Is the "Results tab in Plan Activity" extendible? Can I modify the view or plug in my custom reports into it?
It would be a major improvement if we had a link on the Test Results Summary to view which test cases had failed or passed. I.e. in your screenshot above, be able to view the 5 Passed, 4 Failed, 3 Active. There doesn't seem to be an easy way to do this in 2010 which is a real flaw. I can easily see how many tests have failed but not which ones (unless I trawl through all my test cases in the Run Tests window - which if you have a few hundred tests in a regression suite is not ideal)
I just want to validate one thing can I run manual test without save any running result? Because sometimes I just want to try something by the running the test cases and I don't want MTM to recorded this running result unless I want it
@Ayman - Could you elaborate as to what enhancements you were expecting? We have added the exploratory testing flow in MTM in addition to the test runner functionality, as pointed above in point no 2. Yes, there have been many enhancements in lab too, you no longer have to install test agents on the computers that you compose into a lab environment. Lab Management does this for you when you create or repair your lab environments. See the following links:
@Tarun - Sorry, this is not so currently but is definitely there on the backlog.
@Stephen - I understand your point, this is currently not supported, but we will surely consider either what you said or else making it extensible such that the view can be customized.
Is MTM accessible through web interface?
@Suresh - This is not supported currently, sorry. We have this high on our backlog though.
Thanks Nivedita. Another question, how hard it is to customize MTM execution/rut state, for example if you want to introduce a state called "In Progress" for test cases that were started running but for some reason it was haulted in between. Example: Lets say a test case has 12 steps, I ran 8 steps but I had a situation to stop the exeuction, and come back and continue from step 9 instead of repeating from step 1.
@niveditabawa_MSFT - Though having been using MTLM since inception, i like the improvements in new version but not having web access is a big surpise. This is a basic need as far as usuage by other team members in a project goes... you want entire stakeholders, managers, execcutives , program managers etc in industry to get feel of what MTLM test plans are about, how test suites with results look like and last thing they would like to do is install a tool to view that information. As a user i would suggest web access for MTLM is MUST have if we want this tool to be adopted in every company across globe.
@QualitySpreader - Thanks for your feedback. I understand your requirement. This is high on our backlog and we will definitely incorporate your feedback.
@Suresh- We have this item in our backlog. I would request you to add your vote at visualstudio.uservoice.com/.../2038995-allow-tests-to-be-paused-allow-other-tests-to-be-
Hi one of our biggest problems is that if I start running a test case the Status should be set to "In Progress" untill the test run has finished and the Status is set to Pass or Fail. If you are working on a big project with say 100 testers then a significant number of test cases will have started being executed but not finished - on daily status reports they show up as Failed. they should show up as In progress.
@JB - We have this item in our backlog. I would request you to add your vote at visualstudio.uservoice.com/.../2038995-allow-tests-to-be-paused-allow-other-tests-to-be-
Hi, is it possible to integerate "Coded UI Testbuilder" functionality in MTM. So that during recording objects can be selected and added to the UImap.
Now we have to do this step in VisualStudio after importing a testcase.