In Visual Studio 2010, we introduced the ability to run tests in parallel.  Many machines today have multiple CPU’s or a CPU with multiple cores, so it makes sense that we may want our tests to run in such a way to use all the cores we have available on our machine.  This will effectively increase the number of tests running at the same time, which will reduce the time to run all the tests.

Decisions

We looked at all the different test types and functionality that we have, and made the following decisions:

1. The only test type that we would support for multi-CPU/core execution is the unit test.  The decision was based on various things such as: Is the test type doing UI testing? Is the test already doing some form of parallel execution? Is the test type hosted in a different host adapter (we may not have control over how the tests are executed)?  Are tests part of another test which may ensure a certain order of execution?  Is the test a unit test extension? (we may not have control over how the test is executed)? The answer to all these questions led us to support only unit tests.
2. Data Adapters cannot be enabled.  Since data collection is based on test events, having data collection while multiple tests are execution quickly could slow things down.  Also, the data itself may not correctly be isolated to the test specified, but would include data or information from other tests that are executing.
3. This only occurs for local execution through Visual Studio or MSTest.  Since remote execution allows you to already divide up tests across multiple agents, we have decided not to enable this scenario for remote execution.  Also, this currently is only available in Visual Studio or MSTest.

Requirements

So, you ask, get on with telling me what I need to do!

Okay, but first we must remind you, and it also kind of goes without saying, but I will say it anyway; you need a machine with multiple cores.  Virtual machines can be used but you will need to make sure that the machine has multiple CPU’s.  An easy way to find out is to view the task manager.

My machine is single CPU, but has multiple cores.

Also, your tests must be thread-safe.  Only you can ensure that they are.  Failure to do so can result in incorrect results, deadlocks, and a lot of head-aches.  Take some time to review the thread-safe link and review your code for thread-safety.

How to: Enable parallel test execution

Okay, so now, how do we do it?  I will put all the steps from above into this list

1. Ensure you have a multi-core/CPU machine (see above requirement).
2. Ensure you are running only unit tests (see above decisions)
3. Ensure your tests are thread-safe (see above requirement)
4. Ensure you do not have any data adapters on (see above decisions)
5. Ensure you are running locally (see above decisions)
6. Modify your test settings file.
   Right-click the test setting file and select “Open With”
   
   
1. Open as Xml
   
   
2. Set the parallelTestCount attribute on the Execution element
    
   
   Options are:
   attribute not specified = 1 CPU/Core used (default)
   0 = Auto configure: We will use as many tests as we can based on your CPU and core count
   n = The number n of tests to run in parallel to use (if you do not want to use all of your CPU/cores)
   
3. Save your settings… and you are done.

Result

I have two cores, and so here is my resulting run:

As you can see, we have two tests running at the same time.  If we were running this in serial, we would at least take 10 seconds.  With parallel tests we got this down to 6 seconds

Of course there are many other factors that affect this number.  There is the cost of starting and tearing down the run, so you will see a lesser effect if you have a few tests.  It also depends on the number of CPU/cores you have and of course how fast your tests execute.

Good luck and hope this helps speed things up for you.

Bruce Taimana
Program Manager
Visual Studio Team Test