Writing a Visual Studio 2012 Unit Test Adapter

Writing a Visual Studio 2012 Unit Test Adapter

Rate This
  • Comments 12

Visual Studio 2012 comes with an extensible Unit Test Framework. Many of your favorite Unit Test frameworks (xUnit.Net, nUnit..) already have built adapters to Visual Studio.  Peter Provost maintains a list of adapters here.  If you have another unit test framework you like, you can build an adapter super easy. In this article, I will walk you through the steps required to build a Visual Studio 2012 Unit Test Adapter.

 

For the purposes of this discussion, I am going to write a Unit Test Adapter for XML files. The XML file will contain tests in the following format.

image

After we install the new XML Test Adapter, we will see the following status in Test Explorer.

image

 

Step 1: Create a Discoverer

image

A Discoverer implements the ITestDiscoverer interface & has a single Method to discover tests.

Here is the code for our XMLTestDiscoverer.

image

I load the XML and read the name attribute on each Test node.

I create a TestCase and send it to Discovery Sink (which is a component of the Unit Test Framework).

Step 2: Create an Executor

image

An Executor implements the ITestExecutor interface which has 3 methods. 

  • Cancel is called to cancel the execution of tests.
  • RunTests has two overloads.  The first one takes a list of test cases and the second one a list of source file names.

 

For the XML Test Executor, I need to read the outcome attribute of the Test nodes. I see that the code is similar to what I have in TestDiscoverer. I refactor the code into a GetTests Method as shown below. I pass in null for the discoverySink when calling this method from the Executor.

image

The full code for the Test Executor is shown below. In the RunTests method, I create a TestResult and pass it to the FrameworkHandle (another component of Unit Test Framework)

image

 

Step 3: Create a TestContainerDiscoverer

image

Test Container Discoverer provides Visual Studio Test Explorer Integration. It has a list of Test Containers and the Unique identifier for the Executor. It also has an event handler for cases where Test Containers are updated.

The constructor for XMLTestContainerDiscoverer registers various event handlers for Visual Studio Solution and Project events.

image

 

Here is my code which identifies TestContainers. Visual Studio APIs are used to identify the elements in the loaded projects and pick out the xml files.

 

image

You can see similar code present in the Chutzpah Adapter. I have taken the Chutzpah code and modified it to built by Container Discoverer. nUnit & xUnit.Net have a different implementations of Test Container Discoverers.

Step 4: Build an Installer

You would have seen in the earlier code snippets that XMLTestContainerDiscoverer is implemented as MEF component. I will use a VSIX project to install all the components that we built earlier. Here is how the VSIX Manifest looks like.

image

 

The full source code for my XML Adapter is attached.

Summary

In this article, I showed you how to build a unit test adapter for XML files and integrate it with Visual Studio. After you install the VSIX from this project, you can open any solution with XML files in the  format defined earlier and you will be able to see the tests listed in Test Explorer. You can run them from the Test explorer and it will display the outcomes defined in the XML file.

 

Resources

I have referred to 2 articles to build this one. Special thanks to Matthew Manela & Bhuvaneshwari K.

1. http://blogs.msdn.com/b/bhuvaneshwari/archive/2012/03/13/authoring-a-new-visual-studio-test-adapter.aspx

2. http://matthewmanela.com/blog/anatomy-of-the-chutzpah-test-adapter-for-vs-2012-rc/

 

For a detailed list of topics related to Unit Testing in Visual Studio 2012, visit http://blogs.msdn.com/b/mathew_aniyan/archive/2012/05/17/content-index-for-unit-test.aspx

 

Mathew Aniyan

Program Manager – Visual Studio ALM.

Attachment: XmlTestAdapter.zip
Leave a Comment
  • Please add 4 and 6 and type the answer here:
  • Post
  • Awesome.

  • Glad to see this kind of support.

  • Too bad the test runner is junk.  The one in VS2010 was great.

  • Hi, is it only possible to run test with a custom adapter locally in VS or is there also a way to run tests using the same adapter on TFS centrally?

  • @Niels

    Instructions for using 3rd party test adapters is available in the blog - blogs.msdn.com/.../build-on-the-team-foundation-service.aspx

    Same procedure can be used for any test adapter you build.

  • hi Mathew, what would be the license terms for this project source?Does it come under Microsoft public license?

  • Hello :)

    I have a little problem.

    In the cancel method i use the frameworkHandle to SendMessage to Visual Studio. When i open visual studio, run tests and cancel the tests the message is displayed in execution output.

    If i try to run and cancel again the message not is displayed int the output.

    Its a normal behavior? I need shutdown a process in cancel method, but only the first time i use the cancel it works.

  • Nice post. In order to find these in the GUI, I just had to add the .xml file as an existing item to a C# test project. Is there a preferred way to structure a custom test project? Would it make sense to define a template for doing that so that in the New Project wizard I would have "New XML Test Project", or is that overkill?

  • By the way, if you install this adapter, and then write your own adapter and happen to use ".xml" as your extension, that seems to cause a conflict. It would appear that for best results, no two ITestDiscoverer derived children should use the same FileExtension attribute.

    For debugging, you need to set the VSIX project as the startup project for debugging, since it's configured to start the experimental instance of devenv.exe.

    -Jay

  • For some reason, I'm not able to stop at breakpoints in the XmlTestExecutor methods, or to see any System.Diagnostics.Trace statements from there. (I'm able to break in other classes and methods in the project). Is there some kind of caching or optimization that happens to this class that prevents debugging it?

  • OK, so to debug the discoverer and executor, you need to attach to those child processes of devenv.exe specifically.

  • i am wondering where the "sources" element comes from. I am modifing a persons test adapter code (Idiscoverer) and unfortunately it is recusing. this posses an issue given that i have to deploy the test execution framework (huget) and it has test cases in my output directory. I am rather surprised that it doesn't just load from what is in the solution. Generally this won't be a problem in my test environmetn since i can actually pass the test target foo.ps1 in my case. However for folks working in visual studio the test exporer is loading everything.

Page 1 of 1 (12 items)