Buck Hodges

Visual Studio ALM (VSALM, formerly VSTS) - Team Foundation Service/Server (TFS) - MSDN

November, 2006

Posts
  • Buck Hodges

    How to run tests in a build without test metadata files and test lists (.vsmdi files)

    • 46 Comments

    [UPDATE 6/16/2010]  The VSTS 2008 release added support for test containers (/testcontainer) in the product, and the 2010 release added support for test categories.  This post now only applies to TFS 2005.

    Since the beginning, running tests in Team Build (or MSBuild in general) has meant having to use .vsmdi files to specify the tests to run.  Tons of people have complained about it, as it's a burden to create and edit the files, as either VSTS for Testers or the full suite is required in the 2005 release, and merging changes to the file is painful when multiple developers are updating the file.  While mstest.exe has a /testcontainer option for simply specifying a DLL or load/web test file, the test tools task used with MSBuild does not expose an equivalent property.

    Attached to this post is a zip archive containing the files needed to run tests without metadata.  There are three files.

    1. TestToolsTask.doc contains the instructions for installing the new task DLL and modifying the TfsBuild.proj files to use it.
    2. Microsoft.TeamFoundation.Build.targets is a replacement for the v1 file by the same name that resides on the build machine.
    3. Microsoft.TeamFoundation.PowerToy.QualityTools.dll contains the new TestToolsTask class that extends the TestToolsTask class that shipped in v1 and exposes a TestContainers property that is is the equivalent of mstest.exe's /testcontainer option.

    After you read the instructions (please, read the instructions), you'll be able to run all of your unit tests by simply specifying the DLLs or even specifying a file name pattern in TfsBuild.proj.  Here are a couple of examples.  The TestToolsTask.doc file explains how they work, including what %2a means.

    <TestContainerInOutput Include="HelloWorldTest.dll" />
    <TestContainerInOutput Include="%2a%2a\%2aTest.dll" />
    <TestContainer Include="$(SolutionRoot)\TestProject\WebTest1.webtest" />

    This new task will be included in future releases of the Team Foundation Power Toys (soon to be called Power Tools).  The TestToolsTask in the shipping product will support test containers and Team Build will support using it in the next release of the product.

    I started with code and documentation that someone else wrote and finished it off.  Thanks to Clark Sell for digging up an old email thread about the original work and Tom Marsh, development lead in VSTS Test, for pointing me to the code and doc.  Thanks to Aaron Hallberg and Patrick Carnahan, Team Build developers, for their help.  Thanks to Brian Harry for letting me post it.

    Please post your feedback in the comments to this post.  We'd like to know if this hits the mark, or if there's something else we need to support.

    [UPDATE 4/26/2007] New features: Test categories and test names

    Pierre Greborio, a developer over in MSTV, has contributed a great new feature: test categories.  Those of you who have used NUnit are probably familiar with the Category attribute.  Test categories allow you to execute specific groups of unit tests.  Unlike the test container feature, the test category feature will not be in Orcas.

    While the details are discussed in the TestToolsTask.doc included in the zip file attached to this blog post, here's the quick version.

    1. Add the Category attribute to your unit test method.
    2. Specify the category to run in your tfsbuild.proj file.

    The other feature that's new in this release is the support for test names.  This is equivalent to the mstest.exe /test command line switch.  The name that's specified is implicitly a wildcard match.  Specifying "Blue" as the test name, for example, will execute every test method that has Blue in the name, including DarkBlue and BlueLight.

    Pierre did his testing on Vista.  Because the dll is not signed, he ran into some trust issues.  If you hit a similar problem, he recommends this forum post for how to get it to work.

    [UPDATE 11/9/2006] Bug fix

    I've updated the attachment with a new zip archive file.  Unfortunately, the task I posted originally didn't result in builds being marked as Failed when the tests had errors.  I missed that in my testing.  The reason for the problem is that the v1 Team Build logger looks for the task name, TestToolsTask, and the power toy task was originally called TestToolsTaskEx.  With this new release, the task has the same name as the original v1 task, so that builds will be marked as failed when the tests fail.

    If you downloaded the original release, you simply need to copy the Microsoft.TeamFoundation.Build.targets and Microsoft.TeamFoundation.PowerToys.Tasks.QualityTools.dll files from the zip to get the fix (see the Word doc for the paths).

    Thanks to Thomas for pointing out this bug!

    tags: , , , , , ,

  • Buck Hodges

    Updated version of new TestToolsTask

    • 12 Comments

    I've updated the TestToolsTask that supports using test containers, in addition to the test metadata files supported by v1.  Unfortunately, the task I posted originally didn't result in builds being marked as failed when the tests had errors.  I missed that fact in my testing.  The reason for the problem is that the v1 Team Build logger looks for the task name, TestToolsTask, and the power toy task was originally called TestToolsTaskEx.  With this new release, the task has the same name as the original v1 task, so that builds will be marked as failed when the tests fail.

    If you downloaded the original release, you simply need to copy the Microsoft.TeamFoundation.Build.targets and Microsoft.TeamFoundation.PowerToys.Tasks.QualityTools.dll files from the zip to get the fix (see the Word doc for the paths).

    Thanks to Thomas for pointing out this bug!

    The updated version is attached to the original post.  I apologize for the inconvenience.

    tags: , , , , , ,

  • Buck Hodges

    All of the Sysinternals tools in one download

    • 3 Comments

    Microsoft TechNet put them all in one convenient download.

    Sysinternals Suite

    Published: November 1, 2006

    Introduction

    The entire set of Sysinternals Utilities have been rolled up into a single Suite of tools. This file contains all the individual tools and help files.

    Download SysinternalsSuite (8 MB)

    tags: ,

  • Buck Hodges

    Continuous Integration in Team Build for Orcas

    • 5 Comments

    Brian Harry has written a post about an SD Times article about TFS.  Since he is the head of Team Foundation, his posts tend to have interesting tidbits in them, such as the following.

    It talks about the lack of continuous integration and the ability to schedule builds. It also talks about lack of a gui editor for build definitions. I think it's fair to say that TFS Build is the weakest of our "out of the box" feature experiences. It does a good job serving as an integration point - pulling together data from version control, work item tracking, testing, the build process and populating the warehouse. However, much of it is manual. Build is one of our biggest areas of investment in Orcas - including better extensibility, continuous integration and lots of other features.

    The emphasis on the last part is mine, of course, since I didn't want anyone to miss it.  CI and other significant enhancements to Team Build are alive and kicking, and we are really excited about it.  And when we say CI is in Team Build for Orcas, we're not just talking about bolting on some add-on.  It's a first-class part of Team Build.

    If you care about these features, it will be really important to use them and give us feedback when they show up in a CTP (I don't know when that will be).  The sooner you give us feedback after the release of a CTP or beta -- and this is true across the product -- the more likely it is that we can make changes to address your feedback.

    Brian ends the post by mentioning his TFS roadmap, which he intends to publish by the end of the year.  He has sent it around internally, and it contains a fair amount of detail.  There's lots of good stuff to talk about in the near future, so keep an eye out for it.

    Happy Thanksgiving!

    tags: , , , , ,

  • Buck Hodges

    Building software against different versions of .NET with MSBuild in Orcas

    • 1 Comments

    Faisal Mohamood, a program manager on the MSBuild team, has written a pair of posts about the "multi-targeting" feature in the Orcas version of MSBuild.  Multi-targeting allows you to build your code against the .NET 2.0, 3.0, or 3.5 frameworks, according to your requirements.  This is great because it allows folks to have some projects use the newest framework and other projects remain on an older framework by simply specifying the tool set in the project files.

    MSBuild, Orcas, and Multi-targeting

    So if you were curious enough to dig into what the October 2006 Orcas CTP contained, you might have seen something related to MSBuild - namely, Multi-Targeting. Here's what the release notes for the CTP says, with respect to MSBuild and Multi-Targeting:

    MSBuild Multi-targeting: Support multitargeting within the IDE by enabling Visual Studio to leverage MSBuild using the tasks and targets that were shipped in Visual Studio 2005. Additionally, command line solutions will build using the toolset appropriate for the .NET Framework version that is being targeted.

    I just wanted to provide more clarity around what this means, because it does not entirely present an accurate picture. To make things worse, the topic of multi-targeting itself is a bit cloudy and confusing, so I want to shed some light on the big picture.

    Multi-Targeting : How does it work?

    So, the next question obviously is, what happens when you build a project using a specific ToolsVersion, and what are the mechanics in action behind the scenes?

    To understand this, we have to know a bit about how our project files are described.

    tags: , , , ,

  • Buck Hodges

    Where does the console output for tests go when run within Team Build?

    • 3 Comments

    This question came up on the internal discussion alias.  Tom Marsh, development lead on VSTS for Testers, responded saying, "the console output is saved with the unit test results. If you download the results and open them in the IDE, you will see the console output in the results view for the test that created it."

    tags: , , , , ,

  • Buck Hodges

    Opportunity to provide feedback on Community Technology Previews (CTPs)

    • 0 Comments

    Brian Harry would like your feedback on how and when we do CTPs.  Be sure to post comments on his blog!

    Feedback on VS Community Technology Previews (CTPs)

    We recently had an internal email thread with some people on our advisory councils giving us feedback on our CTP process.  There was some interesting and somewhat surprising feedback.  I wanted to solicit broader feedback from the community on what you want from the CTP process.  Here's some questions to get you thinking about it:

    • What do you like about VS CTPs?
    • What don't you like?
    • How often should we produce them?
    • How many have you installed?
    • How do you feel about the quality of the ones we (VS) have shipped in the past year or so?
    • What could we do to make them more useful for you?

    tags: , , , ,

Page 1 of 1 (7 items)