Aaron Hallberg : MSBuild

TfsBuild.rsp and Logging Verbosity in Orcas

aaronhallberg — Thu, 03 May 2007 19:55:00 GMT

In the 2.0 .NET Framework, MSBuild's default verbosity for its FileLogger was "normal". As such, if you needed diagnostic logging information in a Team Build build (e.g. for debugging an issue with the build process) you needed to add the following to your TfsBuild.rsp file:

/verbosity:diagnostic

-or more briefly-

/v:diag

In the 3.5 .NET Framework, lots of things have changed with respect to the MSBuild file logger. First of all, the default verbosity for the FileLogger is now "diagnostic" rather than "normal". Secondly, the FileLogger now has its own command-line options - /fileLogger (/fl) turns on file logging, while /fileLoggerParameters (/flp) is used to pass parameters into the file logger. So - to set logging versobity back to normal in a Team Build build in Orcas (e.g. when you are not debugging an issue with the build process and want to keep your log files small) you will need to add the following to your TfsBuild.rsp file:

/fileLoggerParameters:verbosity=normal

-or more briefly-

/flp:v=n

Another thing you may notice if you look closely is that Team Build no longer passes too many options to MSBuild directly through the command-line. Instead, we dynamically generate a new copy of TfsBuild.rsp that contains the standard Team Build options followed by whatever you have in your checked in TfsBuild.rsp (if anything). As such, a typical TfsBuild.rsp file on a build machine might look something like this:

### Begin Team Build Generated Arguments ###

/m:4
/nologo
/noconsolelogger
/dl:BuildLogger,<BuildLogger Assembly>;<BuildLogger Parameters>
/fl /flp:logfile=BuildLog.txt;encoding=Unicode;
/p:BuildDefinition="BuildDefinition"
/p:DropLocation="\\MACHINENAME\DropLocation"
/p:BuildProjectFolderPath="$/TeamProject/TeamBuildTypes/BuildDefinition"
/p:BuildUri="vstfs:///Build/Build/1"
/p:TeamFoundationServerUrl="http://tfs:8080"
/p:TeamProject="TeamProject"
/p:SourcesSubdirectory="Sources"
/p:BinariesSubdirectory="Binaries"
/p:TestResultsSubdirectory="TestResults"
/p:SourceGetVersion="C3"
/p:LastGoodBuildLabel=""
/p:LastBuildNumber=""
/p:LastGoodBuildNumber=""
/p:NoCICheckInComment="***NO_CI***"
/p:IsDesktopBuild="false"
/p:TeamBuildRefPath="C:\Program Files\Microsoft Visual Studio 9.0\Common7\IDE\PrivateAssemblies"
/t:EndToEndIteration
TFSBuild.proj
### End Team Build Generated Arguments ###
### Begin Checked In TfsBuild.rsp Arguments ###
# This is a response file for MSBuild
# Add custom MSBuild command line options in this file
/flp:verbosity=normal
### End Checked In TfsBuild.rsp Arguments ###

Again, if you look carefully here you may notice the /m:4 option at the top of the sample rsp file. This has to do with MSBuild 3.5's new multi-process feature - more on this in a subsequent post!

Adding Build Steps to Team Build in orcas

aaronhallberg — Wed, 28 Mar 2007 18:00:00 GMT

Way back in August I did a post (my 2nd ever!) on adding build steps to Team Build using a custom task. I thought I would revisit this topic in the context of the latest version of Team Build, now available in the March CTP of Orcas.

In Orcas, you'll have two options for adding build steps - the built-in BuildStep task, or your own custom task using the brand new Team Build Object Model (OM).

The BuildStep task can be used in two distinct ways: you can add build steps and you can update build steps:

  <Target Name="DoStuff">

    <BuildStep TeamFoundationServerUrl="$(TeamFoundationServerUrl)"
               BuildUri="$(BuildUri)"
               Message="Doing some stuff...">
      <Output TaskParameter="Id" PropertyName="StepId" />
    </BuildStep>

    <!-- Do some stuff here... -->

    <BuildStep TeamFoundationServerUrl="$(TeamFoundationServerUrl)"
               BuildUri="$(BuildUri)"
               Id="$(StepId)"
               Status="Succeeded" />

    <OnError ExecuteTargets="MarkBuildStepAsFailed" />
    
  </Target>

  <Target Name="MarkBuildStepAsFailed">
    
    <BuildStep TeamFoundationServerUrl="$(TeamFoundationServerUrl)"
               BuildUri="$(BuildUri)"
               Id="$(StepId)"
               Status="Failed" />
    
  </Target>

In this example, a build step is added, some stuff happens, and the build step is updated. Note the error handling which marks the step as Failed when an error occurs in the DoStuff target.

If you don't have any stuff to do, and just want to add an informational build step, you can ignore the ID output property and set the Status immediately when you add the build step.

    <BuildStep TeamFoundationServerUrl="$(TeamFoundationServerUrl)"
               BuildUri="$(BuildUri)"
               Message="This is an informational message."
               Status="Succeeded" />

And that's it!

There are still situations where you'll want to add build steps using your own custom tasks, of course. In this case, you can use the new OM, which should be a bit simpler than using the old web service methods.

using System;
using Microsoft.Build.Framework;
using Microsoft.TeamFoundation.Client;
using Microsoft.TeamFoundation.Build.Client;

namespace CustomTasks
{
    public abstract class TeamBuildTask : ITask
    {
        /// <summary>
        /// Put real task logic in this method.
        /// </summary>
        /// <returns>True if task is successful, otherwise false.</returns>
        protected abstract bool ExecuteInternal();

        /// <summary>
        /// Returns the name of the build step to be added for this task.
        /// </summary>
        /// <returns>Name of the build step to be added.</returns>
        protected abstract string GetBuildStepName();

        /// <summary>
        /// Returns the message of the build step to be added for this task - this is the
        /// string displayed in the Team Build GUI.
        /// </summary>
        /// <returns>Message of the build step to be added.</returns>
        protected abstract string GetBuildStepMessage();

        /// <summary>
        /// ITask implementation - BuildEngine property.
        /// </summary>
        public IBuildEngine BuildEngine
        {
            get
            {
                return m_buildEngine;
            }
            set
            {
                m_buildEngine = value;
            }
        }

        /// <summary>
        /// ITask implementation - HostObject property.
        /// </summary>
        public ITaskHost HostObject
        {
            get
            {
                return m_hostObject;
            }
            set
            {
                m_hostObject = value;
            }
        }

        /// <summary>
        /// The Url of the Team Foundation Server.
        /// </summary>
        [Required]
        public string TeamFoundationServerUrl
        {
            get
            {
                return m_tfsUrl;
            }
            set
            {
                m_tfsUrl = value;
            }
        }

        /// <summary>
        /// The Uri of the Build for which this task is executing.
        /// </summary>
        [Required]
        public string BuildUri
        {
            get
            {
                return m_buildUri;
            }
            set
            {
                m_buildUri = value;
            }
        }

        /// <summary>
        /// Lazy init property that gives access to the TF Server specified by TeamFoundationServerUrl.
        /// </summary>
        protected TeamFoundationServer Tfs
        {
            get
            {
                if (m_tfs == null)
                {
                    if (String.IsNullOrEmpty(TeamFoundationServerUrl))
                    {
                        // Throw some exception.
                    }
                    m_tfs = TeamFoundationServerFactory.GetServer(TeamFoundationServerUrl);
                }
                return m_tfs;
            }
        }

        /// <summary>
        /// Lazy init property that gives access to the BuildServer service of the TF Server.
        /// </summary>
        protected IBuildServer BuildServer
        {
            get
            {
                if (m_buildServer == null)
                {
                    m_buildServer = (IBuildServer)Tfs.GetService(typeof(IBuildServer));
                }
                return m_buildServer;
            }
        }

        /// <summary>
        /// Lazy init property that gives access to the Build specified by BuildUri.
        /// </summary>
        protected IBuildDetail Build
        {
            get
            {
                if (m_build == null)
                {
                    m_build = (IBuildDetail)BuildServer.GetBuild(new Uri(BuildUri), null, QueryOptions.None);
                }
                return m_build;
            }
        }

        /// <summary>
        /// ITask implementation - Execute method.
        /// </summary>
        /// <returns>
        /// True if the task succeeded, false otherwise.
        /// </returns>
        public bool Execute()
        {
            bool returnValue = false;

            try
            {
                AddBuildStep();
                returnValue = ExecuteInternal();
            }
            catch (Exception e)
            {
                AddExceptionBuildStep(e);
                throw;
            }
            finally
            {
                UpdateBuildStep(returnValue);
            }

            return returnValue;
        }

        private void AddBuildStep()
        {
            m_buildStep = InformationNodeConverters.AddBuildStep(Build, GetBuildStepName(), GetBuildStepMessage());
        }

        private void UpdateBuildStep(bool result)
        {
            m_buildStep.Status = result ? BuildStepStatus.Succeeded : BuildStepStatus.Failed;
            m_buildStep.FinishTime = DateTime.Now;
            m_buildStep.Save();
        }

        private void AddExceptionBuildStep(Exception e)
        {
            try
            {
                IBuildStep buildStep = InformationNodeConverters.AddBuildStep(Build, "Exception", e.Message, DateTime.Now, BuildStepStatus.Failed);
            }
            catch
            {
                // Eat any exceptions.
            }
        }

        private IBuildEngine m_buildEngine;
        private ITaskHost m_hostObject;
        private string m_tfsUrl;
        private string m_buildUri;
        private TeamFoundationServer m_tfs;
        private IBuildServer m_buildServer;
        private IBuildDetail m_build;
        private IBuildStep m_buildStep;
    }
}

There are a few things here which require some explanation.

IBuildServer.

IBuildServer is the top-level interface in the OM. It exposes methods for getting/querying builds, build definitions (formerly known as build types), build agents (formerly non-existent), etc.

IBuildDetail.

IBuildDetail represents an individual build in the OM. It exposes methods such as Stop and Delete; properties such as StartTime and RequestedBy; and events (which support polling for changes) such as StatusChanging and PollingCompleted. It also exposes a tree of arbitrary build information nodes which are used to store all standard Team Build data and can also be used to add any arbitrary custom data.

InformationNodeConverters and IBuildStep.

The InformationNodeConverters class can be used to get (and add) standard Team Build data types from the generic build information nodes associated with a build. Each well-known data type is exposed as an interface - build steps are exposed via IBuildStep.

I'll go into more details on the OM in later posts - hopefully this will be enough info to get those of you playing around with the March CTP started and to get those of you not playing around with the CTP interested in doing so!

Passing custom properties to individual solutions in Team Build

aaronhallberg — Mon, 19 Mar 2007 22:54:00 GMT

Gautam Goenka posted an article on this topic way back on April 20, 2006. It included a targets file which overrode the standard Team Build CoreCompile target and allowed user-specified properties to be passed into the MSBuild tast that Team Build uses to build the solution in the SolutionToBuild item group. This approach is fine if you want to pass the same custom property values into each solution in the SolutionToBuild item group, but what if you want to pass different property values into each solution?

Manish Agarwal posted an article that could help get you started here. His goal was to enable redirecting assemblies to solution-specific subdirectories, but it was easily extendable to passing other user-specified properties on a solution-specific basis. Unfortunately, it also some problems, including breaking the calculation of errors/warnings during the compilation phase of the build.

Before pressing on, I should say that we do not recommend overriding the Core* targets in a Team Build build. The primary reason here is that you will almost certainly be broken after you upgrade to Orcas if you override these targets, since most of them will be changing radically in that new version. See this post by Buck Hodges for more details here. The good news here is that the issues that caused people to override the Core* targets in Team Build v1 have been addressed in Orcas, so you should no longer find it necessary to do this sort of thing.

Having said all that, attached you will find a new CoreCompile override that will allow you pass custom property values into each solution via solution-specific metadata. For example, if you wanted each solution to be signed with a different key file you could do something like:

    <SolutionToBuild Include="$(SolutionRoot)\foo.sln">
      <Properties>SignAssembly=true;AssemblyOriginatorKeyFile=C:\foo.snk</Properties>
    </SolutionToBuild>
    <SolutionToBuild Include="$(SolutionRoot)\bar.sln">
      <Properties>SignAssembly=true;AssemblyOriginatorKeyFile=C:\bar.snk</Properties>
    </SolutionToBuild>

NOTE: My original example here, which purported to put individual binaries into individual subdirectories, was broken! Thanks to Esteban Garcia for pointing this out, and sorry for any trouble I might have caused anybody... If you want to put your binaries into individual subdirectories, try out the new attachment and do something like:

    <SolutionToBuild Include="$(SolutionRoot)\foo.sln">
      <Subdirectory>foo</Subdirectory>
    </SolutionToBuild>
    <SolutionToBuild Include="$(SolutionRoot)\bar.sln">
      <Subdirectory>bar</Subdirectory>
    </SolutionToBuild>

To use this modified CoreCompile target, just:

Download the attached CoreCompileOverride.targets file and check it in alongside TfsBuild.proj. (Alternatively, you can install this file somewhere on your build machine(s) and modify the import directory in step 2)

Add an <import> statement to your TfsBuild.proj file - something like:

<Import Project="$(MSBuildProjectDirectory)\CoreCompileOverride.targets" />

Add your custom properties to each item in the SolutionToBuild item group.

Hopefully you'll find this useful! Let me know via comments if you run into any issues with the attached file, etc.

The Triumphant Return?

aaronhallberg — Thu, 14 Dec 2006 18:26:00 GMT

Wow - it's been a long time. Sorry for the insanely long delay between posts, loyal reader(s). A lot has happened since I last managed to post something on my blog:

I've been working like crazy, along with the rest of the Team Build team, to get our Continuous Integration solution finished up for the Orcas release (see Brian Harry's blog post here for the full TFS roadmap, or Buck Hodge's post here for more details on the Team Build features in Orcas).
I got hooked on that Ugly Betty show. At first the hourlong format put me off, but now I can't imagine it being any shorter.
Control of the House and Senate shifted to the Democrats.
Britney and Kevin broke up, just days after my wife and I dressed as the two of them for Halloween. Coincidence? Perhaps...
Vista, Office, and the Zune all shipped. One very positive aspect of the Zune launch from my point of view - my favorite radio station, KEXP, was heavily involved and various excellent bands (e.g. Band of Horses) have been featured in ad spots, etc.

In any case, during a conference call the other day, a Team Build user expressed a desire to easily insert build steps into a build from within a csproj file... In previous posts I have laid out custom tasks which, as part of their execution, insert build steps. In this post, I lay out a simpler custom task which inserts arbitrary text as a build step - think of it as a <Message> task in Team Build form.

Here's the code:

using System;
using Microsoft.Build.Framework;
using Microsoft.Build.Utilities;
using Microsoft.TeamFoundation.Client;
using Microsoft.TeamFoundation.Build.Proxy;

namespace CustomTasks
{
    public class BuildStepTask : Task
    {
        /// <summary>
        /// Execute the task logic.
        /// </summary>
        /// <returns>True. Exceptions thrown on failure.</returns>
        public override bool Execute()
        {
            BuildStore.AddBuildStep(BuildUri, Text, Text);
            BuildStore.UpdateBuildStep(BuildUri, Text, DateTime.Now, m_buildStatus);

            return true;
        }

        /// <summary>
        /// The text of the build step.
        /// </summary>
        [Required]
        public String Text
        {
            get
            {
                return m_text;
            }
            set
            {
                m_text = value;
            }
        }

        /// <summary>
        /// The status of the build step. If not specified, BuildStepStatus.Succeeded will be assumed.
        /// </summary>
        public String Status
        {
            get
            {
                return m_buildStatus.ToString();
            }
            set
            {
                // Go ahead and throw an exception if the value is invalid.
                m_buildStatus = (BuildStepStatus)Enum.Parse(typeof(BuildStepStatus), value);
            }
        }

        /// <summary>
        /// The Url of the Team Foundation Server.
        /// </summary>
        [Required]
        public String TeamFoundationServerUrl
        {
            get
            {
                return m_tfsUrl;
            }
            set
            {
                m_tfsUrl = value;
            }
        }

        /// <summary>
        /// The Uri of the Build for which this task is executing.
        /// </summary>
        [Required]
        public String BuildUri
        {
            get
            {
                return m_buildUri;
            }
            set
            {
                m_buildUri = value;
            }
        }

        /// <summary>
        /// Lazy init property that gives access to the TF Server specified by TeamFoundationServerUrl.
        /// </summary>
        protected TeamFoundationServer Tfs
        {
            get
            {
                if (m_tfs == null)
                {
                    m_tfs = TeamFoundationServerFactory.GetServer(TeamFoundationServerUrl);
                }
                return m_tfs;
            }
        }

        /// <summary>
        /// Lazy init property that gives access to the BuildStore service of the TF Server.
        /// </summary>
        protected BuildStore BuildStore
        {
            get
            {
                if (m_buildStore == null)
                {
                    m_buildStore = (BuildStore)Tfs.GetService(typeof(BuildStore));
                }
                return m_buildStore;
            }
        }

        private String m_text;
        private BuildStepStatus m_buildStatus = BuildStepStatus.Succeeded;
        private String m_tfsUrl;
        private String m_buildUri;
        private TeamFoundationServer m_tfs;
        private BuildStore m_buildStore;
    }
}

And here's a snippet from the csproj file I used to test it out:

  <UsingTask TaskName="CustomTasks.BuildStepTask" AssemblyFile="..\TeamBuildTypes\HelloWorld\CustomTasks.dll" />

  <Target Name="AfterCompile" Condition=" '$(BuildUri)' != '' ">
    <BuildStepTask TeamFoundationServerUrl="$(TeamFoundationServerUrl)"
                   BuildUri="$(BuildUri)"
                   Text="Finished compiling $(AssemblyName)" />
  </Target>

Note the condition on the Target to avoid running it outside of a Team Build environment.

Hopefully this post will be the first of many in the months to come.

Determining Whether Tests Passed in Team Build

aaronhallberg — Thu, 21 Sep 2006 20:26:00 GMT

In a forum post a while back, I laid out a method for determining whether tests had passed during a build. More recently, I have linked to this forum post in advising others on similar problems. Unfortunately, as a sharp user pointed out in this same thread, my solution doesn't actually work, since it relies on a property that is not accessible in Team Build v1!

So - to remedy the situation I have written a custom task which can be used to determine whether tests have succeeded or not. This task takes advantage of the GetTestResultsForBuild method of the Microsoft.TeamFoundation.Build.Proxy.BuildStore class.

As always, this task is provided as a sample only; its awesomeness cannot be guaranteed, etc. The task inherits from the TeamBuildTask class I presented in an earlier post.

using System;
using System.Web.Services;
using Microsoft.Build.Framework;
using Microsoft.Build.Utilities;
using Microsoft.TeamFoundation.Client;
using Microsoft.TeamFoundation.Build.Common;
using Microsoft.TeamFoundation.Build.Proxy;

namespace MyNamespace
{
    public class CheckForTestSuccess : TeamBuildTask
    {
        /// <summary>
        /// The ConfigurationToBuild Item Group for the Build.
        /// </summary>
        [Required]
        public ITaskItem[] ConfigurationToBuild
        {
            get
            {
                return m_configurationToBuild;
            }
            set
            {
                m_configurationToBuild = value;
            }
        }

        /// <summary>
        /// An output property which will be true if all test runs succeeded and false otherwise.
        /// </summary>
        [Output]
        public bool TestSuccess
        {
            get
            {
                return m_testSuccess;
            }
            set
            {
                m_testSuccess = value;
            }
        }

        /// <summary>
        /// Returns the name of the build step to be added for this task.
        /// </summary>
        /// <returns>Name of the build step to be added.</returns>
        protected override string GetBuildStepName()
        {
            return "CheckForTestSuccess";
        }

        /// <summary>
        /// Returns the message of the build step to be added for this task - this is the
        /// string displayed in the Team Build GUI.
        /// </summary>
        /// <returns>Message of the build step to be added.</returns>
        protected override string GetBuildStepMessage()
        {
            return "Checking for test success.";
        }

        /// <summary>
        /// Put real task logic in this method.
        /// </summary>
        /// <returns>True if task is successful, otherwise false.</returns>
        protected override bool ExecuteInternal()
        {
            string platform;
            string flavor;
            TestResultData[] testResults;

            m_testSuccess = true;

            foreach (ITaskItem configuration in ConfigurationToBuild)
            {
                platform = configuration.GetMetadata("PlatformToBuild");
                flavor   = configuration.GetMetadata("FlavorToBuild");

                testResults = BuildStore.GetTestResultsForBuild(BuildUri, platform, flavor);

                foreach (TestResultData testResult in testResults)
                {
                    if (!testResult.RunPassed)
                    {
                        m_testSuccess = false;
                        break;
                    }
                }
                if (!m_testSuccess)
                {
                    break;
                }
            }

            return true;
        }

        private ITaskItem[] m_configurationToBuild;
        private bool m_testSuccess;
    }
}

To use this task, you'll need to do something like the following in TfsBuild.proj:

<UsingTask TaskName="CheckForTestSuccess" AssemblyFile="CustomTasks.dll" />

<Target Name="AfterTest">
	<CheckForTestSuccess TeamFoundationServerUrl="$(TeamFoundationServerUrl)" 
		BuildUri="$(BuildUri)" 
		ConfigurationToBuild="@(ConfigurationToBuild)">
		<Output TaskParameter="TestSuccess" PropertyName="TestSuccess" />
	</CheckForTestSuccess>
</Target>

The property TestSuccess would then be 'true' if and only if all test runs succeeded for the build.

Building a Specific Version with Team Build

aaronhallberg — Tue, 19 Sep 2006 17:40:00 GMT

Team Build by default gets the latest version of your sources (or tip) from source control. For the most part, this is probably the behavior you would want and expect. But not always - you might want to reproduce a particular build that got deleted, for example; or you might want to build your latest changest without including changes other users might have checked in.

Unfortunately, getting (and therefore building) a specific version from source control is not supported out of the box in Team Build version 1... The Get task used in Team Build (this can be found in Microsoft.TeamFoundation.Build.targets) looks like this:

<Target Name="CoreGet"
	Condition=" '$(IsDesktopBuild)' != 'true' "
	DependsOnTargets="$(CoreGetDependsOn)" >

	<!-- Get all the latest sources from the given workspace-->	
	<Get Condition=" '$(SkipGet)' != 'true' " 
		Workspace="$(WorkspaceName)" 
		Recursive="$(RecursiveGet)" 
		Force="$(ForceGet)" />
	  
</Target>

The Get task also a Version property that is not used here. It defaults to "T", or the tip version. To get and build a specific version from source control, you will need to set this property.

There are several options here:

You can override the CoreGet target in TfsBuild.proj.
You can modify the CoreGet target in Microsoft.TeamFoundation.Build.targets directly.
You can set the SkipGet property to true and override BeforeGet or AfterGet instead.

Modifying Microsoft.TeamFoundation.Build.targets is not a good idea - this would need to be done on every build machine, could get wiped out during an upgrade or installation repair, etc. Overriding CoreGet is usually not recommended either, since the CoreXX targets will almost certainly be changing in the next release of Team Build. In this case, however, overriding CoreGet (option 1) is essentially indistinguishable from setting SkipGet and overriding BeforeGet / AfterGet (option 3) and is simpler.

So - to override CoreGet in TfsBuild.proj, you will want to do something like the following:

<Target Name="CoreGet"
	  Condition=" '$(IsDesktopBuild)'!='true' "
	  DependsOnTargets="$(CoreGetDependsOn)" >

	<!-- Get all the latest sources from the given workspace-->
	<Get Condition=" '$(SkipGet)'!='true' "
		Workspace="$(WorkspaceName)"
		Recursive="$(RecursiveGet)"
		Force="$(ForceGet)"
		Version="$(VersionToGet)" />
	  
</Target>

At this point, you just need to set the VersionToGet property to whatever version you would like to get. This can be done in all the normal MSBuild ways - via the command line (put a string in your TfsBuild.rsp file similar to /p:VersionToGet=<VersionSpec>), declaratively in a PropertyGroup element, programmatically with a CreateProperty task, or via an environment variable.

To reproduce a particular build, you will typically want to set VersionToGet to a label-based version spec similar to "L<build number>" (unless you have modified your build process to use some other labeling scheme). For example, if you build number is "MyBuild_20060919.1" you could do something like this:

<PropertyGroup>
	<VersionToGet>LMyBuild_20060919.1</VersionToGet>
</PropertyGroup>

MSBuild Batching - Generate a Cross-Product (continued)

aaronhallberg — Thu, 07 Sep 2006 23:06:00 GMT

As promised, here is the code for the ItemGroupCrossProduct task presented in a previous post:

using System;
using System.Collections.Generic;
using Microsoft.Build.Framework;
using Microsoft.Build.Utilities;

namespace CustomTasks
{
    public class ItemGroupCrossProduct : Task
    {
        [Required]
        public ITaskItem[] ItemGroup1
        {
            get
            {
                return m_itemGroup1;
            }
            set
            {
                m_itemGroup1 = value;
            }
        }

        [Required]
        public string MetaData1
        {
            get
            {
                return m_metaData1;
            }
            set
            {
                m_metaData1 = value;
            }
        }

        [Required]
        public ITaskItem[] ItemGroup2
        {
            get
            {
                return m_itemGroup2;
            }
            set
            {
                m_itemGroup2 = value;
            }
        }

        [Required]
        public string MetaData2
        {
            get
            {
                return m_metaData2;
            }
            set
            {
                m_metaData2 = value;
            }
        }

        [Output]
        public ITaskItem[] CombinedItemGroup
        {
            get
            {
                return m_combinedItemGroup;
            }
        }

        /// <summary>
        /// Task implementation - Execute method.
        /// </summary>
        /// <returns>
        /// True if the task succeeded, false otherwise.
        /// </returns>
        public override bool Execute()
        {
            List<TaskItem> newItems = new List<TaskItem>();

            foreach (ITaskItem item1 in ItemGroup1)
            {
                foreach (ITaskItem item2 in ItemGroup2)
                {
                    TaskItem newItem = new TaskItem(item2.ItemSpec);
                    newItem.SetMetadata(MetaData1, item1.GetMetadata(MetaData1));
                    newItem.SetMetadata(MetaData2, item2.GetMetadata(MetaData2));
                    newItems.Add(newItem);
                }
            }

            m_combinedItemGroup = newItems.ToArray();

            return true;
        }

        private ITaskItem[] m_itemGroup1;
        private string m_metaData1;
        private ITaskItem[] m_itemGroup2;
        private string m_metaData2;
        private ITaskItem[] m_combinedItemGroup;
    }
}

The idea of the task is to take in two collections of Items and two corresponding metadata property names and produce a new Item collection which contains the cross product of the two inputs. This is a cross product in a rather limited sense - the resulting Items will contain only the two metadata properties specified in the input properties. Any other metadata properties in either of the input Item collections will be ignored. Additionally, the Identity values of the resulting Items are determined by the ItemGroup2 Items.

MSBuild Batching - Generating a Cross-Product

aaronhallberg — Tue, 05 Sep 2006 17:13:00 GMT

I've been playing around with the batching functionality in MSBuild quite a bit lately, and thought I'd share my various little discoveries with the world... If you aren't sure what batching is in MSBuild, check out this MSDN page. Alternatively, here's my five second explanation - batching allows you to simulate a foreach loop over the Items in an ItemGroup. For example:

<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="PrintFoo" xmlns="http://schemas.microsoft.com/developer/msbuild/2003" >

	<ItemGroup>
		<Foo Include="foo1">
			<FooMetadata>1</FooMetadata>
		</Foo>
		<Foo Include="foo2">
			<FooMetadata>2</FooMetadata>
		</Foo>
	</ItemGroup>

	<Target Name="PrintFoo">

		<Message Importance="high" Text="FooMetadata=%(Foo.FooMetadata)" />

	</Target>

</Project>

The Message task in the above MSBuild file is effectively saying something like:

foreach (Item foo in ItemGroup)
{
    <Message Importance="high" Text="FooMetadata=" + foo.FooMetadata />
}

To try it out, just cut and paste the code into a text file and run MSBuild.exe on it - you should get the following output:

Project "c:\foo.txt" (default targets):

Target PrintFoo:
FooMetadata=1
FooMetadata=2

This isn't quite right, since really the batching occurs over collections of items with identical values for the referenced metadata property (see the above reference MSDN page for details on this). But - to a first approximation this is what is going on.

Things get interesting when you want to do a loop within a loop... Consider the following:

<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="PrintFooAndBar" xmlns="http://schemas.microsoft.com/developer/msbuild/2003" >

	<ItemGroup>
		<Foo Include="foo1">
			<FooMetadata>1</FooMetadata>
		</Foo>
		<Foo Include="foo2">
			<FooMetadata>2</FooMetadata>
		</Foo>
	</ItemGroup>

	<ItemGroup>
		<Bar Include="bar1">
			<BarMetadata>a</BarMetadata>
		</Bar>
		<Bar Include="bar2">
			<BarMetadata>b</BarMetadata>
		</Bar>
	</ItemGroup>

	<Target Name="PrintFooAndBar">

		<Message Importance="high" Text="FooMetadata=%(Foo.FooMetadata), BarMetadata=%(Bar.BarMetadata)" />

	</Target>

</Project>

The first time I tried this, I fully expected to get output like:

Project "c:\foobar.txt" (default targets):

Target PrintFooAndBar:
    FooMetadata=1, BarMetadata=a
    FooMetadata=1, BarMetadata=b
    FooMetadata=2, BarMetadata=a
    FooMetadata=2, BarMetadata=b

Instead, though, I got the following:

Project "c:\foobar.txt" (default targets):

Target PrintFooAndBar:
    FooMetadata=, BarMetadata=a
    FooMetadata=, BarMetadata=b
    FooMetadata=1, BarMetadata=
    FooMetadata=2, BarMetadata=

When MSBuild encounters two collections to batch over, that is, it apparently batches one and then the other - not both at the same time. Back in our foreach syntax, it does something like:

foreach (Item foo in ItemGroup)
{
    <Message Importance="high" Text="FooMetadata=" + foo.FooMetadata + ", BarMetadata=" />
}

foreach (Item bar in ItemGroup)
{
    <Message Importance="high" Text="FooMetadata=, BarMetadata=" + bar.BarMetadata />
}

How, then, can we get MSBuild to do something more like:

foreach (Item foo in ItemGroup)
{
	foreach (Item bar in ItemGroup)
	{
		<Message Importance="high" Text="FooMetadata=" + foo.FooMetadata + ", BarMetadata=" + bar.BarMetadata />
	}
}

I have come up with two solutions - perhaps you have others? If so, please post them in the comments!

(1) Use an MSBuild task to call back into our project file.

In this solution, we use the $(MSBuildProjectFile) property (a reserved property in MSBuild) to call back into our project file.

<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="PrintFooAndBar" xmlns="http://schemas.microsoft.com/developer/msbuild/2003" >

	<ItemGroup>
		<Foo Include="foo1">
			<FooMetadata>1</FooMetadata>
		</Foo>
		<Foo Include="foo2">
			<FooMetadata>2</FooMetadata>
		</Foo>
	</ItemGroup>

	<ItemGroup>
		<Bar Include="bar1">
			<BarMetadata>a</BarMetadata>
		</Bar>
		<Bar Include="bar2">
			<BarMetadata>b</BarMetadata>
		</Bar>
	</ItemGroup>

	<Target Name="PrintFooAndBar">

		<MSBuild Projects="$(MSBuildProjectFile)" Targets="InternalPrintFooAndBar" Properties="Foo=%(Foo.FooMetadata)" />

	</Target>

	<Target Name="InternalPrintFooAndBar">

		<Message Importance="high" Text="FooMetadata=$(Foo), BarMetadata=%(Bar.BarMetadata)" />

	</Target>

</Project>

There are various downsides here. The Message task does get executed for each combination of FooMetadata and BarMetadata, but in two separate invocations of the MSBuild task. Our output looks like this:

Target PrintFooAndBar:
__________________________________________________
Project "c:\foobar.txt" is building "c:\foobar.txt" (InternalPrintFooAndBar target(s)):

    Target InternalPrintFooAndBar:
        FooMetadata=1, BarMetadata=a
        FooMetadata=1, BarMetadata=b
    __________________________________________________
    Project "c:\foobar.txt" is building "c:\foobar.txt" (InternalPrintFooAndBar target(s)):

    Target InternalPrintFooAndBar:
        FooMetadata=2, BarMetadata=a
        FooMetadata=2, BarMetadata=b

Additionally, the collection aspect of batching that I alluded to earlier is lost in this approach. That is, in the InternalPrintFooAndBar target, the Foo item group has been reduced to a single property value, with all of its ItemGroupness thrown away.

(2) Write a custom task to generate the cross product.

In this solution, a custom task is used to explicitly generate a new ItemGroup with the cross product of two input ItemGroups. (More specifically, the cross product of two sets of Metadata values - these could potentially come from the same ItemGroup)

<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="PrintFooAndBar" xmlns="http://schemas.microsoft.com/developer/msbuild/2003" >

	<UsingTask TaskName="CustomTasks.ItemGroupCrossProduct" AssemblyFile="E:\My Documents\Visual Studio 2005\Projects\BlogProjects\CustomTasks\bin\Debug\CustomTasks.dll" />

	<ItemGroup>
		<Foo Include="foo1">
			<FooMetadata>1</FooMetadata>
		</Foo>
		<Foo Include="foo2">
			<FooMetadata>2</FooMetadata>
		</Foo>
	</ItemGroup>

	<ItemGroup>
		<Bar Include="bar1">
			<BarMetadata>a</BarMetadata>
		</Bar>
		<Bar Include="bar2">
			<BarMetadata>b</BarMetadata>
		</Bar>
	</ItemGroup>

	<Target Name="PrintFooAndBar">

		<ItemGroupCrossProduct ItemGroup1="@(Foo)" Metadata1="FooMetadata" ItemGroup2="@(Bar)" Metadata2="BarMetadata">

			<Output TaskParameter="CombinedItemGroup" ItemName="CombinedItems" />

		</ItemGroupCrossProduct>

		<Message Importance="high" Text="FooMetadata=%(CombinedItems.FooMetadata), BarMetadata=%(CombinedItems.BarMetadata)" />

	</Target>

</Project>

This post is getting too long, and I have other work to do, so - the code and an explanation of the Task wil have to wait until another day... (I have to do something to keep you coming back)

Adding Custom Loggers to Team Build

aaronhallberg — Wed, 30 Aug 2006 16:02:00 GMT

Most of the magic in a Team Build is done using either (a) customized tasks, or (b) a customized logger. Well, most of the build machine magic - there's plenty of magic in the client, the Application Tier, and the Data Tier. All right - even on the build machine I suppose plenty of magic happens within MSBuild itself. In any case, we should be able to agree that some of the magic in a Team Build happens using customized tasks and loggers, right? They're responsible for making most of the web service calls that populate the database with information about the build, interacting with source control and work item tracking, keeping track of the errors/warnings encountered during the build, etc.

Yesterday we looked at using tasks to add customized build steps during a build. Today I'd like to show you how to add a customized logger... For more information on MSBuild loggers, check out MSBuild Logging on MSDN.

Step 1. Figure out why you want to add a logger.

I was thinking about this last night - would a Team Build user ever want to add a custom logger? The IDC team, implementers of Team Build 1.0, have a couple of posts on different types of custom loggers:

An XML Logger. This might be useful for diagnostic purposes - it will show you every single event raised by MSBuild during the course of a build. I wouldn't recommend it for large builds, unless you feel like filling up your hard disk!

An errors/warnings logger that creates log files per solution. This could be pretty useful if you are running into tons of errors/warnings in your build and want to separate them out into more managable chunks (by default Team Build creates log files per configuration).

Neither of these seemed seemed particularly compelling to me, though - I could see wanting to use them once or twice, but not all the time. How about this - out of the box TFS supports notifications on build completion or on build quality change. For certain types of builds, though, one might want to send out notifications on build start. Right? Right? Oh fine, it's not the most interesting thing in the world, but it's something...

If you come up with other interesting reasons to write your own loggers, please let me know!

Step 2. You'll want to actually write your custom logger.

I'm not going to post the entire sample this time - just the relevant bit.

public class CustomLogger : ILogger
{

...

    public void Initialize(IEventSource source)
    {
        source.BuildStarted += new BuildStartedEventHandler(HandleBuildStartedEvent);
    }

    public void HandleBuildStartedEvent(Object sender, BuildStartedEventArgs args)
    {
        // Write fancy email sending code here!
    }

    ...
}

Step 3. You'll need to tell MSBuild about your logger, using the /logger command-line switch.

One of the files created for each build type is TfsBuild.rsp. This is an MSBuild response file, and can contain MSBuild.exe command-line switches. Team Build will pass this file to MSBuild.exe whenever a build is started (for the given build type).

So - check out TfsBuild.rsp from source control, add something like the following:

/logger:CustomLogger,CustomLogger.dll

...and check it back in.

Step 4. Make sure that your logger assembly is available during the build.

The easiest thing here is probably to add it to the same directory in source control as the TfsBuild.rsp file you just modified - Team Build will then sync CustomLogger.dll along with TfsBuild.proj, TfsBuild.rsp, and WorkspaceMapping.xml. In this case, you won't need to give a full path to the assembly in the rsp file, and the above example should just work.

Adding BuildSteps to Team Build through a Custom Task

aaronhallberg — Tue, 29 Aug 2006 22:14:00 GMT

Team Build displays Build Steps in the build report form within Visual Studio. By default, build steps are added at various points during the course of a build - while getting sources (in the Get task), compiling solutions / projects, copying files to the drop location, etc. Team Build allows users to insert their own build steps using the publicly accessible BuildStore web service - in particular, the AddBuildStep and UpdateBuildStep methods. The following sample (I make no claims as to the awesomeness or lack thereof of this sample, etc.) TeamBuildTask class illustrates how this can be done:

using System;
using System.Web.Services;
using Microsoft.Build.Framework;
using Microsoft.TeamFoundation.Client;
using Microsoft.TeamFoundation.Build.Common;
using Microsoft.TeamFoundation.Build.Proxy;

/// <summary>
/// Summary description for Class1
/// </summary>
namespace MyNamespace
{
    public abstract class TeamBuildTask : ITask
    {
        /// <summary>
        /// Put real task logic in this method.
        /// </summary>
        /// <returns>True if task is successful, otherwise false.</returns>
        protected abstract bool ExecuteInternal();

        /// <summary>
        /// Returns the name of the build step to be added for this task.
        /// </summary>
        /// <returns>Name of the build step to be added.</returns>
        protected abstract string GetBuildStepName();

        /// <summary>
        /// Returns the message of the build step to be added for this task - this is the
        /// string displayed in the Team Build GUI.
        /// </summary>
        /// <returns>Message of the build step to be added.</returns>
        protected abstract string GetBuildStepMessage();

        /// <summary>
        /// ITask implementation - BuildEngine property.
        /// </summary>
        public IBuildEngine BuildEngine
        {
            get
            {
                return m_buildEngine;
            }
            set
            {
                m_buildEngine = value;
            }
        }

        /// <summary>
        /// ITask implementation - HostObject property.
        /// </summary>
        public ITaskHost HostObject
        {
            get
            {
                return m_hostObject;
            }
            set
            {
                m_hostObject = value;
            }
        }

        /// <summary>
        /// The Url of the Team Foundation Server.
        /// </summary>
        [Required]
        public string TeamFoundationServerUrl
        {
            get
            {
                return m_tfsUrl;
            }
            set
            {
                m_tfsUrl = value;
            }
        }

        /// <summary>
        /// The Uri of the Build for which this task is executing.
        /// </summary>
        [Required]
        public string BuildUri
        {
            get
            {
                return m_buildUri;
            }
            set
            {
                m_buildUri = value;
            }
        }

        /// <summary>
        /// Lazy init property that gives access to the TF Server specified by TeamFoundationServerUrl.
        /// </summary>
        protected TeamFoundationServer Tfs
        {
            get
            {
                if (m_tfs == null)
                {
                    if (String.IsNullOrEmpty(TeamFoundationServerUrl))
                    {
                        // Throw some exception.
                    }
                    m_tfs = TeamFoundationServerFactory.GetServer(TeamFoundationServerUrl);
                }
                return m_tfs;
            }
        }

        /// <summary>
        /// Lazy init property that gives access to the BuildStore service of the TF Server.
        /// </summary>
        protected BuildStore BuildStore
        {
            get
            {
                if (m_buildStore == null)
                {
                    m_buildStore = (BuildStore)Tfs.GetService(typeof(BuildStore));
                }
                return m_buildStore;
            }
        }

        /// <summary>
        /// ITask implementation - Execute method.
        /// </summary>
        /// <returns>
        /// True if the task succeeded, false otherwise.
        /// </returns>
        public bool Execute()
        {
            bool returnValue = false;

            try
            {
                AddBuildStep();
                returnValue = ExecuteInternal();
            }
            catch (Exception e)
            {
                AddExceptionBuildStep(e);
                throw;
            }
            finally
            {
                UpdateBuildStep(returnValue);
            }

            return returnValue;
        }

        private void AddBuildStep()
        {
            BuildStore.AddBuildStep(BuildUri, GetBuildStepName(), GetBuildStepMessage());
        }

private void UpdateBuildStep(bool result)
{

            BuildStepStatus status = result ? BuildStepStatus.Succeeded : BuildStepStatus.Failed;
            BuildStore.UpdateBuildStep(BuildUri, GetBuildStepName(), DateTime.Now, status);
        }

        private void AddExceptionBuildStep(Exception e)
        {
            try
            {
                BuildStore.AddBuildStep(BuildUri, "Exception", e.Message);
                BuildStore.UpdateBuildStep(BuildUri, "Exception", DateTime.Now, BuildStepStatus.Failed);
            }
            catch
            {
                // Eat any exceptions.
            }
        }

        private IBuildEngine m_buildEngine;
        private ITaskHost m_hostObject;
        private string m_tfsUrl;
        private string m_buildUri;
        private TeamFoundationServer m_tfs;
        private BuildStore m_buildStore;
    }
}

To use this base class, just override the ExecuteInternal method - put the actual task logic here. Then override the GetBuildStepName and GetBuildStepMessage methods to specify the Name of the build step (which serves as its ID and should therefore be reasonably unique) and the Message of the build step (which will be the string displayed in the build report form).

The Execute method will add a build step (with the specified Name and Message) when task execution starts, and update the build step with the appropriate status and time when execution completes. If an exception is thrown by the ExecuteInternal method, an exception build step will be added (if possible) by the AddExceptionBuildStep method.

Note the TeamFoundationServerUrl and BuildUri properties. These properties will need to be set for derived tasks to function properly, and can be easily set to the TeamFoundationServerUrl and BuildURI properties available during a Team Build (e.g. within TfsBuild.proj).

<Target Name="BeforeBuild">
	<SomeCustomTask 
		TeamFoundationServerUrl="$(TeamFoundationServerUrl)"
		BuildUri="$(BuildURI)" />
</Target>

Happy building!