Aaron Hallberg : Team Build Orcas

Orcas Object Model Documentation

aaronhallberg — Fri, 04 May 2007 16:36:00 GMT

I put together a chm file for the Orcas Microsoft.TeamFoundation.Build.Client and Microsoft.TeamFoundation.Build.Common namespaces the other day using Sandcastle and the invaluable Sandcastle Help File Builder, so here it is. Please keep in mind that this is preliminary documentation, everything is subject to change before RTM, etc.

We'd love to hear what you think of the Orcas Team Build features, including the Object Model, so please download Beta 1, check out these docs, and post any comments/question/concerns you might have.

Note: attachment removed. Please grab the Beta 2 documentation instead.

-Aaron

TfsBuild.rsp and Logging Verbosity in Orcas

aaronhallberg — Thu, 03 May 2007 19:55:00 GMT

In the 2.0 .NET Framework, MSBuild's default verbosity for its FileLogger was "normal". As such, if you needed diagnostic logging information in a Team Build build (e.g. for debugging an issue with the build process) you needed to add the following to your TfsBuild.rsp file:

/verbosity:diagnostic

-or more briefly-

/v:diag

In the 3.5 .NET Framework, lots of things have changed with respect to the MSBuild file logger. First of all, the default verbosity for the FileLogger is now "diagnostic" rather than "normal". Secondly, the FileLogger now has its own command-line options - /fileLogger (/fl) turns on file logging, while /fileLoggerParameters (/flp) is used to pass parameters into the file logger. So - to set logging versobity back to normal in a Team Build build in Orcas (e.g. when you are not debugging an issue with the build process and want to keep your log files small) you will need to add the following to your TfsBuild.rsp file:

/fileLoggerParameters:verbosity=normal

-or more briefly-

/flp:v=n

Another thing you may notice if you look closely is that Team Build no longer passes too many options to MSBuild directly through the command-line. Instead, we dynamically generate a new copy of TfsBuild.rsp that contains the standard Team Build options followed by whatever you have in your checked in TfsBuild.rsp (if anything). As such, a typical TfsBuild.rsp file on a build machine might look something like this:

### Begin Team Build Generated Arguments ###

/m:4
/nologo
/noconsolelogger
/dl:BuildLogger,<BuildLogger Assembly>;<BuildLogger Parameters>
/fl /flp:logfile=BuildLog.txt;encoding=Unicode;
/p:BuildDefinition="BuildDefinition"
/p:DropLocation="\\MACHINENAME\DropLocation"
/p:BuildProjectFolderPath="$/TeamProject/TeamBuildTypes/BuildDefinition"
/p:BuildUri="vstfs:///Build/Build/1"
/p:TeamFoundationServerUrl="http://tfs:8080"
/p:TeamProject="TeamProject"
/p:SourcesSubdirectory="Sources"
/p:BinariesSubdirectory="Binaries"
/p:TestResultsSubdirectory="TestResults"
/p:SourceGetVersion="C3"
/p:LastGoodBuildLabel=""
/p:LastBuildNumber=""
/p:LastGoodBuildNumber=""
/p:NoCICheckInComment="***NO_CI***"
/p:IsDesktopBuild="false"
/p:TeamBuildRefPath="C:\Program Files\Microsoft Visual Studio 9.0\Common7\IDE\PrivateAssemblies"
/t:EndToEndIteration
TFSBuild.proj
### End Team Build Generated Arguments ###
### Begin Checked In TfsBuild.rsp Arguments ###
# This is a response file for MSBuild
# Add custom MSBuild command line options in this file
/flp:verbosity=normal
### End Checked In TfsBuild.rsp Arguments ###

Again, if you look carefully here you may notice the /m:4 option at the top of the sample rsp file. This has to do with MSBuild 3.5's new multi-process feature - more on this in a subsequent post!

Team Build Object Model - Queueing a Build

aaronhallberg — Tue, 24 Apr 2007 17:55:00 GMT

Two of the biggest Team Build changes in Orcas (Beta 1 is now available here) are (a) the addition of an Object Model, and (b) the addition of build queueing. In Team Build v1, by contrast, the only API available for Team Build was the web service; and only a single build (per Team Project per Build Machine) could be executed at a time - subsequent build start requests just failed.

The basic process for queueing a build is illustrated in the following sample:

using System;
using Microsoft.TeamFoundation.Client;
using Microsoft.TeamFoundation.Build.Client;

namespace OrcasBlogProjects
{
    class Program
    {
        static void Main(string[] args)
        {
            IQueuedBuild queuedBuild = QueueBuild("http://tfs:8080", "Team Project", "Nightly Build");

            Console.WriteLine(String.Format("Build successfully queued. Position: {0}.", queuedBuild.QueuePosition));
        }

        public static IQueuedBuild QueueBuild(String tfs, String teamProject, String buildDefinition)
        {
            // Get the specified team foundation server.
            TeamFoundationServer tfs = TeamFoundationServerFactory.GetServer(tfs);

            // Get the IBuildServer - the main point of entry to the Team Build OM.
            IBuildServer buildServer = (IBuildServer)tfs.GetService(typeof(IBuildServer));

            // Get the build definition for which a build is to be queued.
            IBuildDefinition definition = buildServer.GetBuildDefinition(teamProject, buildDefinition);

            // Create a build request for the build definition.
            IBuildRequest request = definition.CreateBuildRequest();

            // Queue a build.
            return buildServer.QueueBuild(request, QueueOptions.None);
        }
    }
}

Note the major steps in this process - (1) get the build definition, (2) create a request, and (3) queue the build. An overload of QueueBuild will be available in Beta 2 that takes a definition directly - when using all default options (see below for more on this), this overload can be used.

Step (1) is fairly straightforward, but even here there are various options available. The example illustrates obtaining a build definition from the Team Project name and the Definition name. Another overload of the GetBuildDefinition method takes a build definition URI:

public IBuildDefinition GetBuildDefinition(
    Uri buildDefinitionUri
)

In addition to the two Get methods, several Query methods are also available for build definitions. The most interesting for these purposes is the following:

public IBuildDefinitionQueryResult QueryBuildDefinitions(
    IBuildDefinitionSpec buildDefinitionSpec
)

The returned IBuildDefinitionQueryResult contains an array of matched definitions, and an array of Failures, if any occurred. The input IBuildDefinitionSpec allows wildcards in its Name property, which allows you to probe a particular team project for build definitions.

Step (2) offers many opportunities for customization. In particular, the IBuildRequest interface allows you to update many of the defaults for the requested build prior to queueing it. Settable properties include:

The build agent for which the requested build is queued. Build Agents are a new concept in Orcas, but can be thought of as v1 build machines.
The version for which sources are retrieved for the build.
The drop location to which binaries and log files are copied when the build is complete.
The priority of the requested build. Queue priorities are used in combination with queue times to determine which build should be started on an idle agent when multiple builds are in the queue.
The user for whom the build is being requested.
The maximum acceptable queue position - if this position cannot be satisfied, an exception will be thrown.
Last, but certainly not least, any command-line arguments which should be passed into MSBuild for the requested build.

Contrast this with the properties settable when starting a build in v1 - just the build machine and the build directory.

Beta 1 does not have a go-live license, but I hope that various folks out there will start downloading the VPCs and experimenting with the new Team Build Object Model - we'd love to get your feedback on it as soon as possible!

Adding Build Steps to Team Build in orcas

aaronhallberg — Wed, 28 Mar 2007 18:00:00 GMT

Way back in August I did a post (my 2nd ever!) on adding build steps to Team Build using a custom task. I thought I would revisit this topic in the context of the latest version of Team Build, now available in the March CTP of Orcas.

In Orcas, you'll have two options for adding build steps - the built-in BuildStep task, or your own custom task using the brand new Team Build Object Model (OM).

The BuildStep task can be used in two distinct ways: you can add build steps and you can update build steps:

  <Target Name="DoStuff">

    <BuildStep TeamFoundationServerUrl="$(TeamFoundationServerUrl)"
               BuildUri="$(BuildUri)"
               Message="Doing some stuff...">
      <Output TaskParameter="Id" PropertyName="StepId" />
    </BuildStep>

    <!-- Do some stuff here... -->

    <BuildStep TeamFoundationServerUrl="$(TeamFoundationServerUrl)"
               BuildUri="$(BuildUri)"
               Id="$(StepId)"
               Status="Succeeded" />

    <OnError ExecuteTargets="MarkBuildStepAsFailed" />
    
  </Target>

  <Target Name="MarkBuildStepAsFailed">
    
    <BuildStep TeamFoundationServerUrl="$(TeamFoundationServerUrl)"
               BuildUri="$(BuildUri)"
               Id="$(StepId)"
               Status="Failed" />
    
  </Target>

In this example, a build step is added, some stuff happens, and the build step is updated. Note the error handling which marks the step as Failed when an error occurs in the DoStuff target.

If you don't have any stuff to do, and just want to add an informational build step, you can ignore the ID output property and set the Status immediately when you add the build step.

    <BuildStep TeamFoundationServerUrl="$(TeamFoundationServerUrl)"
               BuildUri="$(BuildUri)"
               Message="This is an informational message."
               Status="Succeeded" />

And that's it!

There are still situations where you'll want to add build steps using your own custom tasks, of course. In this case, you can use the new OM, which should be a bit simpler than using the old web service methods.

using System;
using Microsoft.Build.Framework;
using Microsoft.TeamFoundation.Client;
using Microsoft.TeamFoundation.Build.Client;

namespace CustomTasks
{
    public abstract class TeamBuildTask : ITask
    {
        /// <summary>
        /// Put real task logic in this method.
        /// </summary>
        /// <returns>True if task is successful, otherwise false.</returns>
        protected abstract bool ExecuteInternal();

        /// <summary>
        /// Returns the name of the build step to be added for this task.
        /// </summary>
        /// <returns>Name of the build step to be added.</returns>
        protected abstract string GetBuildStepName();

        /// <summary>
        /// Returns the message of the build step to be added for this task - this is the
        /// string displayed in the Team Build GUI.
        /// </summary>
        /// <returns>Message of the build step to be added.</returns>
        protected abstract string GetBuildStepMessage();

        /// <summary>
        /// ITask implementation - BuildEngine property.
        /// </summary>
        public IBuildEngine BuildEngine
        {
            get
            {
                return m_buildEngine;
            }
            set
            {
                m_buildEngine = value;
            }
        }

        /// <summary>
        /// ITask implementation - HostObject property.
        /// </summary>
        public ITaskHost HostObject
        {
            get
            {
                return m_hostObject;
            }
            set
            {
                m_hostObject = value;
            }
        }

        /// <summary>
        /// The Url of the Team Foundation Server.
        /// </summary>
        [Required]
        public string TeamFoundationServerUrl
        {
            get
            {
                return m_tfsUrl;
            }
            set
            {
                m_tfsUrl = value;
            }
        }

        /// <summary>
        /// The Uri of the Build for which this task is executing.
        /// </summary>
        [Required]
        public string BuildUri
        {
            get
            {
                return m_buildUri;
            }
            set
            {
                m_buildUri = value;
            }
        }

        /// <summary>
        /// Lazy init property that gives access to the TF Server specified by TeamFoundationServerUrl.
        /// </summary>
        protected TeamFoundationServer Tfs
        {
            get
            {
                if (m_tfs == null)
                {
                    if (String.IsNullOrEmpty(TeamFoundationServerUrl))
                    {
                        // Throw some exception.
                    }
                    m_tfs = TeamFoundationServerFactory.GetServer(TeamFoundationServerUrl);
                }
                return m_tfs;
            }
        }

        /// <summary>
        /// Lazy init property that gives access to the BuildServer service of the TF Server.
        /// </summary>
        protected IBuildServer BuildServer
        {
            get
            {
                if (m_buildServer == null)
                {
                    m_buildServer = (IBuildServer)Tfs.GetService(typeof(IBuildServer));
                }
                return m_buildServer;
            }
        }

        /// <summary>
        /// Lazy init property that gives access to the Build specified by BuildUri.
        /// </summary>
        protected IBuildDetail Build
        {
            get
            {
                if (m_build == null)
                {
                    m_build = (IBuildDetail)BuildServer.GetBuild(new Uri(BuildUri), null, QueryOptions.None);
                }
                return m_build;
            }
        }

        /// <summary>
        /// ITask implementation - Execute method.
        /// </summary>
        /// <returns>
        /// True if the task succeeded, false otherwise.
        /// </returns>
        public bool Execute()
        {
            bool returnValue = false;

            try
            {
                AddBuildStep();
                returnValue = ExecuteInternal();
            }
            catch (Exception e)
            {
                AddExceptionBuildStep(e);
                throw;
            }
            finally
            {
                UpdateBuildStep(returnValue);
            }

            return returnValue;
        }

        private void AddBuildStep()
        {
            m_buildStep = InformationNodeConverters.AddBuildStep(Build, GetBuildStepName(), GetBuildStepMessage());
        }

        private void UpdateBuildStep(bool result)
        {
            m_buildStep.Status = result ? BuildStepStatus.Succeeded : BuildStepStatus.Failed;
            m_buildStep.FinishTime = DateTime.Now;
            m_buildStep.Save();
        }

        private void AddExceptionBuildStep(Exception e)
        {
            try
            {
                IBuildStep buildStep = InformationNodeConverters.AddBuildStep(Build, "Exception", e.Message, DateTime.Now, BuildStepStatus.Failed);
            }
            catch
            {
                // Eat any exceptions.
            }
        }

        private IBuildEngine m_buildEngine;
        private ITaskHost m_hostObject;
        private string m_tfsUrl;
        private string m_buildUri;
        private TeamFoundationServer m_tfs;
        private IBuildServer m_buildServer;
        private IBuildDetail m_build;
        private IBuildStep m_buildStep;
    }
}

There are a few things here which require some explanation.

IBuildServer.

IBuildServer is the top-level interface in the OM. It exposes methods for getting/querying builds, build definitions (formerly known as build types), build agents (formerly non-existent), etc.

IBuildDetail.

IBuildDetail represents an individual build in the OM. It exposes methods such as Stop and Delete; properties such as StartTime and RequestedBy; and events (which support polling for changes) such as StatusChanging and PollingCompleted. It also exposes a tree of arbitrary build information nodes which are used to store all standard Team Build data and can also be used to add any arbitrary custom data.

InformationNodeConverters and IBuildStep.

The InformationNodeConverters class can be used to get (and add) standard Team Build data types from the generic build information nodes associated with a build. Each well-known data type is exposed as an interface - build steps are exposed via IBuildStep.

I'll go into more details on the OM in later posts - hopefully this will be enough info to get those of you playing around with the March CTP started and to get those of you not playing around with the CTP interested in doing so!

Orcas Changes

aaronhallberg — Wed, 21 Mar 2007 16:46:00 GMT

The March CTP of Orcas is now available (Buck has a post on this here), and with it comes your first chance to play around with the new and improved Team Build. I'm pretty excited about the changes in the build process (defined in Microsoft.TeamFoundation.Build.targets), so I figured I'd mention some of them:

We've added hooks (targets which can be overridden) to allow executing custom logic before/after the compilation of individual configurations (BeforeCompileConfiguration and AfterCompileConfiguration, respectively) and before/after the compilation of individual solutions (BeforeCompileSolution and AfterCompileSolution, respectively). In V1 the only available hook was before/after everything was compiled (BeforeCompile and AfterComile, respectively) - these targets are still available as well.
We've added the ability to pass custom properties into your solutions when they are built. This can be done either for every solution, using the $(CustomPropertiesForBuild) property; or for individual solutions, using the Properties metadata in the SolutionToBuild item group.
We've added "convenience" properties for doing incremental gets and incremental builds, called (strangely enough) $(IncrementalGet) and $(IncrementalBuild). Setting them to true will make all the changes necessary to do an incremental get or build, respectively. In V1, it was not possible to make a distinction between these two choices, since the Clean target wiped out all sources, rather than just deleting binaries.
We've added the ability to retrieve the list of files gotten, replaced and deleted by the Get task. Just set the $(GetPopulateOutput) property to true, and three item groups named Gets, Replaces, and Deletes will be populated when the Get task executes.
All the properties of the build will now be available to the build script, thanks to the inclusion of a new task called GetBuildProperties, which is executed right at the beginning of the build process.
All the settable properties of the build can now be set at any point using another new task called SetBuildProperties.
A final new task, called BuildStep, can be used to insert and update build steps during the course of the build without having to write a custom task.
Output directories can be customized for individual solutions and even for individual projects within your solutions. I'll do another post at some point illustrating this in greater depth.
For you VC++ developers, we've provided "pass-through" properties for all the VCBuild task properties, including: $(VCBuildAdditionalLibPaths), $(VCBuildAdditionalOptions), $(VCBuildToolPath), and $(VCBuildUseEnvironment). Hopefully this will allow folks to accomplish what they need with VCBuild without having to specify everything as crazy escaped XML in the $(AdditionalVCOverrides) property!

There are lots of other changes as well (including Continuous Integration support, scheduled builds, build queueing, an Object Model, etc.), but these are the MSBuild-related highlights. We're really excited about this new version and look forward to getting your feedback on it!