As you know, MSBuild in .NET 3.5 adds support for building projects concurrently. MSBuild.exe exposes this support with the new /m switch, and because Team Build uses MSBuild to build projects, it will get a speed up as well. In this release, Visual Studio doesn't use this to build managed projects concurrently, but you can write your own host that can build in parallel.

There are some limitations. In 3.5, you can't build in-memory projects concurrently, only project files. Also, I don't like the API much. Msbuild.exe was the only host we were focusing on in 3.5 and we didn't make it pretty. That will change.

 Anyway, here's an example. It assumes you have a sleep.exe in the same directory as the projects. So "sleep 3" will pause for 3 seconds. Our objective here is to run two sleeps concurrently :-)

using System;
using Microsoft.Build.BuildEngine;
using Microsoft.Build.Framework;
using System.Reflection;

namespace HostMsBuildExeTest
{
    class Program
    {
        [STAThread]
        static void Main(string[] args)
        {                     
            // We need to tell MSBuild where msbuild.exe is, so it can launch child nodes
            string parameters = @"MSBUILDLOCATION=" + System.Environment.GetFolderPath(System.Environment.SpecialFolder.System) + @"\..\Microsoft.NET\Framework\v3.5";
           
            // We need to tell MSBuild whether nodes should hang around for 60 seconds after the build is done in case they are needed again
            bool nodeReuse = true; // e.g.
            if (!nodeReuse)
            {
                parameters += ";NODEREUSE=false";
            }

            // We need to tell MSBuild the maximum number of nodes to use. It is usually fastest to pick about the same number as you have CPU cores
            int maxNodeCount = 3; // e.g.

            // Create the engine with this information
            Engine buildEngine = new Engine(null, ToolsetDefinitionLocations.Registry | ToolsetDefinitionLocations.ConfigurationFile, maxNodeCount, parameters);

            // Create a file logger with a matching forwarding logger, e.g.
            FileLogger fileLogger = new FileLogger();
            fileLogger.Verbosity = LoggerVerbosity.Detailed;
            Assembly engineAssembly = Assembly.GetAssembly(typeof(Engine));
            string loggerAssemblyName = engineAssembly.GetName().FullName;
            LoggerDescription fileLoggerForwardingLoggerDescription = new LoggerDescription("Microsoft.Build.BuildEngine.ConfigurableForwardingLogger", loggerAssemblyName, null, String.Empty, LoggerVerbosity.Detailed);

            // Create a regular console logger too, e.g.
            ConsoleLogger logger = new ConsoleLogger();
            logger.Verbosity = LoggerVerbosity.Normal;

            // Register all of these loggers
            buildEngine.RegisterDistributedLogger(fileLogger, fileLoggerForwardingLoggerDescription);
            buildEngine.RegisterLogger(logger);

            // Do a build
            buildEngine.BuildProjectFile("root.proj");

            // Finish cleanly
            buildEngine.Shutdown();
        }
    }
}

Now for testing purposes, I created three projects like this:

root.proj
Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003" ToolsVersion="3.5">
  <Target Name="t">
    <Message Importance="high" Text="## in root building children ##"/>
    <MSBuild Projects="1.csproj;2.csproj" BuildInParallel="true"/>
    <Message Importance="high" Text="## in root done building ##"/>
  </Target>
</Project>

1.csproj

<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003" ToolsVersion="3.5">
  <Target Name="t">
     <Message Importance="high" Text="## starting 1 ##"/>
    <Exec Command="sleep 3"/>
     <Message Importance="high" Text="## finishing 1 ##"/>
  </Target>
</Project>

2.csproj

<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003" ToolsVersion="3.5">
  <Target Name="t">
    <Message Importance="high" Text="## starting 2 ##"/>
    <Exec Command="sleep 3"/>
     <Message Importance="high" Text="## finishing 2 ##"/>
  </Target>
</Project> 

Notice that the root project uses the <MSBuild> task with the parameter BuildInParallel="true". This is where the parallelism starts: you still only invoke the build on a single root project. Without BuildInParallel="true", the <MSBuild> task will build serially. In a tree of projects, you would want every node to build its children with BuildInParallel="true".

Here's the output I get:

c:\test>test.exe
Build started 10/21/2007 5:02:55 PM.
     1>Project "c:\test\projects\root.proj" on node 0 (defaul
       t targets).
         ## in root building children ##
     1>Project "c:\test\projects\root.proj" (1) is building "
       c:\test\projects\1.csproj" (2) on node 1 (default tar
       gets).
         ## starting 1 ##
     1>Project "c:\test\projects\root.proj" (1) is building "
       c:\test\projects\2.csproj" (3) on node 2 (default tar
       gets).
         ## starting 2 ##
     1>t:
         ## in root done building ##
     1>Done Building Project "c:\test\projects\root.proj" (de
       fault targets).
     3>t:
         ## finishing 2 ##
     3>Done Building Project "c:\test\projects\2.proj" (de
       fault targets).
     2>t:
         ## finishing 1 ##
     2>Done Building Project "c:\test\projects\1.proj" (de
       fault targets).

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:03.21

As you can see, the two 3-second sleeps ran concurrently, so the build took 3.21 seconds overall.

Just to prove it I changed the "3" to "1" and got this:

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:06.11 

There's lots more to say about multiproc support, especially to explain the types of loggers I attached here, and that will be the subject of future posts. Even if you don't host MSBuild, you may want to create "traversal" projects (something like solution files, but nestable) that build their children in parallel. I'll explain that too in due course. 

Just to be clear though, for most of us, it isn't necessary to know any of this. We can just point msbuild.exe at our solution file and throw the "/m" switch and our build will be faster.

Dan