MSBuild Team Blog

The MSBuild Team Blog Is Moving!

msbuild — Fri, 13 Nov 2009 17:34:00 GMT

We aren’t going away -- we’re just moving to a better home. To give you even more news, articles, and walkthroughs on Visual Studio, the MSBuild Team blog is moving to become part of The Visual Studio Blog. As part of The Visual Studio Blog, we’ll be joining the rest of the Visual Studio Platform Team in blogging not only about MSBuild, but other topics such as the core IDE and UI, the editor, the project system, VS extensibility, and more.

So please make sure to update your RSS readers to our new home.

See you all at the new blog location – http://blogs.msdn.com/visualstudio!

Inline Tasks CTP Walkthrough Update

msbuild — Sat, 01 Nov 2008 03:14:00 GMT

Well the CTP is now out, and I hope all are enjoying a preview of the new features being released in Visual Studio 2010.

Of particular interest is the walkthrough "How to Create an Inline Task".

In this walkthrough, "How to Create an Inline Task", we are highlighting our new feature "Inline Tasks" that allows you to write tasks in C# right inside your MSBuild (Project) file. This means you no longer have to build a separate assembly. It helps make custom build steps easy. Our sample walkthrough highlights how you might write a custom build rule to send an email when your build is complete.

However, with the quick pace to get the CTP bits out the door, we realized that some critical files for the Inline Tasks walkthrough did not make it onto the VPC.

In order to provide you with this really cool walkthrough, we have updated the documentation and have provided the missing bits in a zip file.

Please check out the post in our forums for more information.

http://social.msdn.microsoft.com/Forums/en-US/vs2010ctpdeploy/thread/f30edddb-4183-499b-942b-105d58a78a86/

Thanks,

Chuck England
Visual Studio Platform
Program Manager - Project and Build

MSBuild Extensions Pack releases to web

msbuild — Tue, 14 Oct 2008 17:22:00 GMT

I am very happy to pass on news from Mike Fourie that has released v1 of a remarkably extensive MSBuild Extensions Pack on codeplex. In his words:

The MSBuild Extension Pack is the successor to the FreeToDev MSBuild Tasks Suite and provides a collection of over 170 MSBuild tasks designed for the .net 3.5 Framework. A high level summary of what the tasks currently cover includes the following:

• System Items: Certificates, COM+, Console, Date and Time, Drives, Environment Variables, Event Logs, Files and Folders, GAC, Network, Performance Counters, Registry, Services, Sound
• Code: Assemblies, CAB Files, Code Signing, File Detokenisation, GUID’s, Mathematics, Strings, Threads, Zip
• Applications: BizTalk 2006, Email, IIS7, MSBuild, SourceSafe, StyleCop, Team Foundation Server, Visual Basic 6, WMI

Each task is documented and provided with an example in the help file. Where applicable, tasks are remote enabled, simply specify a MachineName and the task will target the remote machine.

Additional tasks and improvements to the documentation will be released frequently. I would encourage users to make use of the Issues and Discussions features on CodePlex and to contribute code to help drive this project forward.

Mike's goal is to make this the standard MSBuild task pack. With the energy he is putting into it, he has a good chance of that. If you're interested in contributing drop him a line at mailto:feedback@msbuildextensionpack.com

-- Dan

Microsoft Source Analysis releases to web

msbuild — Fri, 23 May 2008 19:36:00 GMT

Six months ago, the developers on MSBuild began to use an internal tool then called "StyleCop" in order to clean up our source code going forward. Think of it as a little like "FXCop" for your C# style. It flags as build warnings all kinds of deviations from a set of fairly strict style rules. We excluded all our existing source code files from inspection, since they generated thousands of warnings, and used it for all new source files, and for heavily rewritten old code, and required every check-in to be clean.

For a few days, it was frustrating, mainly because you have to accept its rules over your own ingrained conventions. I don't like some of them. Frankly however many of these conventions are important only for consistency, and not intrinsically better or worse than any others.

After that first week, we were surprised to find that once we got used to the rules, it was little or no effort to avoid the warnings, and the readability and consistency of our source code has improved considerably. I thought I would have to push the team to use it, but they picked it up enthusiastically, and one of the developers even enabled it on all our unit test code as well. There's no more "interesting" use of tabs, or personalized curly placement, and code reviews can focus entirely on the content of the change. As others joined our team, they picked it up easily too.

Well, StyleCop has just been released as "Microsoft Source Analysis". It's a free download from here. I highly recommend it.

Dan

What's up with xxx.sln.cache?

msbuild — Mon, 11 Feb 2008 20:54:00 GMT

We have an infrastructure in developer division for performance measurement. Before a feature crew integrates into their product unit branch, or a product unit does one of their periodic integrations of changes into a main branch, they must pass RPS ("regression prevention system"). This runs of the order of 100 performance scenarios in a clean, isolated environment to get very precise numbers and a whole bunch of performance counters. If certain counters regress - elapsed time, disk I/O, or memory allocations I think - the run is rejected and the performance regression must be fixed before integration can occur. Towards Orcas RTM MSBuild was still missing a performance goal for certain command line solution build scenarios and we had to find a quick way to speed them up.

When you launch MSBuild.exe on a solution file -- either directly, or using Team Build - some code has to run to create an in-memory equivalent MSBuild format project file that MSBuild can actually build. Ideally this ought to be trivial. We would essentially construct a project file that simply ran the MSBuild task on all projects listed in the solution. In reality not all projects in the solution are in MSBuild format, most especially VC projects. It turns out that we have to load and evaluate every project in the solution to get some information we need to order the solution file correctly with respect to those VC projects, and in 3.5 we also have to figure out possible parallelism between projects in the solution. After this is done, we build the result, which of course involves loading and evaluating those projects again to actually build them.

You may have set MSBuildEmitSolution=1 in the past to see the resulting file. Editing this isn't supported, because the format will change from version to version, and it needs to be reconstructed if the solution is changed. However, it can be useful.

The solution cache file is a simple serialization of the in-memory solution file to disk. Much like MSBuildEmitSolution, but we read it too. When asked to build a solution file, MSBuild will use the solution cache if it's present and is valid for the configuration, tools version, and MSBuild version requested, and is up to date with respect to the solution file itself. This saves the time spent constructing the in memory solution project. It can go right ahead and build it.

This isn't very pretty, because it's exposing ugly implementation details in a plain text file. Also, there's no intermediate directory for a solution file like there is for a project, so we have to write it in the same directory, which is impolite, and not desirable if you wish to keep your source tree clean (for example, you have redirected your intermediate and output directories elsewhere). There is one other issue. You can't safely build the solution concurrently with two different configurations, because the configuration isn't part of the filename, and the content of the file is configuration specific. Team Build needs to do this.

The workaround for both issues is to set an environment variable to disable creating this file. Set MSBuildUseNoSolutionCache=1.

When solutions and VC projects are MSBuild format this kind of problem will go away.

Dan

[edit: fixed typo spotted by Cullen Waters]

Article in .NET Developer's Journal

msbuild — Mon, 07 Jan 2008 23:58:00 GMT

There's a new article by Xin Yan and myself published in the .NET Developer's Journal.

We cover MSBuild from scratch, but also the new file format features in .NET 3.5 and our plans for the future. Check it out.

Dan

More tools...

msbuild — Mon, 03 Dec 2007 21:00:00 GMT

Two new MSBuild community tools to check out!

Eugene from Attrice has blogged about the Visualization support in the new edition of MSBuild SideKick. (There's also another pretty visualizer on CodePlex.)

Partho pointed me at a debugger he's created. It's on CodePlex too, which means you can help improve it. It runs as a logger, taking advantage of the rich event information that loggers receive, and the fact that loggers currently run synchronously with the build. (This will likely change in future versions of MSBuild.) Debuggers built on the logging infrastructure are about as good as you can get until we have debugger support built into MSBuild - but for many people that can be a lot better than no debugger.

Nice work.

Dan

Response to the feature poll

msbuild — Fri, 30 Nov 2007 20:11:00 GMT

There were over 80 responses to my recent post asking for feedback on where MSBuild should be heading (if the graph doesn't appear, it's here).

Thank you all for your thoughtful allocations! Let's go through each one in decreasing order of "investment".

1) Debugger. Debugging by gazing at diagnostic logs and putting in <Message> tasks is less than ideal, and I've probably done this more than anyone, but I would never have expected that a debugger would be the #1 request. A while back, one of us hacked up a proof of concept for debugging MSBuild in VS by using this technique, and it worked rather well. Apparently this is a perfectly supported route, and presumably much easier than writing a new debugging engine for VS. If and when we support this publicly, it would most likely be by this route. I hope we can.

2) Converting other project types. Deployment (.vdproj) was the highest mentioned by far, with some other mentions of Biztalk. Some good news here. WiX support integrated into Visual Studio has just appeared in the CTP of Visual Studio Rosario. It's based on the existing Votive plugin for Visual Studio, “productized” into the Visual Studio box. Rob Mensching has some detailed information. Of course, .vdproj isn't the same as WiX. I've passed on the feedback from this blog to the owners of this effort (who also own .vdproj) so they can see how many people would love to see .vdproj be MSBuild based. If you grab the CTP you can let them know exactly what you want via opening Connect issues -- Jeff Beehler's blog has information about how to do this. Open issues there so they get feedback, and vote them up the list :-)

3) Converting the .sln file. No surprise this is a big deal. Visual Studio itself is built by MSBuild, and for that purpose we didn’t use .sln files – we use files named "dirs.proj" to represent nodes in the build tree. Each dirs.proj does little more than list the projects and dirs.proj’s below it, and import simple build rules that use the <MSBuild> task to continue the tree traversal. Building Visual Studio itself from a huge solution file would be impossible. So we’re well aware how much pain they cause outside Microsoft. The hard part here from the MSBuild point of view is getting VS to read and write these dirs.proj files instead of .sln files.

4) Visualization. This was the other big surprise for me as I figured that you would say "complete more of the basics first". Also, of the items on this list, this would be one of the easiest for folks outside of Microsoft to accomplish. A couple months ago a couple of us spent a week and prototyped a WPF visualizer. Chris wrote a SQL logger that listened to just the regular, standard logger events and wrote to a database using the free SQL Server Compact Edition. Cliff wrote a visualizer based on the free GLEE graphing package which queried the database. It had two views -- bubble and line of target dependencies, and bubble and line for project dependencies. We built parts of VS using multiprocessor support (“/m” switch) with this logger attached, then pointed the visualizer at the database and we could see, for example, projects causing bottlenecks in the build. Because the visualizer was based on a database, there was some UI to do simple queries, like color all projects with names matching pattern X. Of course, this could go a lot further. What sort of visualizations would you want to see?

5) Native code/VC support. More good news here! Both command line and IDE support is now planned for the next release of Visual Studio after Visual Studio 2008 ("VS10"). Some of our friends downstairs in VC have been working on it for a little while now. As more details emerge we'll pass them on. (Putting my “platform” rather than “Visual Studio” hat on, you may have noticed that earlier this year, a competing IDE product began to offer native code build based on the MSBuild platform.)

6) Distributed build. Inside Microsoft, both the Visual Studio build team (the team that builds VS itself) and the Windows build team have been asking for this, amongst others. Six months ago one of us did a proof of concept of MSBuild based distributed build running on some large subtrees of the VS code and got near linear scalability in some cases - better than linear up to 4 machines in one particularly IO-bound subtree. We have a good idea how we would do this, but it isn't yet scheduled to happen. This seems to be a feature aimed more at the "build lab" than the developer. On the developer's machine, we typically aren't making full use of the cores she already has, and the kinds of builds she does tend to be smaller and perhaps less parallelizeable at the project level and the transmission and provisioning overhead would be significant.

7) Inline tasks. This is one of my personal favorites. Rather than say "go write and compile and deploy a task" I'd like to say "write some Powershell/batch/jscript and paste it in here and use it like a task elsewhere". Again, we've done a proof of concept. One particular kind of inline task we use internally we call "DataDrivenToolTask". You probably know that tasks that wrap a command line tool generally derive from ToolTask. These all have to do the same tiresome job: map friendly properties to command line switches. When we created tasks to wrap the native compiler and linker for building Visual Studio internally, this code became unwieldy. Yet it each mapping is a simple variation of the other. So we threw out the old tasks and created XML files that stated the tool name, defined mappings between property names and switches, described the type of the switch parameter, indicated how the switch is negated, what it could legally be combined with, and so on; then we wrote a generator tool that created the task from these. (Actually it creates a partial class, deriving from ToolTask, which we can combine with a little special code for that tool if we need to.) This turned out to work very well. What I would like to see is the ability to put this XML directly under the UsingTask element, so the task is created dynamically. That's probably not something anyone would do for something as huge as CL, but for tools with just a handful of parameters - think regsvr32 - one could make a nice typesafe readable task and pass it around to your friends by simply pasting it into an email. It's my hope that this would be one of our first types of inline tasks.

8) Multiproc performance. We're embarking on a significant rewrite of the "back end" of the MSBuild engine in VS10 and expect that we'll be able to improve the rather "basic" multiproc scheduling that MSBuild does in v3.5/Orcas: both to improve performance on existing machines and work better on the many core machines that will become increasingly common over the next few years. Our expectation is that this refactoring will make extending to distributed build more straightforward in a future release.

Skipping forward a few (the missing #10 probably should have been "add more built-in tasks") I'd like to mention ..

11) Extensible dependency checking. This was a bit confusingly described. There are really two parts to this: support transitive dependencies, and provide an extensibility point for arbitrary dependency checking plugins (such as an assembly interface comparer). The transitive dependency support is really essential for reasonable support of native code. C/C++, IDL, and .RC files amongst others all pull in files indirectly via #include and it's not feasible or reasonable to expect a developer to describe in their build process the full transitive closure of files that are read by their build in all circumstances: yet to do a trustworthy incremental build MSBuild must check the timestamps of all these files. This problem actually arises in many situations, with arbitrary file types. We have been solving this problem internally for the build of Visual Studio itself using a technology we created called Tracker, and it's proven itself to be solid and reliable. The other part is the extensibility point, which is perhaps what most people were actually voting on, and seems less urgent.

12) Assembly versioning support was suggested. Many of us have used Neil’s AssemblyInfoTask. Again, we hear you and there's good news. TeamBuild is planning to have a similar, but official and supported solution to this shipping in Rosario.

14) Object model improvements weren't on my list but probably should be: they were suggested by several people. Today the MSBuild object model is somewhat clunky and certainly incomplete. It was created to do just exactly what Visual Studio needed for VB and C# projects and no more. It mixes up the evaluated and unevaluated states amongst other problems. We expect to considerably improve this OM - not least because VC in VS10 will need it improved – to make it truly generic and powerful. (I already have a small group of people interested in giving detailed feedback on the design, but if you'd like to join them, please send me an email at msbuild@microsoft.com).

My caveat again -- nothing is set in stone until it appears in the final box, (or else is officially announced!) - resources and priorities change, risks to the product are managed, customer feedback sometimes shifts our direction directly or indirectly. In particular, work we need to do to support larger Visual Studio features has to happen. I hope though that you'll get value from my blogging candidly about our ideas and likely plans, while you realize we can’t make firm commitments to particular features at this stage. Enough caveats :-)

Any thoughts?

Dan

"This posting provided AS-IS, with no warranties"

MSBuild 3.5 "Orcas" has now shipped

msbuild — Thu, 22 Nov 2007 03:30:00 GMT

MSBuild 3.5 "Orcas" has now shipped. You can get the free download from here. It's included in the free Express Editions and of course Visual Studio 2008 itself.

I'll blog about what's new in MSBuild 3.5 in due course, but the main features are:

-- Multiprocessor support -- currently command line only, just build your solutions with /m switch. Includes a new improved console logger optimized for multiproc builds

-- Multitargeting support -- use MSBuild 3.5 to build projects targeting .NET 2.0 if you wish; mix targets within a tree or solution

-- Performance improvements. Your should see improvements in full builds, but most especially in incremental builds. We have seen some larger incremental build scenarios double in speed. On Vista, we see even bigger improvements, due to SuperFetch. This is all without even enabling multiproc!

A couple of more minor features

-- ItemDefinitionGroups -- "types" for items.

-- PropertyGroup/ItemGroup inside targets just like outside -- no more unreadable CreateItem/CreateProperty

-- Ability to remove items from lists and modify item metadata during the build -- no more "sloshing" into another list

And of course hundreds of bug fixes -- including especially the "generate resource locking" problem.

Also new since 2.0 was released -- MSBuild now builds all of Visual Studio itself. This involved converting thousands of projects, mostly native code: helping us learn about supporting huge trees, and trialling native-code build support for future release.

I'll have to blog about all of these individually, but for now -- I'm proud of what my team did -- go try it out.

Dan

PS Installation is side-by-side with .NET 2.0, so you should be able to install 3.5 without breaking any 2.0 apps or affecting VS2005 at all. However there is one caveat you should be aware of -- .NET 3.5 installs .NET 2.0 SP1, and this adds some new types to 2.0 assemblies. The means you should be aware that it is possible your app built on a machine with 2.0 SP1 installed and using the new types may not work on a machine without SP1. I've blogged about how to deal with this.

[edit: fixed font]

How would you spend $100 on MSBuild?

msbuild — Sat, 17 Nov 2007 21:43:00 GMT

We're currently planning for our next version (aka, "Dev10" - no code name this time) and subsequent releases. In that spirit, I'd like to do a quick poll of MSBuild aficionados to help us keep our "vision" for MSBuild aligned with yours, our customers.

First, a caveat - while ideally I would like to rank the results and work down the list, the reality is that we have other constraints: although we are a separate build platform, our biggest customer by far is Visual Studio and its customers. There's a lot more Visual Studio users than direct MSBuild users, so when Visual Studio needs to add a feature that requires work on MSBuild, we have to help. Then there's other constraints, like team resources. There's no guarantee which of these will appear in what version, and some of them may not be worth any investment. Some of them I just made up.

OK. Here’s some brainstormed features. Some of them are properly Visual Studio features, but they overlap closely with build:

1) Higher performance multiprocessor support. For example, we suspect there is plenty of room to improve the scheduling we do, and find speedups elsewhere in the code. As more and more of us have multicore machines, this might be a good place to invest.

2) VC support. This means converting VC projects (.vcproj) to MSBuild format, a customizable and extensible build process entirely defined by MSBuild .targets files rather than in makefiles or built into the VC project system, reuseable tasks for native code tools just like we have today for managed code, changing the VS project system for VC projects to sit on top of MSBuild format projects and build process, and replacing use of vcbuild.exe with msbuild.exe.

3) Support for other Microsoft project types that aren't yet in MSBuild format: Deployment/MSI (.vdproj), SQL Reporting (.rptproj), Biztalk (.btproj), Speech server (.prproj) etc -- whether currently supported by a VS project system or not. This is essentially a process of internal evangelization and encouragement for us.

4) Conversion of Visual Studio solution files (.sln files) and their (rudimentary) build process to MSBuild format and the VS support for reading and writing these, opening the way to create a targets file useful for traversing a tree of projects, and to let VS cleanly support n-level project hierarchies.

5) Extensible up-to-date checking: a way to plug in up-to-date checker extensions that you could use on selected Targets as an alternative to the simple timestamp checking you are currently restricted to. Perhaps including a ready-made extension that would by some means automatically support transitive dependencies - header files are an example of these; or to compare public interfaces for significant changes.

6) Distributed Build. (Like multiprocessor build we are now shipping, but building spread over a set of machines that you have pre-provisioned; possibly opening the way for future Team Build support for it too.)

7) Extensible reuseable inline tasks. This means the ability to create extensions that consume the language or description format of your choice and to create a task that you could use without explicit compilation or deployment. For example, a Powershell extension that you could use to create tasks implemented in Powershell. Think of putting script underneath a <UsingTask> element. These inline tasks would be easy to create and share online. Might include ready-made extensions for Powershell, and/or a data-description format for tool switches, so you could quickly create a task wrapping a command line tool without any compilation.

8) Typing and scoping for items and properties. For example, declaring an item to contain a path; letting a property fall out of scope at the end of a target.

9) Extensible functions. Today we have only 'exists(..)' and it's stepchild 'hastrailingslash(...)'. This would be an extension allowing new functions to be created for use in conditional expressions, and possibly elsewhere, and some functions like perhaps combinepath($(root),$(file)). Perhaps more built-in metadata to go along with %(Filename) and such.

11) An MSBuild debugger with full Visual Studio support for stepping, inspecting locals, and breakpoints. Today you have to use <Message> tasks and gaze at diagnostic logging output.

12) Visualization for project and target dependencies, mining of what files are consumed by what projects, impact analysis for changes to particular files, possibly leading to support for build refactoring at the project/tree level. This is essentially richer logging and datamining and a high level on a huge source tree.

So now here's the poll. If you had only $100 to spend, how would you distribute it amongst them?

Dan

(Edit: gave Distributed build a unique number :-) )

What are Targets, Tasks, and Tools?

msbuild — Wed, 14 Nov 2007 23:35:00 GMT

I've heard these confused in the context of MSBuild, so let's talk a little about what they are:

* A TARGET is a grouping of tasks (often 1) designed to do a particular job. For example, a Link target would be designed to produce a final binary from object files. Targets can declare other targets that they depend on, and MSBuild will make sure those are run first. At present targets are the unit of incremental build: declaring Inputs and Outputs on a target will cause MSBuild to compare their timestamps and skip the target if the inputs are up to date. Because a target is the unit of build, it is the entity that other parts of the build depend on or invoke (via CallTarget or the MSBuild task from another project). The implementation of a target can be modified later, so long as it's "contract" ("consume X and Y properties and list of object files O to produce binary Z") is preserved. A future version will likely have an extensibility model for up-to-date checking, and that will likely plug in at the target level. Once a target has run once in the context of a project, global properties, toolsversion, and build, it will be immediately skipped next time it is invoked, without even checking timestamps. When you write a target, always check that it skips properly in the incremental build case. MSBuild logs information about targets during the build, and any extra logging you want has to happen from a task, especially the <Message>, <Warning>, and <Error> tasks. Targets can be batched, although this is rarely done.

A TASK is a reuseable implementation of some build action. In this case the only task in the Link target would likely be the task named Link. It is fed parameters, including the files to link, does the link, and the build proceeds. At present tasks must be .NET assemblies deriving from the interface Microsoft.Build.Framework.ITask. Tasks can consume and emit items and properties. By design, they have no other insight into the state of the build. They should not discover files on disk - this makes them opaque and less reuseable. PropertyGroup and ItemGroup inside targets can be thought of as tasks, but they have special syntax and powers. If your target is starting to get too "clever", write a task. We hear loud and clear that not everybody wants to write and compile code to create a task, and we'll address this future. Tasks can be batched, and this is commonly done either to force a task to run once for each source file (something like Sources="'%(Compile.Identity)'") or to filter out certain input files, by filtering in the condition on metadata on source items.

A TOOL is an implementation detail of some tasks. A Link task may happen to currently wrap a tool named link.exe and it may happen that the parameters on this Link task pretty much wrap 1:1 with the command line switches on this tool named link.exe. However those are implementation details. One day the link task could be modified to load a "link.dll". Or the owners of link.exe could flip the /clr switch in order to implement ITask themselves. Tasks that wrap tools will want to derive from the ready-made abstract class named Microsoft.Build.Utilities.ToolTask, which provides much of the functionality they will need. Many tasks do not wrap tools. [ Aside: For example, during development of MSBuild v1, we initially implemented resource generation by wrapping resgen.exe. However a key scenario was to build a typical VB or C# project without even having the .NET SDK installed, but resgen.exe did not ship in the .NET Framework and we couldn't persuade them to move it. So we wrote our own resource generation task in code, called GenerateResource. (Resgen.exe is just a thin wrapper over BCL APIs anyway.) A similar situation exists for al.exe, but we did not write a task to do that without the tool: al.exe is some pretty hairy code. This is why you need VS or the .NET SDK installed to build projects with satellite assemblies. But I digress.]

Unlike makefiles, the inputs and outputs on a Target are not necessarily what the “build action”, ie the task, operates on. The task operates only on the parameters it is given. Typically however the Inputs on the Target would be among the inputs passed to the task, but this is not necessarily true. For example the Link target’s Inputs might include $(MSBuildProjectFullPath), so that the target is forced to run if the project file itself has been edited - yet the Link task would not be passed the project file.

Dan

Enabling multiprocessor support in an MSBuild host

msbuild — Mon, 22 Oct 2007 02:29:00 GMT

As you know, MSBuild in .NET 3.5 adds support for building projects concurrently. MSBuild.exe exposes this support with the new /m switch, and because Team Build uses MSBuild to build projects, it will get a speed up as well. In this release, Visual Studio doesn't use this to build managed projects concurrently, but you can write your own host that can build in parallel.

There are some limitations. In 3.5, you can't build in-memory projects concurrently, only project files. Also, I don't like the API much. Msbuild.exe was the only host we were focusing on in 3.5 and we didn't make it pretty. That will change.

Anyway, here's an example. It assumes you have a sleep.exe in the same directory as the projects. So "sleep 3" will pause for 3 seconds. Our objective here is to run two sleeps concurrently :-)

using System;
using Microsoft.Build.BuildEngine;
using Microsoft.Build.Framework;
using System.Reflection;

namespace HostMsBuildExeTest
{
    class Program
    {
        [STAThread]
        static void Main(string[] args)
        {
            // We need to tell MSBuild where msbuild.exe is, so it can launch child nodes
            string parameters = @"MSBUILDLOCATION=" + System.Environment.GetFolderPath(System.Environment.SpecialFolder.System) + @"\..\Microsoft.NET\Framework\v3.5";

            // We need to tell MSBuild whether nodes should hang around for 60 seconds after the build is done in case they are needed again
            bool nodeReuse = true; // e.g.
            if (!nodeReuse)
            {
                parameters += ";NODEREUSE=false";
            }

// We need to tell MSBuild the maximum number of nodes to use. It is usually fastest to pick about the same number as you have CPU cores
int maxNodeCount = 3; // e.g.

// Create the engine with this information
Engine buildEngine = new Engine(null, ToolsetDefinitionLocations.Registry | ToolsetDefinitionLocations.ConfigurationFile, maxNodeCount, parameters);

            // Create a file logger with a matching forwarding logger, e.g.
            FileLogger fileLogger = new FileLogger();
            fileLogger.Verbosity = LoggerVerbosity.Detailed;
            Assembly engineAssembly = Assembly.GetAssembly(typeof(Engine));
            string loggerAssemblyName = engineAssembly.GetName().FullName;
            LoggerDescription fileLoggerForwardingLoggerDescription = new LoggerDescription("Microsoft.Build.BuildEngine.ConfigurableForwardingLogger", loggerAssemblyName, null, String.Empty, LoggerVerbosity.Detailed);

            // Create a regular console logger too, e.g.
            ConsoleLogger logger = new ConsoleLogger();
            logger.Verbosity = LoggerVerbosity.Normal;

            // Register all of these loggers
            buildEngine.RegisterDistributedLogger(fileLogger, fileLoggerForwardingLoggerDescription);
            buildEngine.RegisterLogger(logger);

// Do a build
buildEngine.BuildProjectFile("root.proj");

            // Finish cleanly
            buildEngine.Shutdown();
        }
    }
}

Now for testing purposes, I created three projects like this:

root.proj
Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003" ToolsVersion="3.5">
<Target Name="t">
    <Message Importance="high" Text="## in root building children ##"/>
    <MSBuild Projects="1.csproj;2.csproj" BuildInParallel="true"/>
    <Message Importance="high" Text="## in root done building ##"/>
</Target>
</Project>

1.csproj

2.csproj

Notice that the root project uses the <MSBuild> task with the parameter BuildInParallel="true". This is where the parallelism starts: you still only invoke the build on a single root project. Without BuildInParallel="true", the <MSBuild> task will build serially. In a tree of projects, you would want every node to build its children with BuildInParallel="true".

Here's the output I get:

c:\test>test.exe
Build started 10/21/2007 5:02:55 PM.
     1>Project "c:\test\projects\root.proj" on node 0 (defaul
       t targets).
         ## in root building children ##
     1>Project "c:\test\projects\root.proj" (1) is building "
       c:\test\projects\1.csproj" (2) on node 1 (default tar
       gets).
         ## starting 1 ##
     1>Project "c:\test\projects\root.proj" (1) is building "
       c:\test\projects\2.csproj" (3) on node 2 (default tar
       gets).
         ## starting 2 ##
     1>t:
         ## in root done building ##
     1>Done Building Project "c:\test\projects\root.proj" (de
       fault targets).
     3>t:
         ## finishing 2 ##
     3>Done Building Project "c:\test\projects\2.proj" (de
       fault targets).
     2>t:
         ## finishing 1 ##
     2>Done Building Project "c:\test\projects\1.proj" (de
       fault targets).

Build succeeded.
0 Warning(s)
0 Error(s)

Time Elapsed 00:00:03.21

As you can see, the two 3-second sleeps ran concurrently, so the build took 3.21 seconds overall.

Just to prove it I changed the "3" to "1" and got this:

Build succeeded.
0 Warning(s)
0 Error(s)

Time Elapsed 00:00:06.11

There's lots more to say about multiproc support, especially to explain the types of loggers I attached here, and that will be the subject of future posts. Even if you don't host MSBuild, you may want to create "traversal" projects (something like solution files, but nestable) that build their children in parallel. I'll explain that too in due course.

Just to be clear though, for most of us, it isn't necessary to know any of this. We can just point msbuild.exe at our solution file and throw the "/m" switch and our build will be faster.

Dan

Manifest resource names changed for .resources files

msbuild — Fri, 19 Oct 2007 21:34:00 GMT

Juergen Bayer notified us of an issue introduced in MSBuild in .NET 3.5 "Orcas". The problem is if you have any items of type EmbeddedResource in your project file that are actually .resources format, rather than the usual .resx. In other words, for some reason you have already converted them from human-readable form. We believe the impact should not be widespread since usually projects use .resx files. Plus, "Orcas" is essentially locked down at this point, so it's too late to fix it for this release.

When we embed a resource in an assembly, we pick a name for the stream. This name is fairly unimportant, as long as it's distinct and consistent. Its only purpose is to specify the stream your resource manager should read. In VS2003, VS2005, through .NET 2.0 SP1, the stream name for .resources itself ended with the .resources extension. In VS2008/.NET 3.5, we inadvertently broke this: it no longer has .resources on the end. The effect will be at runtime your resources will not load properly. The workaround (apart from changing to use .resx instead which we will create .resources from for you) is to add a Logical Name metadata to the affected resources to specify the correct name, something like this:

    <EmbeddedResource Include="Resources\Images.resources" >
       <LogicalName>$(RootNamespace).Resources.Images.resources</LogicalName>
    </EmbeddedResource>

Of course, the names we pick for streams originating in .resx files did not get changed.

Dan

Using MSBuild as a generic scripting language

msbuild — Fri, 19 Oct 2007 20:04:00 GMT

I just got a pretty interesting mail from Dave Hickey at Premera, describing how his team is using MSBuild as a generic scripting language:

We thought you might be interested in how we use MSBuild here. Sure, we use it for compiling our .NET projects, but we’ve adopted it as our new Batch scripting language, replacing a windows-based KSH (korn shell).

We have hundreds of activities that run at scheduled times, fired by Autosys. We’ve created a tool that spawns MSBuild and passes xml project files. We’ve extended the logging so that a state file is also written after each task. When the script is rerun, the state file is checked and the last resumable target is then used. This saves us from having to repeat all steps in a script due to a db deadlock or some other problem.

The main benefits we see using MSBuild for batch are:

·         Logging is standardized and built in.

·         Projects are resume-able, thanks to a CallTargetResume class we wrote that acts on the statefile.

·         Syntax is standardized because its xml. Tools like MSBuild sidekick become available. Very little programming required to maintain, once the toolset is established.

·         It leverages all the include, batching and import features you guys wrote.

·         Large development community.

·         MSDN MSBuild docs for free.

·         MS Supported

We considered using Nant, but opted for MSBuild as a standard for the reasons above. Anyway, hope you found this ‘atypical’ use of MSBuild interesting.

Now, MSBuild is intended to be only a language and platform for build processes. Powershell is much more suited to generic scripting, and has some great UI support now too, but to the best of my knowledge it doesn't have any built-in logging capability yet. That's one of the things I miss most myself from cmd.exe batch script.

Dan

Multitargeting against .NET 2.0

msbuild — Tue, 09 Oct 2007 23:57:00 GMT

If you're using the new VS 2008 multitargeting features to target .NET 2.0 you should be aware that in VS 2008 they have a limitation related to service packs.

In .NET 2.0 SP1, the CLR team has added a few types to existing .NET 2.0 assemblies. For example, DateTimeOffset has been added to mscorlib.dll.

Since VS 2008 installs .NET 2.0 SP1 (sometimes known as "redbits"), with VS 2008 you will be able to successfully build and test an app that uses DateTimeOffset, even if you explicitly tell VS 2008 that you wish to target .NET 2.0. But then if you move this app to a machine that only has the original .NET 2.0 on it, it won't run. This is really a limitation in the Visual Studio multitargeting implementation, at least assuming that the CLR team made the right choice in adding this to an existing assembly.

This limitation may be addressed in the next version of Visual Studio. Meanwhile, if you want your app to work on machines without .NET 2.0 SP1, make sure you verify that it compiles on such a machine. You don't need to install Visual Studio to compile it, of course -- MSBuild 2.0 can do it. If your project was created by Visual Studio 2008, it will use the property $(MSBuildToolsPath) in it, which is new in MSBuild 3.5. Also, Visual Studio 2008 updated the solution version number to something MSBuild 2.0 won’t recognize (that's another story). To work around these issues, specify the project directly, and set a property, like this

%windir%\microsoft.net\framework\v2.0.50727\msbuild.exe /p:msbuildtoolspath=%windir%\microsoft.net\framework\v2.0.50727 myProjectCreatedByVS2008.csproj

If this builds successfully without .NET 2.0 SP1 installed, you’ll know you haven’t used one of these types.

By the way, .NET 2.0 SP1 will be available as a standalone install, or it will automatically be installed when you install .NET 3.5 or Visual Studio 2008, or Vista SP1, or "Longhorn" server.

[Author: Dan]

Update: Scott has generated a libcheck diff showing all the new types in SP1. Take a look here: http://www.hanselman.com/blog/content/binary/org2.0to2.0/APIChangesorg2.0to2.0.html

Also, Krzysztof has created an FXCop rule to check for these changes: http://blogs.msdn.com/kcwalina/archive/2007/10/02/Multi_2D00_TargetingAndFxCop.aspx