MSBuild Team Blog : msbuild

Article in .NET Developer's Journal

msbuild — Mon, 07 Jan 2008 23:58:00 GMT

There's a new article by Xin Yan and myself published in the .NET Developer's Journal.

We cover MSBuild from scratch, but also the new file format features in .NET 3.5 and our plans for the future. Check it out.

Dan

More tools...

msbuild — Mon, 03 Dec 2007 21:00:00 GMT

Two new MSBuild community tools to check out!

Eugene from Attrice has blogged about the Visualization support in the new edition of MSBuild SideKick. (There's also another pretty visualizer on CodePlex.)

Partho pointed me at a debugger he's created. It's on CodePlex too, which means you can help improve it. It runs as a logger, taking advantage of the rich event information that loggers receive, and the fact that loggers currently run synchronously with the build. (This will likely change in future versions of MSBuild.) Debuggers built on the logging infrastructure are about as good as you can get until we have debugger support built into MSBuild - but for many people that can be a lot better than no debugger.

Nice work.

Dan

MSBuild 3.5 "Orcas" has now shipped

msbuild — Thu, 22 Nov 2007 03:30:00 GMT

MSBuild 3.5 "Orcas" has now shipped. You can get the free download from here. It's included in the free Express Editions and of course Visual Studio 2008 itself.

I'll blog about what's new in MSBuild 3.5 in due course, but the main features are:

-- Multiprocessor support -- currently command line only, just build your solutions with /m switch. Includes a new improved console logger optimized for multiproc builds

-- Multitargeting support -- use MSBuild 3.5 to build projects targeting .NET 2.0 if you wish; mix targets within a tree or solution

-- Performance improvements. Your should see improvements in full builds, but most especially in incremental builds. We have seen some larger incremental build scenarios double in speed. On Vista, we see even bigger improvements, due to SuperFetch. This is all without even enabling multiproc!

A couple of more minor features

-- ItemDefinitionGroups -- "types" for items.

-- PropertyGroup/ItemGroup inside targets just like outside -- no more unreadable CreateItem/CreateProperty

-- Ability to remove items from lists and modify item metadata during the build -- no more "sloshing" into another list

And of course hundreds of bug fixes -- including especially the "generate resource locking" problem.

Also new since 2.0 was released -- MSBuild now builds all of Visual Studio itself. This involved converting thousands of projects, mostly native code: helping us learn about supporting huge trees, and trialling native-code build support for future release.

I'll have to blog about all of these individually, but for now -- I'm proud of what my team did -- go try it out.

Dan

PS Installation is side-by-side with .NET 2.0, so you should be able to install 3.5 without breaking any 2.0 apps or affecting VS2005 at all. However there is one caveat you should be aware of -- .NET 3.5 installs .NET 2.0 SP1, and this adds some new types to 2.0 assemblies. The means you should be aware that it is possible your app built on a machine with 2.0 SP1 installed and using the new types may not work on a machine without SP1. I've blogged about how to deal with this.

[edit: fixed font]

How would you spend $100 on MSBuild?

msbuild — Sat, 17 Nov 2007 21:43:00 GMT

We're currently planning for our next version (aka, "Dev10" - no code name this time) and subsequent releases. In that spirit, I'd like to do a quick poll of MSBuild aficionados to help us keep our "vision" for MSBuild aligned with yours, our customers.

First, a caveat - while ideally I would like to rank the results and work down the list, the reality is that we have other constraints: although we are a separate build platform, our biggest customer by far is Visual Studio and its customers. There's a lot more Visual Studio users than direct MSBuild users, so when Visual Studio needs to add a feature that requires work on MSBuild, we have to help. Then there's other constraints, like team resources. There's no guarantee which of these will appear in what version, and some of them may not be worth any investment. Some of them I just made up.

OK. Here’s some brainstormed features. Some of them are properly Visual Studio features, but they overlap closely with build:

1) Higher performance multiprocessor support. For example, we suspect there is plenty of room to improve the scheduling we do, and find speedups elsewhere in the code. As more and more of us have multicore machines, this might be a good place to invest.

2) VC support. This means converting VC projects (.vcproj) to MSBuild format, a customizable and extensible build process entirely defined by MSBuild .targets files rather than in makefiles or built into the VC project system, reuseable tasks for native code tools just like we have today for managed code, changing the VS project system for VC projects to sit on top of MSBuild format projects and build process, and replacing use of vcbuild.exe with msbuild.exe.

3) Support for other Microsoft project types that aren't yet in MSBuild format: Deployment/MSI (.vdproj), SQL Reporting (.rptproj), Biztalk (.btproj), Speech server (.prproj) etc -- whether currently supported by a VS project system or not. This is essentially a process of internal evangelization and encouragement for us.

4) Conversion of Visual Studio solution files (.sln files) and their (rudimentary) build process to MSBuild format and the VS support for reading and writing these, opening the way to create a targets file useful for traversing a tree of projects, and to let VS cleanly support n-level project hierarchies.

5) Extensible up-to-date checking: a way to plug in up-to-date checker extensions that you could use on selected Targets as an alternative to the simple timestamp checking you are currently restricted to. Perhaps including a ready-made extension that would by some means automatically support transitive dependencies - header files are an example of these; or to compare public interfaces for significant changes.

6) Distributed Build. (Like multiprocessor build we are now shipping, but building spread over a set of machines that you have pre-provisioned; possibly opening the way for future Team Build support for it too.)

7) Extensible reuseable inline tasks. This means the ability to create extensions that consume the language or description format of your choice and to create a task that you could use without explicit compilation or deployment. For example, a Powershell extension that you could use to create tasks implemented in Powershell. Think of putting script underneath a <UsingTask> element. These inline tasks would be easy to create and share online. Might include ready-made extensions for Powershell, and/or a data-description format for tool switches, so you could quickly create a task wrapping a command line tool without any compilation.

8) Typing and scoping for items and properties. For example, declaring an item to contain a path; letting a property fall out of scope at the end of a target.

9) Extensible functions. Today we have only 'exists(..)' and it's stepchild 'hastrailingslash(...)'. This would be an extension allowing new functions to be created for use in conditional expressions, and possibly elsewhere, and some functions like perhaps combinepath($(root),$(file)). Perhaps more built-in metadata to go along with %(Filename) and such.

11) An MSBuild debugger with full Visual Studio support for stepping, inspecting locals, and breakpoints. Today you have to use <Message> tasks and gaze at diagnostic logging output.

12) Visualization for project and target dependencies, mining of what files are consumed by what projects, impact analysis for changes to particular files, possibly leading to support for build refactoring at the project/tree level. This is essentially richer logging and datamining and a high level on a huge source tree.

So now here's the poll. If you had only $100 to spend, how would you distribute it amongst them?

Dan

(Edit: gave Distributed build a unique number :-) )

What are Targets, Tasks, and Tools?

msbuild — Wed, 14 Nov 2007 23:35:00 GMT

I've heard these confused in the context of MSBuild, so let's talk a little about what they are:

* A TARGET is a grouping of tasks (often 1) designed to do a particular job. For example, a Link target would be designed to produce a final binary from object files. Targets can declare other targets that they depend on, and MSBuild will make sure those are run first. At present targets are the unit of incremental build: declaring Inputs and Outputs on a target will cause MSBuild to compare their timestamps and skip the target if the inputs are up to date. Because a target is the unit of build, it is the entity that other parts of the build depend on or invoke (via CallTarget or the MSBuild task from another project). The implementation of a target can be modified later, so long as it's "contract" ("consume X and Y properties and list of object files O to produce binary Z") is preserved. A future version will likely have an extensibility model for up-to-date checking, and that will likely plug in at the target level. Once a target has run once in the context of a project, global properties, toolsversion, and build, it will be immediately skipped next time it is invoked, without even checking timestamps. When you write a target, always check that it skips properly in the incremental build case. MSBuild logs information about targets during the build, and any extra logging you want has to happen from a task, especially the <Message>, <Warning>, and <Error> tasks. Targets can be batched, although this is rarely done.

A TASK is a reuseable implementation of some build action. In this case the only task in the Link target would likely be the task named Link. It is fed parameters, including the files to link, does the link, and the build proceeds. At present tasks must be .NET assemblies deriving from the interface Microsoft.Build.Framework.ITask. Tasks can consume and emit items and properties. By design, they have no other insight into the state of the build. They should not discover files on disk - this makes them opaque and less reuseable. PropertyGroup and ItemGroup inside targets can be thought of as tasks, but they have special syntax and powers. If your target is starting to get too "clever", write a task. We hear loud and clear that not everybody wants to write and compile code to create a task, and we'll address this future. Tasks can be batched, and this is commonly done either to force a task to run once for each source file (something like Sources="'%(Compile.Identity)'") or to filter out certain input files, by filtering in the condition on metadata on source items.

A TOOL is an implementation detail of some tasks. A Link task may happen to currently wrap a tool named link.exe and it may happen that the parameters on this Link task pretty much wrap 1:1 with the command line switches on this tool named link.exe. However those are implementation details. One day the link task could be modified to load a "link.dll". Or the owners of link.exe could flip the /clr switch in order to implement ITask themselves. Tasks that wrap tools will want to derive from the ready-made abstract class named Microsoft.Build.Utilities.ToolTask, which provides much of the functionality they will need. Many tasks do not wrap tools. [ Aside: For example, during development of MSBuild v1, we initially implemented resource generation by wrapping resgen.exe. However a key scenario was to build a typical VB or C# project without even having the .NET SDK installed, but resgen.exe did not ship in the .NET Framework and we couldn't persuade them to move it. So we wrote our own resource generation task in code, called GenerateResource. (Resgen.exe is just a thin wrapper over BCL APIs anyway.) A similar situation exists for al.exe, but we did not write a task to do that without the tool: al.exe is some pretty hairy code. This is why you need VS or the .NET SDK installed to build projects with satellite assemblies. But I digress.]

Unlike makefiles, the inputs and outputs on a Target are not necessarily what the “build action”, ie the task, operates on. The task operates only on the parameters it is given. Typically however the Inputs on the Target would be among the inputs passed to the task, but this is not necessarily true. For example the Link target’s Inputs might include $(MSBuildProjectFullPath), so that the target is forced to run if the project file itself has been edited - yet the Link task would not be passed the project file.

Dan