MSBuild Team Blog : Faisal Mohamood

7 Steps to MSBuild

msbuild — Tue, 05 Dec 2006 04:10:00 GMT

Brennan has a great post about all things MSBuild, and covers good breadth on both concepts, as well as the many things you may want to drive with MSBuild (unit testing, web deployment projects, etc). Check it out!

[ Author : Faisal Mohamood ]

Multi-Targeting : How does it work?

msbuild — Wed, 15 Nov 2006 04:30:00 GMT

So in my last post, I described the multi-targeting feature at a very high level. I discussed how there will be three frameworks that you can build "for", and how there will be two toolsets - i.e. .NET Framework 2.0 / MSBuild 2.0 toolset, and the .NET Framework 3.5 / MSBuild 3.5 toolset.

Recall that the capability we have added to MSBuild as a part of this feature is the ability for you to take a project, and build it using either toolset. MSBuild 3.5 supports a new command line parameter known as ToolsVersion that allows you to specify which toolset you want to build using, and is invoked as shown:

msbuild WindowsApplication1.csproj /ToolsVersion:3.5

ToolsVersion is the parameter that you use to force a project to build using a specific toolset. There are only two toolsets available out of the box, but you can also set up "custom" toolsets depending on your needs, and MSBuild can be used to build projects using those that you have defined. More on that later.

So, the next question obviously is, what happens when you build a project using a specific ToolsVersion, and what are the mechanics in action behind the scenes?

To understand this, we have to know a bit about how our project files are described. A standard Visual Studio managed project includes an import statement like the one shown below that is responsible for pulling in all the MSBuild / managed code build process into the project:

In MSBuild 2.0 (Visual Studio 2005 release), $(MSBuildBinPath) evaluated to the location of MSBuild 2.0, i.e. the .NET Framework 2.0 install path. In order to import the new version of Microsoft.CSharp.targets, naturally this needs to change to location of Microsoft.CSharp.targets that ship along with MSBuild 3.5. So in essence, specifying a ToolsVersion value at the command line is essentially equivalent to choosing from a predefined set of valid values for $(MSBuildBinPath). So specifying a ToolsVersion of 2.0 will cause MSBuildBinPath to evaluate to the .NET Framework 2.0 install path, and 3.5 will cause it to evaluate to the .NET Framework 3.5 install path, etc. Additionally, the targets that ship in MSBuild 3.5 include enhancements to the build process that will cause it to use the new set of compilers and other tasks.

MSBuildBinPath has also been "deprecated" in favor of MSBuildToolsPath - new projects created will use MSBuildToolsPath since that is a better name for what the property represents.

Allowing you to specify which toolset to use is only half the story. Multi-Targeting is also about building your code "targeting" a specific version of the framework - for instance, you might want to build your app only so that it relies on 2.0 assemblies and not on any 3.0 or 3.5 assemblies. This is done using another lever that is available via your project file. You can include a property in your project file known as TargetFrameworkVersion that specifies which target framework you are building your application for. TargetFrameworkVersion can either be v2.0, v3.0 or v3.5 - this means that if you are referencing assemblies that are not in a given TargetFrameworkVersion, then you will get errors/warnings at build time that will tell you so. This prevents you from taking on dependencies that will surprise you when you later turn around and try to deploy your app in a more tightly controlled environment that might only have a smaller subset of the three frameworks installed. Since TargetFrameworkVersion is a new concept, you need to be using ToolsVersion 3.5 in order to succefully build projects while honoring the TargetFrameworkVersion property.

This is a confusing topic, so please feel free to ask questions, and provide feedback so that we can help clarify anything on the subject.

[ Author : Faisal Mohamood ]

MSBuild / Visual Studio aware error messages and message formats

msbuild — Fri, 03 Nov 2006 03:17:00 GMT

MSBuild recognizes error messages and warnings that have been specially formatted by many command line tools that typically write to the console. For instance, take a look at the following error messages - they are all properly formatted to be MSBuild and Visual Studio friendly.

Main.cs(17,20): warning CS0168: The variable 'foo' is declared but never used

C:\dir1\foo.resx(2) : error BC30188: Declaration expected.

cl : Command line warning D4024 : unrecognized source file type 'foo.cs', object file assumed

error CS0006: Metadata file 'System.dll' could not be found.

These messages confirm to special format that is shown below, and comprise 5 parts - the order of these parts are important and should not change:

Origin (Required)

Origin can be blank. If present, the origin is usually a tool name, like 'cl' in one of the examples. But it could also be a filename, like 'Main.cs' shown in another example. If it is a filename, then it must be an absolute or a relative filename, followed by an optional paranthesized line/column information in one of the following forms:

(line) or (line-line) or (line-col) or (line,col-col) or (line,col,line,col)

Lines and columns start at 1 in a file - i.e. the beginning of a file is 1, and the leftmost column is 1. If the Origin is a tool name, then it must not change based on local - i.e. it needs to be locale neutral.

Subcategory (Optional)

Subcategory is used to classify the category itself further, and should not be localized.

Category (Required)

Category must be either 'error' or 'warning'. Case does not matter. Like origin, category must not be localized.

Code (Required)

Code identifies an application specific error code / warning code. Code must not be localized and it must not contain spaces.

Text (Optional)

User friendly text that explains the error, and *must* be localized if you cater to multiple locales.

So, how does MSBuild use Errors and Warnings emitted by external tools?

When MSBuild calls command line tools (for instance, csc.exe or vbc.exe), it looks at the output emitted by the tool to standard out / standard error streams. Any lines that match the error format that I just described will be treated specially - i.e. lines that are recognized as errors or warnings will be turned into build errors and warnings respectively.

So, what's the big deal about this anyway? (Here comes the really cool part) :)

To see the real benefit of this, you have to be building from within Visual Studio. Because MSBuild treats these messages specially, they get logged as first class warnings and errors in the Visual Studio task list. If the Origin specifies line/column information, then double clicking on the message will take you to the source of the error in the offending file.

[ Author : Faisal Mohamood ]

/p property values are immutable - (sort of)....

msbuild — Thu, 05 Oct 2006 18:55:00 GMT

Hello there - sorry for the long absence, but we're back and hopefully I'll keep this going again.

So there was a recent forum post about how property values that are specified from the command line are immutable - i.e. it is not straightforward to change them from within the project file. Well, this is mostly true, but there are exceptions.

Let's take an example:

If I've got a property in my project file defined like this,

<Configuration>Debug</Configuration>

</PropertyGroup>

then, calling msbuild with /p:Configuration=Release will override this value (or alternatively set the value if it is not defined) for all projects that are being built. Properties that we pass in with a /p switch on the command line are known as "Global Properties" and so it will flow down to all projects that are being built - so if you were building a solution file, all projects will get the overridden value of this property.

Now, let's say that you have this defined in your project file:

<AppPath Condition=" '$(AppPath)' == '' ">bin\debug\</AppPath>

<AppPath Condition=" !HasTrailingSlash('$(AppPath)') ">$(AppPath)\</AppPath>

</PropertyGroup>

What I am doing here is that I am defining a default value for AppPath property if AppPath is not defined. Additionally, I want to set it up so that if someone has specified an AppPath property, then if it does not already have a trailing slash, I'd like to add it. Looks reasonable, correct?

Actually this will not work as you expect - and is something to watch out for. What happens when you specify something with a /p:AppPath=blah\ is that all in-project property definitions for AppPath are ignored (or essentially overridden), and so none of the in-project property definitions take effect - even the ones with the condition. This makes sense if you think about it because otherwise what exactly are we supposed to override? Would that be the first definition of the property? Or should we override only those values of the property that are checking against an empty value? There isn't a clear answer. We override all property definitions and what is passed in is the effective value.

Fortunately, there is a way around this if it causes you grief - but it isn't declarative like PropertyGroup. You have to use the CreateProperty task inside a target where you intend to process the property, like so:

</CreateProperty>

Alternatively (and perhaps as a better approach), you can use a property name like AppPath as the public property you expect to override from th command line, but filter it out into a different property inside your targets that you then use for doing the actual work - like this:

<_validatedAppPath>$(AppPath)</_validatedAppPath>

<_validatedAppPath Condition=" '$(_validatedAppPath)' == '' ">bin\debug\</_validatedAppPath>

<_validatedAppPath Condition=" !HasTrailingSlash('$(_validatedAppPath)') ">$(_validatedAppPath)\</_validatedAppPath>

</PropertyGroup>

In this case, you validate the /p value and store it in an internal property that is validated, and never directly overridden from the command line.

Hope this helps, and clears some confusion on "strange behavior" if you ever encountered it.

[ Author: Faisal Mohamood ]

New Feature Feedback Request: /IgnoreProjectExtensions - A new command-line switch

msbuild — Fri, 24 Mar 2006 04:53:00 GMT

I've been in a cave just getting things done toward our Orcas release. But there's not much a Program Manager can do without going back to their customers :) So here I am.

So here's the scoop. We've been debating internally about a new command line switch on msbuild.exe for sometime now. I am not certain that we've come to the right answer ourselves - but customers give us the final word - so tell us what you think about the following:

Today, when you invoke msbuild.exe from the command line and don't specify any project files as arguments, then we do some auto inferral and scanning and decide if we should build anything. If we find either a msbuild project (anything that has an extension of *proj) or a solution file (.sln), we will build either the project or the solution as long as there is only one solution or one project in the directory. If there is a solution and a project, we will give preference to the solution. If there's more than one project or more than one solution, we issue an error message because we can't decide which one to build.

Now, what if you wanted to pick the project instead when a solution is encountered, but didn't want to specify the project name as an argument? Old habits die hard - when building Visual Studio code, we are used to issuing a single command to build the current project instead of having to specify the project name that we are specifying. So the only way to get this to work was to think of supporting a /IgnoreProjectExtensions switch that has the following usage model:

msbuild /IgnoreProjectExtensions:vcproj;sln

Now how exactly does this simplify our lives, you ask? We would use a .rsp file that sits next to msbuild.exe that we will add in our environment - after which, simply issuing msbuild will ignore what we ask it to ignore in the solution file.

What do you think? Something you'd find useful? Or would you rather prefer the "clean" command line interface that we have today, and don't think this switch is worth it? The internal debate is about the fact that we don't want to add command line switch after command line switch to msbuild.exe as we please unless there is a specific customer scenario that we are trying to enable. But DevDiv build is also one of our biggest internal customers :) So we look to you to tell us!

Cheers.

[Author: Faisal Mohamood]

How To: Implementing Custom Tasks - Part I

msbuild — Sun, 22 Jan 2006 02:01:00 GMT

While MSBuild is all about build customization,we never really blogged about what is involved in implementing custom tasks. Partly, I suppose that's because implementing a custom task is as easy as falling off a log most of the time: 1) Subclass Microsoft.Build.Utilities.Task abstract class 2) Implement the Execute method

But there's a lot more to tasks than just that. In this series of posts, I hope to cover all there is to know about MSBuild tasks - both at a conceptual level as well as on variations such as ToolTask, AppDomainIsolatedTask, etc. In this post I'd like to set the context for discussing some of those topics.

A task, in principle is nothing more than an implementation of the GoF Command Object Pattern i.e. an object that knows how to perform an action encapsulated via an Execute method. While it is possible to utilize tasks programmatically via code, the primary motivation was to provide a way of performing self standing units of work within build targets.

In the crudest form, a task is a type that implements the Microsoft.Build.Framework.ITask interface from the Microsoft.Build.Framework assembly. Here's a simple implementation of a task that can set environment variables.

using Microsoft.Build.Framework;

namespace SimpleTask
{
    public class SetEnvironmentVariable : ITask
    {
        private IBuildEngine engine;
        public IBuildEngine BuildEngine
        {
            get { return engine; }
            set { engine = value; }
        }

        private ITaskHost host;
        public ITaskHost HostObject
        {
            get { return host; }
            set { host = value; }
        }

        private string name;

        [Required]
        public string Name
        {
            get { return name; }
            set { name = value; }
        }

private string varValue;

        [Required]
        public string Value
        {
            get { return varValue; }
            set { varValue = value; }
        }

        public bool Execute()
        {
            System.Environment.SetEnvironmentVariable(name, varValue);
            string message = string.Format("Environment Variable {0} set to {1}", name, varValue);
            BuildMessageEventArgs args = new BuildMessageEventArgs(
                message, string.Empty, "SetEnvironmentVariable", MessageImportance.Normal);
            engine.LogMessageEvent(args);

            return true;
        }
    }
}

In addition to the Execute method that performs the actual work, at the minimum all tasks must expose two properties - BuildEngine of type IBuildEngine, and HostObject of type ITaskHost. For the purposes of most tasks, the implementation I have shown here will suffice.

IBuildEngine itself is used by the task to report messages, warnings and errors to the MSBuild engine. ITaskHost is an interface that is used to represent host objects that can form a basis for richer communication between tasks and an environment that hosts MSBuild (such as Visual Studio). ITaskHost is extremely useful in cases where you are hosting the build engine yourself. I will cover the details of this beast in a later post.

.NET properties on a task type allow you define parameters on the task in order to communicate with the task via the project file. So, in the case of the SetEnvironmentVariable task, I have defined Name and Value properties as inputs into the task. My use of the Required attribute on these two properties ensures that the invocation of the tasks will not be possible without specifying those two parameters. In this context, it makes sense because the task cannot perform its function without having those to values specified to it.

Finally, returning true from the Execute method indicates success to the MSBuild engine. I have ignored the possibility of the task failing in this case to keep it simple - look for more details on this subject alone in a subsequent post.

Here's how the task is invoked from a project file:

That's all there is to defining and using custom tasks. Of course, it is way simpler to derive from the abstract class Microsoft.Build.Utilities.Task that is included in the Microsoft.Build.Utilities assembly, and that's what you should be using most of the time unless you require a richer interaction with the MSBuild engine.

In the next post, I will dig into more details on interacting with tasks via the project file. We will examine the types of objects that can be passed in and out of tasks, caveats to watch out for, etc.

[ Author: Faisal Mohamood ]

Platform Sprint 11

msbuild — Thu, 05 Jan 2006 22:29:00 GMT

The new year got off to a good start for us with the MSBuild Platform Sprint 11 Backlog selection meeting. Platform Sprint 10 was focused mostly on getting our Compatibility story in place. All of the work from the last sprint in terms of compatibility has been checked in and is now in use.

This means that we can finally get to the good stuff in Platform Sprint 11. Our sprint goal for this sprint is to deliver a working version of multi-proc build capability. Needless to say, everyone on the sprint is excited about this and we are looking forward to the end of the month to see where we stand. I believe that we will have good parts of the multi-proc functionality working at the end of the sprint, and here's why:

Over the past sprints, we have been thinking a lot about the design of the system - like we discussed here and here. It is time now to start thinking in terms of real implementation milestones - and to that end we decided to think strictly in terms of "User Stories". So, everything that has to do with multi-proc that went on our backlog were real stories that would make sense to you from a customer standpoint, as opposed to items that are more closely aligned with the design or the implementation of the system. For instance, the simplest user story in our case was "Build two simple, independent projects in parallel, with no logging support, no disk-bound tasks". You can think of the simple projects as those that have a single target, and a single Message task. That really is our first customer story for multi-proc. We build successively on individual stories till we eventually get to a fully featured multi-proc build functionality.

Of course there are a lot of other activities going on in this sprint as well. We are working on handful of bugs that we could not get to during the tail end of Whidbey. We are also investing in QA work so that we are able to deliver at an even higher level of quality during Orcas.

I couldn't have asked for a better start with the new year. I am highly optimistic of everything that is to come in the following months. In the meantime, I encourage you to log any issues you run into with MSBuild via the MSDN Product Feedback Center so that we can consider them as bug fixes during the current Platform Sprint 11, or the subsequent one.

[ Author: Faisal Mohamood ]

Well Known Limitation: Dynamic items and properties not emitted until target execution completes

msbuild — Tue, 03 Jan 2006 03:46:00 GMT

Sayed Ibrahim Hashimi has written about a restriction that he ran into when using CreateProperty and CreateItem tasks to dynamically emit properties and items. The issue has to do with not being able to access items and properties that are created within a target until the target execution actually completes.

So, if you were to run CreateProperty or CreateItem and immediately execute the CallTarget task to invoke another target that needed access either an item or property that was just created, you will be out of luck. We don't publish dynamic properties or items until the target that created them is done executing. This is a known issue for Whidbey. Fortunately the workaround is simple: have one target emit the items/properties and finish execution before you run the next target that uses them. You can sequence the execution of these two targets via a DependsOnTargets attribute on a master target - alternatively the master target can use CallTarget to invoke both the targets sequentially.

Thank you Sayed for reporting it via the MSDN Product Feedback Center. We have it on our list of feature items to consider for Orcas, and we will need to either publish items and properties globally even before the target execution completes - or alternatively allow parameter passing into targets.

Feedback and blogging about these topics just gives us a better idea of the pain points our customers are experiencing today. Thanks for the feedback and continue sending us more.

[ Author: Faisal Mohamood ]

MSBee CTP released

msbuild — Tue, 03 Jan 2006 00:38:00 GMT

Craig points out that the first CTP for MSBee went out early adopters last week. If you are interested, send an email to ddcpxsln@microsoft.com and get on the CTP!

We look forward to your feedback as we continue to work on MSBee in the coming weeks.

[ Author: Faisal Mohamood ]

How To: Customizing Project File Intellisense in Visual Studio

msbuild — Tue, 08 Nov 2005 21:49:00 GMT

So Neil pointed out that you can find the MSBuild schemas in either the .NET Framework install path, as well as the Visual Studio install location. The one in the Framework folder ships as a reference (since MSBuild really is part of the .NET redist). The one in the Visual Studio install folder \Program Files\Microsoft Visual Studio 8\xml\Schemas\1033 is used by Visual Studio to provide Intellsense support that we all know and love.

We have designed our schemas to be extensible so that you can add custom Intellisense for your custom properties, items and tasks. Microsoft.Build.xsd is where you add your customizations. We have commented the file with examples of how to go about adding your own items, properties and tasks to the schema so that you can get Intellisense for your custom types. You could go one step further to keep your definitions separate and more manageable - instead of adding your definitions directly to this file, create a new file (for eg. MyCompany.MSBuildTypes.xsd) and import it into the Microsoft.Build.xsd.

Note that Microsoft.Build.xsd itself is tiny and contains almost no real matter - most of the pre-defined MSBuild types are imported via another schema file i.e. Microsoft.Build.CommonTypes.xsd.

[ Author: Faisal Mohamood ]

How To: Recursively Copy Files Using the Task

msbuild — Tue, 08 Nov 2005 01:58:00 GMT

Have you ever run into a situation where you had to recursively copy files from one folder into another as a part of your build process? We certainly have run into it a few times, and I thought I'd share with you a neat trick that involves the use of metadata and the Copy task.

Before you can copy a set of files, you need to be able to recursively add those files to an item list. Here's how you do that when declaring items.

The ** wildcard is used in item includes to indicate recursive includes.

So, once you have an item declared as such, you can rely on a piece of standard meta-data that goes with all items (i.e. RecursiveDir) to accomplish your recursive copy. The RecursiveDir metadata when used will return the evaluated value of the ** metadata for each item - and you can use that value to preserve the folder structure for each file when performing the copy. Here's how you invoke the <Copy> task using this piece of metadata to copy recursively.

Hope this helps! Let us know if these tips/tricks and how-tos are useful.

[ Author: Faisal Mohamood ]

HOW TO: Beyond PreBuild and PostBuild Steps

msbuild — Wed, 12 Oct 2005 21:43:00 GMT

Do you use PreBuild and PostBuild steps that are provided in the IDE? You will find that they are still available in Visual Studio 2005 - but from within the IDE, you only get the ability to run command line batch files / scripts. Essentially, prebuild and postbuild steps get executed using the <Exec> task.

It's all well and good - except we already provide a rich set of tasks for you to use so that you have more flexibility and control over your build. For instance, using the <Copy> task instead of the copy command gives you richer programmatic control over that step. So, what if you wanted to implement PreBuild and PostBuild using MSBuild tasks?

Open up your project file for edit (using the supercool tip I mentioned earlier). You will see that all Visual Studio projects have two targets that are commented out - <BeforeBuild> and <AfterBuild>. All you need to do is really have targets with those names in your project (we call it target overriding - because those targets are actually pre-defined in the Microsoft.Common.targets file that contains the Visual Studio Build process). So it's these targets that will allow you to implement your pre-build and post-build steps using MSBuild tasks instead of relying on command line scripts.

Here's an example of how it would look in the project file:

[Author: Faisal Mohamood]

Are you new to MSBuild? Check out our Hands-On-Lab

msbuild — Tue, 11 Oct 2005 22:05:00 GMT

This MSBuild Hands-On-Lab introduces you to the fundamentals of MSBuild. It talks about the basics such as how to hand-author a simple MSBuild project file, and then builds on that to show you how to take it all the way to more advanced aspects such as incremental builds and target level dependency analysys (TLDA), authoring custom tasks, Visual Studio and how it relates to MSBuild, how Visual Studio integrates MSBuild, and all that good stuff. Check it out!

[Author: Faisal Mohamood]

PDC Session Slide Decks are Online

msbuild — Tue, 11 Oct 2005 21:26:00 GMT

The slide-decks for the two MSBuild talks we presented at the conference have been available on-line for a while now. Check them out: MSBuild - Architecting a Custom Build System by Rajeev Goel and MSBuild - Tips & Tricks by Faisal Mohamood are both available.

[Author: Faisal Mohamood]

Editing MSBuild project files in Visual Studio

msbuild — Thu, 29 Sep 2005 04:50:00 GMT

Being able to edit your Visual Studio generated projects is one of the fundamental goals of MSBuild. Being able to edit your project file to customize and extend is not something that just happened to work :) MSBuild and Visual Studio 2005 were designed right from the start to enable you to do this. If you ask me, this is the "essence of MSBuild"

So let's say you are working on your project, and have your project loaded in the IDE - What's the easiest way for you to be able to quickly drop into the MSBuild project file (either .csproj or .vbproj) so that you can edit the project to customize your build?

Pretty simple - if you are using one of the builds past the June CTP, you can do this in two steps.

1. Right click on your project in solution explorer and select Unload Project

2. Right click on the project (tagged as unavailable in solution explorer) and click "Edit yourproj.csproj"

It's that simple. No more notepad. No more File->Open->blah->blah->blah. Besides, you get full inellisense based on the MSBuild schema! Wait - there's something better. We will not touch any of the customizations you make to your projects. Customizations will work inside the IDE as well as from the command line - but we will not mess with your changes!

[Author: Faisal Mohamood]