MSBuild Property Functions

Visual Studio Blog

April 2nd, 20100 0

Have you ever wanted to do something simple in a build, like get a substring of a property value, and found that MSBuild didn’t have syntax for it? You then had to write a task for it, which was tiresome to do for such a simple operation. What’s more, if you wanted to do this during evaluation – outside of a target – you couldn’t run a task there anyway.

In MSBuild 4.0 we addressed this by adding "property functions" which allow you to execute a variety of regular .NET API calls during evaluation or execution.

Here’s an example. For the default VB or C# project, both the intermediate and final output directories are by default below the project’s directory. Instead, I’m going to move the final outputs to c:outputs<some guid> followed by the usual path. You can see below how I did this. I removed the <OutputPath> property and replaced it with an expression that generated a guid for this project.

Now I reopen the project and hit build to show it worked:

Syntax

There are two syntaxes, as follows. They’re intended to be fairly close to the existing Powershell syntax for calling .NET types. The first is for calling static members:

$([Namespace.Type]::Method(..parameters…))

$([Namespace.Type]::Property)

$([Namespace.Type]::set_Property(value))

The second is for instance members on the String class. You write it as if the property itself is a string.

$(property.Method(..parameters…))

$(property.Property)

$(property.set_Property(value))

Notice that when setting a property, you must use CLR syntax for properties ("set_XXX(value)").

The neat part is that these can all be nested – be sure to match your parentheses correctly of course. We attempt to coerce parameters as far as possible in order to find a method or overload that will work.

If you want to pass strings, quote with back-ticks.

When you pass the result of one expression to another, the types are maintained along the chain. This helps the binder find the member you are trying to call. Only when the final result of the expression needs to go into the build do we coerce it to a string.

Some examples may help:

Examples

Limitations

* You can’t run instance methods on raw strings. For example $("c:foo".get_Length()). They must go into a property first.

* Out parameters won’t work – there are no intermediate values except for the return value. No delegates or generics either.

* If we coerce to the wrong overload, you may be able to use a Convert method to force the correct one.

* By default, you can only call certain members on certain types – selected to be free of side-effects. Here’s the full list:

(1) All members on the following types:

System.Byte
System.Char
System.Convert
System.DateTime
System.Decimal
System.Double
System.Enum
System.Guid
System.Int16
System.Int32
System.Int64
System.IO.Path
System.Math
System.UInt16
System.UInt32
System.UInt64
System.SByte
System.Single
System.String
System.StringComparer
System.TimeSpan
System.Text.RegularExpressions.Regex
System.Version
MSBuild (see below)
Microsoft.Build.Utilities.ToolLocationHelper

(2) Selected members on certain other types:

System.Environment::CommandLine
System.Environment::ExpandEnvironmentVariables
System.Environment::GetEnvironmentVariable
System.Environment::GetEnvironmentVariables
System.Environment::GetFolderPath
System.Environment::GetLogicalDrives
System.IO.Directory::GetDirectories
System.IO.Directory::GetFiles
System.IO.Directory::GetLastAccessTime
System.IO.Directory::GetLastWriteTime
System.IO.Directory::GetParent
System.IO.File::Exists
System.IO.File::GetCreationTime
System.IO.File::GetAttributes
System.IO.File::GetLastAccessTime
System.IO.File::GetL astWriteTime
System.IO.File::ReadAllText

But I want to use other types and custom types ..

The reason we prevent this is to make it more safe to load Visual Studio projects. Otherwise, someone could give you a project that formatted your hard-disk during evaluation. Visual Studio load-time safety is actually more complicated than that – some targets will run and do arbitrary things – but we didn’t want to make new opportunities for badness. We could have made this limitation only apply to Visual Studio, but then it would be possible to have your build work differently on the command line. I’d like to hear your feedback on this – is the list too constraining?

You can decide whether we made the correct call here. Meanwhile there is an unsupported way to call members on arbitrary types: set the environment variable MSBUILDENABLEALLPROPERTYFUNCTIONS=1. You can now use any type in any assembly. Of course, MSbuild has to know what assembly it is in (it knows them for the list above); and the CLR binder still has to be able to find it to load it.

To figure out the assembly, it tries to work up the name. So for this example (assuming the environment variable is set)

$([Microsoft.VisualBasic.FileIO.FileSystem]::CurrentDirectory)

it will look for Microsoft.VisualBasic.FileIO.dll, then Microsoft.VisualBasic.dll (which it will find and load from the GAC) and you will get the value of the current directory.

If that’s not going to work for your assembly, it is possible to pass in a strong name. For example, the above could equivalently be written like this:

$([Microsoft.VisualBasic.FileIO.FileSystem, Microsoft.VisualBasic, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]::CurrentDirectory)

This means you can write your own functions for MSBuild to call – just put the assembly somewhere that the CLR can find it. By doing that, you can (if you set the environment variable) cause your build do do absolutely anything during property evaluation.

Here’s a screenshot of these examples:

Have fun, and let me know what you think. I’d love to get suggestions on how we can improve this.

Dan (Dev lead, MSBuild)

[Update] I’ll post about this separately later, but here’s one other property function that will be useful to some people:

$([MSBuild]::GetDirectoryNameOfFileAbove(directory, filename)

Looks in the designated directory, then progressively in the parent directories until it finds the file provided or hits the root. Then it returns the path to that root. What would you need such an odd function for? It’s very useful if you have a tree of projects in source control, and want them all to share a single imported file. You can check it in at the root, but how do they find it to import it? They could all specify the relative path, but that’s cumbersome as it’s different depending on where they are. Or, you could set an environment variable pointing to the root, but you might not want to use environment variables. That’s where this function comes in handy – you can write something like this, and all projects will be able to find and import it:

[Update 2 – it’s useful to gather all this in one place for reference.]

Built-in MSBuild functions

The full list of built-in [MSBuild] functions, like the one above, are in the MSDN topic here. As well as the unusual "GetDirectoryNameOfFileAbove" that I showed above, they also offer arithmetic (useful, for example, for modifying version numbers), functions to convert to and from the MSBuild escaping format (on rare occasions, that is useful).

Error handling

The functions parser is pretty robust but not necessarily that helpful. Errors you can get include

(1) It doesn’t evaluate but just comes out as a string. Your syntax isn’t recognized as an attempt at a function, most likely you’ve missed a closing parenthesis somewhere.

(2) error MSB4184: The expression "…" cannot be evaluated. It treated it as a function, but probably it couldn’t parse it.

(3) error MSB4184: The expression "…" cannot be evaluated. Method ‘…’ not found. It could parse it, but not find a member it could coerce to, or it was ambiguous. Verify you weren’t calling a static member using instance member syntax. Try to make the call less ambiguous between overloads, either by picking another overload (that perhaps has a unique number of parameters) or using the Convert class to force one of the parameters explicitly to the type the method wants. One common case where this happens is where one overload takes an int, and the other an enumeration.

(4) error MSB4184: The expression "[System.Text.RegularExpressions.Regex]::Replace(d:barlibs;;c:Foolibs;, libx86, ”)" cannot be evaluated. parsing "libx86" – Unrecognized escape sequence l. Here’s an example where it bound the method, but the method threw an exception ("unrecognized escape sequence") because the parameter values weren’t valid.

(5) error MSB4186: Invalid static method invocation syntax: "….". Method ‘System.Text.RegularExpressions.Regex.Replace’ not found. Static method invocation should be of the form: $([FullTypeName]::Method()), e.g. $([System.IO.Path]::Combine(`a`, `b`)).. Hopefully self explanatory. As well as checking your syntax, check that it’s actually a static member rather than an instance member.

Arrays

Arrays are tricky as the C# style syntax "new Foo[]" does not work, and Array.CreateInstance needs a Type object. To get an array, you either need a method or property that returns one, or you use a special case where we can force a string into an array. Here’s an example of the latter case:

$(LibraryPath.Split(`;`))

In this case, the string.Split overload wants a string array, and we’re converting the string into an array with one element.

Regex Example

Here I’m replacing a string in the property "LibraryPath", case insensitively.

<LibraryPath>$([System.Text.RegularExpressions.Regex]::Replace($(LibraryPath), `$(DXSDK_DIR)\lib\x86`, “, System.Text.RegularExpressions.RegexOptions.IgnoreCase))</Library Path>

Here’s how to do the same with string manipulation, less pretty.

<LibraryPath>$(LibraryPath.Remove($(LibraryPath.IndexOf(`$(DXSDK_DIR)libx86`, 0, $(IncludePath.Length), System.StringComparison.OrdinalIgnoreCase)), $([MSBuild]::Add($(DXSDK_DIR.Length), 8))))</LibraryPath>

Future Thoughts

So far in my own work I’ve found this feature really useful, and far, far, better than creating a task. It can make some simple tasks that were impossible possible, and often, easy. But as you can see from the examples above, it often has rough edges and sometimes it can be horrible to read and write. Here’s some ways we can make it better in future:

A "language service" would make writing these expressions much easier to get right. What that means is a better XML editing experience inside Visual Studio for MSBuild format files, that understands this syntax, gives you intellisense, and squiggles errors. (Especially missed closing parentheses!)
A smarter binder. Right now we’re using the regular CLR binder, with some customizations. Powershell has a much more heavily customized binder, and I believe there is now one for the DLR. If we switch to that, it would be much easier to get the method you want, with appropriate type conversion done for you.
Some more methods in the [MSBuild] namespace for common tasks. For example, a method like $([MSBuild]::ReplaceInsensitive(`$(DXSDK_DIR)\lib\x86`, “)) would be easier than the long regular expression example above.
Enable more types and members in the .NET Framework that are safe, and useful.
Make it possible to expose your own functions, that you can use with this syntax, but write in inline code like MSBuild 4.0 allows you to do for tasks.
Offer some similar powers for items and metadata.

What do you think?

[Update 3]

Here’s an example of using a constructor – and of doing so to do a string operation on a piece of metadata.