MSBuild Property Functions

MSBuild Property Functions

  • Comments 22

Have you ever wanted to do something simple in a build, like get a substring of a property value, and found that MSBuild didn't have syntax for it? You then had to write a task for it, which was tiresome to do for such a simple operation. What's more, if you wanted to do this during evaluation – outside of a target – you couldn't run a task there anyway.

In MSBuild 4.0 we addressed this by adding "property functions" which allow you to execute a variety of regular .NET API calls during evaluation or execution.

Here's an example. For the default VB or C# project, both the intermediate and final output directories are by default below the project's directory. Instead, I'm going to move the final outputs to c:\outputs\<some guid>\ followed by the usual path. You can see below how I did this. I removed the <OutputPath> property and replaced it with an expression that generated a guid for this project.

image

Now I reopen the project and hit build to show it worked:

image

 

 

 

Syntax

There are two syntaxes, as follows. They're intended to be fairly close to the existing Powershell syntax for calling .NET types. The first is for calling static members:

$([Namespace.Type]::Method(..parameters…))

$([Namespace.Type]::Property)

$([Namespace.Type]::set_Property(value))

The second is for instance members on the String class. You write it as if the property itself is a string.

$(property.Method(..parameters...))

$(property.Property)

$(property.set_Property(value))

Notice that when setting a property, you must use CLR syntax for properties ("set_XXX(value)").

The neat part is that these can all be nested – be sure to match your parentheses correctly of course. We attempt to coerce parameters as far as possible in order to find a method or overload that will work.

If you want to pass strings, quote with back-ticks.

When you pass the result of one expression to another, the types are maintained along the chain. This helps the binder find the member you are trying to call. Only when the final result of the expression needs to go into the build do we coerce it to a string.

Some examples may help:

 

Examples

 

image

 

Limitations

* You can't run instance methods on raw strings. For example $("c:\foo".get_Length()). They must go into a property first.

* Out parameters won't work – there are no intermediate values except for the return value. No delegates or generics either.

* If we coerce to the wrong overload, you may be able to use a Convert method to force the correct one.

* By default, you can only call certain members on certain types – selected to be free of side-effects. Here's the full list:

(1) All members on the following types:

System.Byte
System.Char
System.Convert
System.DateTime
System.Decimal
System.Double
System.Enum
System.Guid
System.Int16
System.Int32
System.Int64
System.IO.Path
System.Math
System.UInt16
System.UInt32
System.UInt64
System.SByte
System.Single
System.String
System.StringComparer
System.TimeSpan
System.Text.RegularExpressions.Regex
System.Version
MSBuild  (see below)
Microsoft.Build.Utilities.ToolLocationHelper

(2) Selected members on certain other types:

System.Environment::CommandLine
System.Environment::ExpandEnvironmentVariables
System.Environment::GetEnvironmentVariable
System.Environment::GetEnvironmentVariables
System.Environment::GetFolderPath
System.Environment::GetLogicalDrives
System.IO.Directory::GetDirectories
System.IO.Directory::GetFiles
System.IO.Directory::GetLastAccessTime
System.IO.Directory::GetLastWriteTime
System.IO.Directory::GetParent
System.IO.File::Exists
System.IO.File::GetCreationTime
System.IO.File::GetAttributes
System.IO.File::GetLastAccessTime
System.IO.File::GetLastWriteTime
System.IO.File::ReadAllText

But I want to use other types and custom types ..

The reason we prevent this is to make it more safe to load Visual Studio projects. Otherwise, someone could give you a project that formatted your hard-disk during evaluation. Visual Studio load-time safety is actually more complicated than that – some targets will run and do arbitrary things – but we didn't want to make new opportunities for badness. We could have made this limitation only apply to Visual Studio, but then it would be possible to have your build work differently on the command line. I'd like to hear your feedback on this – is the list too constraining?

You can decide whether we made the correct call here. Meanwhile there is an unsupported way to call members on arbitrary types: set the environment variable MSBUILDENABLEALLPROPERTYFUNCTIONS=1. You can now use any type in any assembly. Of course, MSbuild has to know what assembly it is in (it knows them for the list above); and the CLR binder still has to be able to find it to load it.

To figure out the assembly, it tries to work up the name. So for this example (assuming the environment variable is set)

$([Microsoft.VisualBasic.FileIO.FileSystem]::CurrentDirectory)

it will look for Microsoft.VisualBasic.FileIO.dll, then Microsoft.VisualBasic.dll (which it will find and load from the GAC) and you will get the value of the current directory.

If that's not going to work for your assembly, it is possible to pass in a strong name. For example, the above could equivalently be written like this:

$([Microsoft.VisualBasic.FileIO.FileSystem, Microsoft.VisualBasic, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]::CurrentDirectory)

This means you can write your own functions for MSBuild to call – just put the assembly somewhere that the CLR can find it. By doing that, you can (if you set the environment variable) cause your build do do absolutely anything during property evaluation.

Here's a screenshot of these examples:

image

Have fun, and let me know what you think. I'd love to get suggestions on how we can improve this.

Dan (Dev lead, MSBuild)

 

[Update] I'll post about this separately later, but here's one other property function that will be useful to some people:

$([MSBuild]::GetDirectoryNameOfFileAbove(directory, filename)

Looks in the designated directory, then progressively in the parent directories until it finds the file provided or hits the root. Then it returns the path to that root. What would you need such an odd function for? It's very useful if you have a tree of projects in source control, and want them all to share a single imported file. You can check it in at the root, but how do they find it to import it? They could all specify the relative path, but that's cumbersome as it's different depending on where they are. Or, you could set an environment variable pointing to the root, but you might not want to use environment variables. That's where this function comes in handy – you can write something like this, and all projects will be able to find and import it:

  <Import Project="$([MSBuild]::GetDirectoryNameOfFileAbove($(MSBuildThisFileDirectory), EnlistmentInfo.props))\EnlistmentInfo.props" Condition=" '$([MSBuild]::GetDirectoryNameOfFileAbove($(MSBuildThisFileDirectory), EnlistmentInfo.props))' != '' " />

 

[Update 2 – it's useful to gather all this in one place for reference.]

Built-in MSBuild functions

The full list of built-in [MSBuild] functions, like the one above, are in the MSDN topic here. As well as the unusual "GetDirectoryNameOfFileAbove" that I showed above, they also offer arithmetic (useful, for example, for modifying version numbers), functions to convert to and from the MSBuild escaping format (on rare occasions, that is useful).

Error handling

The functions parser is pretty robust but not necessarily that helpful. Errors you can get include

(1) It doesn't evaluate but just comes out as a string. Your syntax isn't recognized as an attempt at a function, most likely you've missed a closing parenthesis somewhere.

(2) error MSB4184: The expression "…" cannot be evaluated. It treated it as a function, but probably it couldn't parse it.

(3) error MSB4184: The expression "…" cannot be evaluated. Method '…' not found. It could parse it, but not find a member it could coerce to, or it was ambiguous. Verify you weren't calling a static member using instance member syntax. Try to make the call less ambiguous between overloads, either by picking another overload (that perhaps has a unique number of parameters) or using the Convert class to force one of the parameters explicitly to the type the method wants. One common case where this happens is where one overload takes an int, and the other an enumeration.

(4) error MSB4184: The expression "[System.Text.RegularExpressions.Regex]::Replace(d:\bar\libs;;c:\Foo\libs;, \lib\x86, '')" cannot be evaluated. parsing "\lib\x86" - Unrecognized escape sequence \l.  Here's an example where it bound the method, but the method threw an exception ("unrecognized escape sequence") because the parameter values weren't valid.

(5) error MSB4186: Invalid static method invocation syntax: "....". Method 'System.Text.RegularExpressions.Regex.Replace' not found. Static method invocation should be of the form: $([FullTypeName]::Method()), e.g. $([System.IO.Path]::Combine(`a`, `b`)).. Hopefully self explanatory. As well as checking your syntax, check that it's actually a static member rather than an instance member.

Arrays

Arrays are tricky as the C# style syntax "new Foo[]" does not work, and Array.CreateInstance needs a Type object. To get an array, you either need a method or property that returns one, or you use a special case where we can force a string into an array. Here's an example of the latter case:

$(LibraryPath.Split(`;`))

In this case, the string.Split overload wants a string array, and we're converting the string into an array with one element.

Regex Example

Here I'm replacing a string in the property "LibraryPath", case insensitively.

<LibraryPath>$([System.Text.RegularExpressions.Regex]::Replace($(LibraryPath), `$(DXSDK_DIR)\\lib\\x86`, ``, System.Text.RegularExpressions.RegexOptions.IgnoreCase))</LibraryPath>

Here's how to do the same with string manipulation, less pretty.

<LibraryPath>$(LibraryPath.Remove($(LibraryPath.IndexOf(`$(DXSDK_DIR)\lib\x86`, 0, $(IncludePath.Length), System.StringComparison.OrdinalIgnoreCase)), $([MSBuild]::Add($(DXSDK_DIR.Length), 8))))</LibraryPath>

Future Thoughts

So far in my own work I've found this feature really useful, and far, far, better than creating a task. It can make some simple tasks that were impossible possible, and often, easy. But as you can see from the examples above, it often has rough edges and sometimes it can be horrible to read and write. Here's some ways we can make it better in future:

  1. A "language service" would make writing these expressions much easier to get right. What that means is a better XML editing experience inside Visual Studio for MSBuild format files, that understands this syntax, gives you intellisense, and squiggles errors. (Especially missed closing parentheses!)
  2. A smarter binder. Right now we're using the regular CLR binder, with some customizations. Powershell has a much more heavily customized binder, and I believe there is now one for the DLR. If we switch to that, it would be much easier to get the method you want, with appropriate type conversion done for you.
  3. Some more methods in the [MSBuild] namespace for common tasks. For example, a method like $([MSBuild]::ReplaceInsensitive(`$(DXSDK_DIR)\\lib\\x86`, ``)) would be easier than the long regular expression example above.
  4. Enable more types and members in the .NET Framework that are safe, and useful.
  5. Make it possible to expose your own functions, that you can use with this syntax, but write in inline code like MSBuild 4.0 allows you to do for tasks.
  6. Offer some similar powers for items and metadata.

What do you think?

[Update 3]

Here's an example of using a constructor – and of doing so to do a string operation on a piece of metadata.

<Message Text="$([System.String]::new('$(_NTDRIVE)$(_NTROOT)\%(ConfigPaths.Identity)').StartsWith('\$(MSBuildProjectDirectory)', System.StringComparison.OrdinalIgnoreCase))"/>

Leave a Comment
  • Please add 3 and 6 and type the answer here:
  • Post
  • I read with *great* interest this blog article because I need to invoke a method in a custom assembly that I've written in order to set a couple properties that are then used in file references.  So the properties have to be set long before any targets are executed.  But yes it is essential that I be able to invoke my own assembly/method.  This would be the magic bullet to solve a problem that I describe in my recent post to the "MSBuild" MSDN forum.  The url to that post is: http://social.msdn.microsoft.com/Forums/en-US/msbuild/thread/cbbd8d58-39d3-4d1b-a153-94adb3dc98f6

    My post is titled "Dealing with References that span Team Projects" and is dated 4/16/2010.

    The environment variable approach for enabling this will be satisfactory, I suppose.  My plan was to put the assembly in the GAC, so I didn't think that would pose much of a security risk, but maybe I'm wrong about that.   Whatever approach is used to enable it, I would like to think that it is a supported approach ... being able to invoke a custom assembly/method is way too important and useful to be unsupported.  Seriously.

    Thanks for listening.

    Mike

  • Dan,

    You give an example of specifying a strong name in your article:

    $([Microsoft.VisualBasic.FileIO.FileSystem, Microsoft.VisualBasic, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]::CurrentDirectory)

    When specifying a strong name,

    I have always seen four components listed -- name, version, culture, and PublicKeyToken, but I have never seen two assembly names listed.  When it's done that way, does the msbuild parser just assume that the last 3 attributes (version, culture, and token) apply only to the second assembly name?  

    Mike

  • OK, I'll concede that it's probably true that I should be able to accomplish what I want by using a method from System.Text.RegularExpressions.Regex, but I'd rather invoke my custom method.

    Mike

  • In the fully-qualified specification, the first part is the fully qualified name of the type, not an assembly name.  In the first example, the system tries to infer the assembly name from the type, and it happens that Microsoft.VisualBasic is both the name of the assembly and part of the fully-qualified type name.

  • Thanks for the clarification on the type name versus assembly name, Cliff.

    Mike

  • Here's a thought:  if invoking a method from a custom assembly that is not placed in the GAC is more of a security risk than invoking one from a GAC'ed assembly, perhaps a compromise solution would be to provide 3 values for the MSBUILDENABLEALLPROPERTYFUNCTIONS environment variable:  1 would enable invocation of GAC'ed assemblies only, 2 would enable invocation of either GAC'ed or non-GAC'ed assemblies, and 0 would disable both.

    Mike

  • In regards to the property function, GetDirectoryNameOfFileAbove, mentioned in your blog article, I agree that this would be a handy function.  Needless to say, it would need to fail gracefully if the file is not found (since it probably won't be found when a TFS server build is performed).  I assume, that it will simply return an empty string in that case.

    Mike

  • @Mkedwards, yes, the function will return blank if it can't find the file. That's why I recommend a condition to check for that, as you see in my example.

    Hope this helps.

    Dan

  • Incidentally a glaring omission here is a way to Split a string (typically a list stored in a property as semicolon separated elements). The problem is that Split is not a static method on the String class. We should have put it on the MSBuild functions.

    If anyone comes up with a trick to make this work, please let me know.

    Dan

  • For the record, the other MSBuild functions that you can use as well as the regular .NET functions listed, are documented here:

    http://msdn.microsoft.com/en-us/library/dd633440.aspx

    For example, you can do arithmetic (useful on version numbers) and escaping/unescaping.

    Dan

  • OK I found a way to do Split, and updated the posting above :-)

    Dan

  • Also there is MakeRelative. Eg

    $([MSBuild]::MakeRelative(`$(InetRoot)`, `$(MSBuildProjectFullPath)`))

    Will give the relative path from $(InetRoot) to the current project file.

    Dan

  • Another example, from a question on StackOverflow. This illustrates "hinting" to the type coercion. The format

    FILEPARSER_1_0_3_BUILD_007

    was desired, where the last digit is padded to three digits. It can be done like this:

    $([System.String]::Format('FILEPARSER_$(Major)_$(Minor)_$(Build)_BUILD_{0:000}',

    $([System.Convert]::ToInt32($(Revision)))))

    In this case, without the ToInt32, MSBuild would treat $(Revision) like a string when it upcasts to Object for the Format method. That would prevent the format specifier doing the numerical padding. It's difficult to see how MSBuild could get this right in the general case, but this isn't too horrible to read, and certainly a lot more concise than other ways of doing it.

    Dan

  • While you allowed System.IO.Directory::GetDirectories , you forgot about System.IO.SearchOption which is used in one of overloads.

    This makes it impossible to enumerate subdirectories :(

  • Example of a trick to use string functions on metadata values. This:

    <Project ToolsVersion="4.0" DefaultTargets="Build" xmlns="schemas.microsoft.com/.../2003">

     <ItemGroup>

       <File Include="a.cs">

         <Base>Base Value</Base>

       </File>

     </ItemGroup>

     <Target Name="Build">

       <ItemGroup>

         <File Include="@(File)">

           <Modified>%(Base) Modified</Modified>

           <Modified2>$([System.String]::Copy('%(File.Modified)').Replace('Modified', 'Replaced'))</Modified2>

         </File>

       </ItemGroup>

       <Message Text="%(File.Base)"/>

       <Message Text="%(File.Modified)"/>

       <Message Text="%(File.Modified2)"/>

     </Target>

    </Project>

    gives:

    Build:

     Base Value

     Base Value Modified

     Base Value Replaced

Page 1 of 2 (22 items) 12