Quick Help on VS2010 Custom Build Rule

Quick Help on VS2010 Custom Build Rule

Rate This
  • Comments 40

Hi my name is Li Shao. I am a Software Design Engineer in Test for the C++ team. In my earlier blog “Visual Studio 2010 C++ Project Upgrade Guide”, I have mentioned about the change of Custom build Rules in VS2010. In this post, I would like to explain in more detail of the new format of the Custom Build Rules and some of its limitations in VS2010.

Custom build rule is a build feature introduced in VS2005. It provides the ability for the users to easily Plug-In third party tools to use in their build process. The custom build rule is defined by “.rules” files. Rules for MASM (Microsoft Micro Assembler) and LIC (License Compiler) are shipped in the box for VS2005 and VS2008.  

In VS2010, due to the migration to MSBuild, the information in the rules file is represented by three files: .XML, .props and .targets files. The XML file defines the property and switch mapping. The props file initializes the property values and the targets file generates the task based on the XML file and has the build logic. MASM and LIC custom build rules are also shipped in VS2010 with this new format. In the next couple of paragraphs, I will use MASM rule as an example to explain some important parts of the new custom build rule format and hope to give you some ideas on how to read custom build rule files if you need to.

 MASM custom build rule is available at %ProgramFiles%\MSBuild\Microsoft.Cpp\v4.0\BuildCustomizations or %ProgramFiles(x86)%\MSBuild\Microsoft.Cpp\v4.0\BuildCustomizations on x64 machine. You have the following in the MASM.XML file. 

<BoolProperty

      Name="GenerateDebugInformation"

      HelpUrl="http://msdn.microsoft.com/library/default.asp?url=/library/en-us/vcmasm/html/vclrfml.asp"

      DisplayName="Generate Debug Information"

      Description="Generates Debug Information.     (/Zi)"

      Switch="/Zi" />

 

With this XML file definition, “GenerateDebugInformation” is mapped to /Zi

In the file MASM.Props, “GenerateDebugInformation” is initialized to “True” (see highlighted line below)

<ItemDefinitionGroup>

  <MASM>

    <NoLogo>true</NoLogo>

    <GenerateDebugInformation>true</GenerateDebugInformation>

    <ObjectFileName>$(IntDir)%(FileName).obj</ObjectFileName>

    <PreserveIdentifierCase>0</PreserveIdentifierCase>

    <WarningLevel>3</WarningLevel>

    <PackAlignmentBoundary>0</PackAlignmentBoundary>

    <CallingConvention>0</CallingConvention>

    <ErrorReporting>0</ErrorReporting>

    <CommandLineTemplate Condition="'$(Platform)' == 'Win32'">ml.exe /c [AllOptions] [AdditionalOptions] /Ta[Inputs]</CommandLineTemplate>

    <CommandLineTemplate Condition="'$(Platform)' == 'X64'">ml64.exe /c [AllOptions] [AdditionalOptions] /Ta[Inputs]</CommandLineTemplate>

    <CommandLineTemplate Condition="'$(Platform)' != 'Win32' and '$(Platform)' != 'X64'">echo MASM not supported on this platform</CommandLineTemplate>

    <ExecutionDescription>Assembling [Inputs]...</ExecutionDescription>

  </MASM>

</ItemDefinitionGroup>

 

The MASM.XML file is imported by the MASM.targets file. MSBuild relies on the Task Generator to generate the task for the custom build rule.

<UsingTask

    TaskName="MASM"

    TaskFactory="XamlTaskFactory"

    AssemblyName="Microsoft.Build.Tasks.v4.0">

  <Task>$(MSBuildThisFileDirectory)$(MSBuildThisFileName).xml</Task>

</UsingTask>

 

The values initialized in MASM.props file is also passed to the targets:

<MASM

      GenerateDebugInformation="%(MASM.GenerateDebugInformation)"

      Inputs="%(MASM.Identity)" />

 

The build logic is in the .targets file as well. For example, these lines define the inputs and outputs for MASM:

Outputs="%(MASM.ObjectFileName)"

Inputs="%(MASM.Identity);%(MASM.AdditionalDependencies);$(MSBuildProjectFile)"

This defines what outputs from MASM need to be passed to Linker/Lib:

<Target

    Name="ComputeMASMOutput"

    Condition="'@(MASM)' != ''">

  <ItemGroup>

    <Link Include="@(MASM->Metadata('ObjectFileName')->Distinct()->ClearMetadata())" Condition="'%(MASM.ExcludedFromBuild)' != 'true'"/>

    <Lib Include="@(MASM->Metadata('ObjectFileName')->Distinct()->ClearMetadata())" Condition="'%(MASM.ExcludedFromBuild)' != 'true'"/>

  </ItemGroup>

</Target >

 

MASM.props and MASM.targets file are imported by project files (.vcxproj) if the custom build rule for MASM is enabled.

<ImportGroup Label="ExtensionSettings">

    <Import Project="$(VCTargetsPath)\BuildCustomizations\masm.props" />

</ImportGroup>

 

<ImportGroup Label="ExtensionTargets">

    <Import Project="$(VCTargetsPath)\BuildCustomizations\masm.targets" />

</ImportGroup>

Limitations and workarounds

VS2010 supports automatic conversion of the .rules file to .xml, .props and .targets files. The new build rules, in general, can be built without any modification. However, there are a few limitations that may require additional modification to the rule files. I will use custom build rules comes with CUDA tool kit and custom build rules that work with YASM tool as examples.

Environment variables as part of the switches

One major limitation is that MSBuild does not support expanding environment variables if they are part of the switches when generating tasks. Take the CUDA rule, for example. You have the following in your original rule file:

<EnumProperty

     Name="NvccCompilation"

     DisplayName="NVCC Compilation Type"

     Description="Select desired output of NVCC compilation (-c/-compile, -cuda, -gpu, -cubin, -ptx)"

     >

  <Values>

    <EnumValue

     Value="0"

     Switch="--compile -o &quot;$(IntDir)\$(InputName).cu.obj&quot;"

     DisplayName="Generate hybrid object file  (--compile / -c)"

      />

    <EnumValue

     Value="1"

     Switch="-cuda -o &quot;$(IntDir)\$(InputName).cu.c&quot;"

     DisplayName="Generate hybrid .c file  (-cuda)"

      />

    />

 

They will be converted into:

<EnumProperty

      Name="NvccCompilation"

      HelpContext="0"

      DisplayName="NVCC Compilation Type"

      Description="Select desired output of NVCC compilation (-c/-compile, -cuda, -gpu, -cubin, -ptx)">

  <EnumValue

    Name="0"

    DisplayName="Generate hybrid object file  (--compile / -c)"

    Switch="--compile -o &quot;$(IntDir)\$(InputName).cu.obj&quot;" />

  <EnumValue

    Name="1"

    DisplayName="Generate hybrid .c file  (-cuda)"

    Switch="-cuda -o &quot;$(IntDir)\$(InputName).cu.c&quot;" />

</EnumProperty>

 

The $(IntDir) and $(IntputName), however, will not be expanded during compile due to the limitation of the task generated by the task generator. As a result, you will see the following command line that $(IntDir) and $(InputName) are not expanded.

-arch sm_10 -ccbin "C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\bin"    -Xcompiler "/EHsc /W3 /nologo /O2 /Zi   /MT  " -I -maxrregcount=32  --compile -o "$(IntDir)\$(InputName).cu.obj" "C:\cuda\cuda\cuda2.cu"

 

To work around this, you need to redefine the value in the converted targets file as follows:

 

<ItemGroup>

  <CUDA_Build_Rule Condition="'%(CUDA_Build_Rule.NvccCompilation)' == '0'">

    <NvccCompilation>-c -o &quot;$(IntDir)\%(Filename).cu.obj&quot;</NvccCompilation>

  </CUDA_Build_Rule>

  <CUDA_Build_Rule Condition="'%(CUDA_Build_Rule.NvccCompilation)' == '1'">

    <NvccCompilation>-cuda -o &quot;$(IntDir)\%(Filename).cu.c&quot;</NvccCompilation>

  </ItemGroup>

 

You can take a look of this post on the modified converted CUDA rule files.

MSBuild evaluation sequences

MsBuild evaluates properties values in sequential order or early evaluation, versus late evaluation in the old VCBuild system. Here is an example of early evaluation: if you have property a= property b; property b= property c; a, however, is not necessary equal to c, as MSBuid will not go back to reevaluate the value of a.

 

As a result of early evaluation, some manual modification of the converted rule files may be needed in certain circumstances. Take YASM rule for example, if your object files are not in the default $(IntDir)\%(Filename).obj defined by the rule file,, but $(IntDir)mpn\%(Filename).obj, you may need to change “Outputs” to “ObjectFileName” in the targets file for custom build rules. For example, in YASM.props, you have this defined:

 

<ObjectFileName>$(IntDir)%(FileName).obj</ObjectFileName>

<PreProc>0</PreProc>

<Parser>0</Parser>

<CommandLineTemplate>yasm -Xvc -f $(Platform) [AllOptions] [AdditionalOptions] [Inputs]</CommandLineTemplate>

<Outputs>%(ObjectFileName)</Outputs>

 

As a result, “Outputs” value is “$(IntDir)%(FileName).obj”.

 

This YASM.props is imported into project file. After the import, the project defines the value for ObjectFileName as such:

 

<YASM>

  <Defines>_WIN64_ABI</Defines>

  <IncludePaths>$(ProjectDir)\</IncludePaths>

  <ObjectFileName>$(IntDir)mpn\%(Filename).obj</ObjectFileName>

  <Debug>true</Debug>

</YASM>

 

At this time, the value of “ObjectFileName”, which is “$(IntDir)mpn\%(Filename).obj” is out of sync with the value of “Outputs”, which still has the value of “$(IntDir)%(FileName).obj”.

 

This is what you have in the targets file after conversion:

 

<Target

    Name="_YASM"

    BeforeTargets="$(YASMBeforeTargets)"

    AfterTargets="$(YASMAfterTargets)"

    Condition="'@(YASM)' != ''"

    DependsOnTargets="$(YASMDependsOn);ComputeYASMOutput"

    Outputs="@(YASM-&gt;Metadata('Outputs')-&gt;Distinct())"

    Inputs="@(YASM);%(YASM.AdditionalDependencies);$(MSBuildProjectFile)">

  <ItemGroup

    Condition="'@(SelectedFiles)' != ''">

    <YASM

      Remove="@(YASM)"

      Condition="'%(Identity)' != '@(SelectedFiles)'" />

  </ItemGroup>

  <ItemGroup>

    <YASM_tlog

      Include="%(YASM.Outputs)"

      Condition="'%(YASM.Outputs)' != '' and '%(YASM.ExcludedFromBuild)' != 'true'">

      <Source>@(YASM, '|')</Source>

    </YASM_tlog>

  </ItemGroup>

 

Since Outputs="@(YASM-&gt;'%(Outputs)')" are what are being passed to the linker, as a result, nothing generated from YASM will be passed. The fix is to change “Outputs” to “ObjectFileName”.

 

<Target

      Name="_YASM"

      BeforeTargets="$(YASMBeforeTargets)"

      AfterTargets="$(YASMAfterTargets)"

      Condition="'@(YASM)' != ''"

      DependsOnTargets="$(YASMDependsOn);ComputeYASMOutput"

      Outputs="@(YASM-&gt;Metadata('ObjectFileName') -&gt;Distinct())"

      Inputs="@(YASM);%(YASM.AdditionalDependencies);$(MSBuildProjectFile)">

  <ItemGroup

   Condition="'@(SelectedFiles)' != ''">

    <YASM

       Remove="@(YASM)"

       Condition="'%(Identity)' != '@(SelectedFiles)'" />

  </ItemGroup>

  <ItemGroup>

    <YASM_tlog

       Include="%(YASM.ObjectFileName)"

       Condition="'%(YASM.ObjectFileName)' != '' and '%(YASM.ExcludedFromBuild)' != 'true'">

      <Source>@(YASM, '|')</Source>

    </YASM_tlog>

  </ItemGroup>

 

Please take a look of this connect bug on the version of custom build rules that can work with YASM tools.

 

You can also find a version of VS2010 custom build rule here that can work with the version of YASM executable (VSYASM) that supports batching.

Escape characters

Another limitation is that quotes around files are not properly handled. You may get errors like the following:

 

                error MSB4023: Cannot evaluate the item metadata "%(Extension)". The item metadata "%(Extension)" cannot be applied to the path ""x64\Release\add_err1_n.obj"". Illegal characters in path.

 

You will need to manually remove the quotes in the project file and/or .props file.

 

No UI support to create Custom Build Rule

One major change is that there is no more UI support in VS2010 to help you create custom build rule, although all your previous custom build rules from VS2005 and VS2008 can be converted to the VS2010 format. We are investigating ways to ease the inconvenience of the lack of UI support. The purpose of the blog is also to help you understand some of the underlying design so that you can make some small changes if needed.

  • Do we have to place our custom rules in that directory or can they live with the project?

  • @ JK: Your custom build rule can live with the project or any directory. You can use the Custom Build Rule UI to navigate to the directory where the custom build rule is, select the .targets file and add the custom build rule to your project.

    Li Shao, C++ Project and Build Team

  • What if the tool your custom build tool produces a C/C++ file as output, which you then want to include with compilation of other C/C++ files, and then obviously have that output linked into the main executable? For instance, Bison and Flex both produce C files as output. I want to have this output compiled by VC++ with other C/C++ files, and finally, have everything linked together. How do you compute the ouput for this scenario? (e.g. I have a target called ComputeBisonOutputs in one rule and ComputeFlexOutputs in another, but I don't know how to "pass the information on" to the rest of the build process.)

  • Do you have the original Rule files from previous version? As a first try, you can go through the conversion process and see if it can give you what you want.

    Li Shao, MSFT

  • It works for me after conversion (also generating cpp/h files). Only batch processing was lost, and I'm a bit clueless about adding it back.

  • You can send email to LISHAO AT Microsoft DOT com and we can discuss off line.

    Li Shao, C++ Project and Build Team

  • Figured out my other question, but have another.

    I have the following:

    <BoolProperty

     Name="SpecifyOutputFileName"

     Switch="-o">

     <Argument Property="OutputFileName" IsRequired="false" />

    </BoolProperty>

    <StringProperty

     Name="OutputFileName"

     Subtype="file"

    />

    Now, the utility requires the switch to be in this format: -o[filename]. How can I get MSBuild to do that. Looking at your XML file for CL, your pre-compiled header and XML Documentation properties both do that (e.g. /doc[filename] or /Yc[filename]), but when my rule runs, I get the following: -o [filename].

    Your help would be appreciated.

  • I had a custom build rule

    CommandLine="[VQShaderCmpDir]\vqshadercmp.exe -rp [Inputs] [OutputCmpXml] [OutputShaderFile] [InputsPP]"

    where

    Name="InputsPP"

    DisplayName="Preprocessed shaders"

    Description="Preprocessed shaders (.fxpp)"

    Switch="[value]"

    Delimited="true"

    But in VS2010 it places [InputsPP] on the command line as "item1;item2;item3" instead of "item1 item2 item3" as it did in 2008.  How do I fix this?

  • @Craig

    If you have the original rule file, conversion should generate what you expect. In the XML file that is generated, you should see something like Switch="-o&quot;[value]&quot;". Please let me know if this is not the case.

    Li Shao, C++ project and Build Team

  • @ JK

    In the XML file that is generated after conversion, you will find something like

    <StringListProperty

         Name="Inputs"

         Category="Command Line"

         IsRequired="true"

         Switch=" ;">

    You can remove the ";" in the "Switch" property. Please let me know if this does not fix your problem.

    Li Shao, C++ Project and Build Team

  • @ Li Shao

    There is no Switch=" ;" for either Inputs or InputsPP.  I sent you the rules and the generated files.

  • @Li Shao

    This is what the conversion is:

    <StringListProperty

         Name="InputsPP"

         HelpContext="0"

         DisplayName="Preprocessed shaders"

         Description="Preprocessed shaders (.fxpp)"

         Separator=";"

         Switch="[value]" />

  • @ JK

    Please try to change the separator to " ".

    Li Shao, C++ Project and Build Team

  • @Li Shao:

    Thanks that helped but I'm still having trouble with the dependencies between custom build rules that worked fine in 2008.

  • @JK

    Please send me the additional information that I requested through email and I will take a look.

    Li Shao, C++ Project and Build Team

Page 1 of 3 (40 items) 123