Quick Help on VS2010 Custom Build Rule

Quick Help on VS2010 Custom Build Rule

Rate This
  • Comments 40

Hi my name is Li Shao. I am a Software Design Engineer in Test for the C++ team. In my earlier blog “Visual Studio 2010 C++ Project Upgrade Guide”, I have mentioned about the change of Custom build Rules in VS2010. In this post, I would like to explain in more detail of the new format of the Custom Build Rules and some of its limitations in VS2010.

Custom build rule is a build feature introduced in VS2005. It provides the ability for the users to easily Plug-In third party tools to use in their build process. The custom build rule is defined by “.rules” files. Rules for MASM (Microsoft Micro Assembler) and LIC (License Compiler) are shipped in the box for VS2005 and VS2008.  

In VS2010, due to the migration to MSBuild, the information in the rules file is represented by three files: .XML, .props and .targets files. The XML file defines the property and switch mapping. The props file initializes the property values and the targets file generates the task based on the XML file and has the build logic. MASM and LIC custom build rules are also shipped in VS2010 with this new format. In the next couple of paragraphs, I will use MASM rule as an example to explain some important parts of the new custom build rule format and hope to give you some ideas on how to read custom build rule files if you need to.

 MASM custom build rule is available at %ProgramFiles%\MSBuild\Microsoft.Cpp\v4.0\BuildCustomizations or %ProgramFiles(x86)%\MSBuild\Microsoft.Cpp\v4.0\BuildCustomizations on x64 machine. You have the following in the MASM.XML file. 

<BoolProperty

      Name="GenerateDebugInformation"

      HelpUrl="http://msdn.microsoft.com/library/default.asp?url=/library/en-us/vcmasm/html/vclrfml.asp"

      DisplayName="Generate Debug Information"

      Description="Generates Debug Information.     (/Zi)"

      Switch="/Zi" />

 

With this XML file definition, “GenerateDebugInformation” is mapped to /Zi

In the file MASM.Props, “GenerateDebugInformation” is initialized to “True” (see highlighted line below)

<ItemDefinitionGroup>

  <MASM>

    <NoLogo>true</NoLogo>

    <GenerateDebugInformation>true</GenerateDebugInformation>

    <ObjectFileName>$(IntDir)%(FileName).obj</ObjectFileName>

    <PreserveIdentifierCase>0</PreserveIdentifierCase>

    <WarningLevel>3</WarningLevel>

    <PackAlignmentBoundary>0</PackAlignmentBoundary>

    <CallingConvention>0</CallingConvention>

    <ErrorReporting>0</ErrorReporting>

    <CommandLineTemplate Condition="'$(Platform)' == 'Win32'">ml.exe /c [AllOptions] [AdditionalOptions] /Ta[Inputs]</CommandLineTemplate>

    <CommandLineTemplate Condition="'$(Platform)' == 'X64'">ml64.exe /c [AllOptions] [AdditionalOptions] /Ta[Inputs]</CommandLineTemplate>

    <CommandLineTemplate Condition="'$(Platform)' != 'Win32' and '$(Platform)' != 'X64'">echo MASM not supported on this platform</CommandLineTemplate>

    <ExecutionDescription>Assembling [Inputs]...</ExecutionDescription>

  </MASM>

</ItemDefinitionGroup>

 

The MASM.XML file is imported by the MASM.targets file. MSBuild relies on the Task Generator to generate the task for the custom build rule.

<UsingTask

    TaskName="MASM"

    TaskFactory="XamlTaskFactory"

    AssemblyName="Microsoft.Build.Tasks.v4.0">

  <Task>$(MSBuildThisFileDirectory)$(MSBuildThisFileName).xml</Task>

</UsingTask>

 

The values initialized in MASM.props file is also passed to the targets:

<MASM

      GenerateDebugInformation="%(MASM.GenerateDebugInformation)"

      Inputs="%(MASM.Identity)" />

 

The build logic is in the .targets file as well. For example, these lines define the inputs and outputs for MASM:

Outputs="%(MASM.ObjectFileName)"

Inputs="%(MASM.Identity);%(MASM.AdditionalDependencies);$(MSBuildProjectFile)"

This defines what outputs from MASM need to be passed to Linker/Lib:

<Target

    Name="ComputeMASMOutput"

    Condition="'@(MASM)' != ''">

  <ItemGroup>

    <Link Include="@(MASM->Metadata('ObjectFileName')->Distinct()->ClearMetadata())" Condition="'%(MASM.ExcludedFromBuild)' != 'true'"/>

    <Lib Include="@(MASM->Metadata('ObjectFileName')->Distinct()->ClearMetadata())" Condition="'%(MASM.ExcludedFromBuild)' != 'true'"/>

  </ItemGroup>

</Target >

 

MASM.props and MASM.targets file are imported by project files (.vcxproj) if the custom build rule for MASM is enabled.

<ImportGroup Label="ExtensionSettings">

    <Import Project="$(VCTargetsPath)\BuildCustomizations\masm.props" />

</ImportGroup>

 

<ImportGroup Label="ExtensionTargets">

    <Import Project="$(VCTargetsPath)\BuildCustomizations\masm.targets" />

</ImportGroup>

Limitations and workarounds

VS2010 supports automatic conversion of the .rules file to .xml, .props and .targets files. The new build rules, in general, can be built without any modification. However, there are a few limitations that may require additional modification to the rule files. I will use custom build rules comes with CUDA tool kit and custom build rules that work with YASM tool as examples.

Environment variables as part of the switches

One major limitation is that MSBuild does not support expanding environment variables if they are part of the switches when generating tasks. Take the CUDA rule, for example. You have the following in your original rule file:

<EnumProperty

     Name="NvccCompilation"

     DisplayName="NVCC Compilation Type"

     Description="Select desired output of NVCC compilation (-c/-compile, -cuda, -gpu, -cubin, -ptx)"

     >

  <Values>

    <EnumValue

     Value="0"

     Switch="--compile -o &quot;$(IntDir)\$(InputName).cu.obj&quot;"

     DisplayName="Generate hybrid object file  (--compile / -c)"

      />

    <EnumValue

     Value="1"

     Switch="-cuda -o &quot;$(IntDir)\$(InputName).cu.c&quot;"

     DisplayName="Generate hybrid .c file  (-cuda)"

      />

    />

 

They will be converted into:

<EnumProperty

      Name="NvccCompilation"

      HelpContext="0"

      DisplayName="NVCC Compilation Type"

      Description="Select desired output of NVCC compilation (-c/-compile, -cuda, -gpu, -cubin, -ptx)">

  <EnumValue

    Name="0"

    DisplayName="Generate hybrid object file  (--compile / -c)"

    Switch="--compile -o &quot;$(IntDir)\$(InputName).cu.obj&quot;" />

  <EnumValue

    Name="1"

    DisplayName="Generate hybrid .c file  (-cuda)"

    Switch="-cuda -o &quot;$(IntDir)\$(InputName).cu.c&quot;" />

</EnumProperty>

 

The $(IntDir) and $(IntputName), however, will not be expanded during compile due to the limitation of the task generated by the task generator. As a result, you will see the following command line that $(IntDir) and $(InputName) are not expanded.

-arch sm_10 -ccbin "C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\bin"    -Xcompiler "/EHsc /W3 /nologo /O2 /Zi   /MT  " -I -maxrregcount=32  --compile -o "$(IntDir)\$(InputName).cu.obj" "C:\cuda\cuda\cuda2.cu"

 

To work around this, you need to redefine the value in the converted targets file as follows:

 

<ItemGroup>

  <CUDA_Build_Rule Condition="'%(CUDA_Build_Rule.NvccCompilation)' == '0'">

    <NvccCompilation>-c -o &quot;$(IntDir)\%(Filename).cu.obj&quot;</NvccCompilation>

  </CUDA_Build_Rule>

  <CUDA_Build_Rule Condition="'%(CUDA_Build_Rule.NvccCompilation)' == '1'">

    <NvccCompilation>-cuda -o &quot;$(IntDir)\%(Filename).cu.c&quot;</NvccCompilation>

  </ItemGroup>

 

You can take a look of this post on the modified converted CUDA rule files.

MSBuild evaluation sequences

MsBuild evaluates properties values in sequential order or early evaluation, versus late evaluation in the old VCBuild system. Here is an example of early evaluation: if you have property a= property b; property b= property c; a, however, is not necessary equal to c, as MSBuid will not go back to reevaluate the value of a.

 

As a result of early evaluation, some manual modification of the converted rule files may be needed in certain circumstances. Take YASM rule for example, if your object files are not in the default $(IntDir)\%(Filename).obj defined by the rule file,, but $(IntDir)mpn\%(Filename).obj, you may need to change “Outputs” to “ObjectFileName” in the targets file for custom build rules. For example, in YASM.props, you have this defined:

 

<ObjectFileName>$(IntDir)%(FileName).obj</ObjectFileName>

<PreProc>0</PreProc>

<Parser>0</Parser>

<CommandLineTemplate>yasm -Xvc -f $(Platform) [AllOptions] [AdditionalOptions] [Inputs]</CommandLineTemplate>

<Outputs>%(ObjectFileName)</Outputs>

 

As a result, “Outputs” value is “$(IntDir)%(FileName).obj”.

 

This YASM.props is imported into project file. After the import, the project defines the value for ObjectFileName as such:

 

<YASM>

  <Defines>_WIN64_ABI</Defines>

  <IncludePaths>$(ProjectDir)\</IncludePaths>

  <ObjectFileName>$(IntDir)mpn\%(Filename).obj</ObjectFileName>

  <Debug>true</Debug>

</YASM>

 

At this time, the value of “ObjectFileName”, which is “$(IntDir)mpn\%(Filename).obj” is out of sync with the value of “Outputs”, which still has the value of “$(IntDir)%(FileName).obj”.

 

This is what you have in the targets file after conversion:

 

<Target

    Name="_YASM"

    BeforeTargets="$(YASMBeforeTargets)"

    AfterTargets="$(YASMAfterTargets)"

    Condition="'@(YASM)' != ''"

    DependsOnTargets="$(YASMDependsOn);ComputeYASMOutput"

    Outputs="@(YASM-&gt;Metadata('Outputs')-&gt;Distinct())"

    Inputs="@(YASM);%(YASM.AdditionalDependencies);$(MSBuildProjectFile)">

  <ItemGroup

    Condition="'@(SelectedFiles)' != ''">

    <YASM

      Remove="@(YASM)"

      Condition="'%(Identity)' != '@(SelectedFiles)'" />

  </ItemGroup>

  <ItemGroup>

    <YASM_tlog

      Include="%(YASM.Outputs)"

      Condition="'%(YASM.Outputs)' != '' and '%(YASM.ExcludedFromBuild)' != 'true'">

      <Source>@(YASM, '|')</Source>

    </YASM_tlog>

  </ItemGroup>

 

Since Outputs="@(YASM-&gt;'%(Outputs)')" are what are being passed to the linker, as a result, nothing generated from YASM will be passed. The fix is to change “Outputs” to “ObjectFileName”.

 

<Target

      Name="_YASM"

      BeforeTargets="$(YASMBeforeTargets)"

      AfterTargets="$(YASMAfterTargets)"

      Condition="'@(YASM)' != ''"

      DependsOnTargets="$(YASMDependsOn);ComputeYASMOutput"

      Outputs="@(YASM-&gt;Metadata('ObjectFileName') -&gt;Distinct())"

      Inputs="@(YASM);%(YASM.AdditionalDependencies);$(MSBuildProjectFile)">

  <ItemGroup

   Condition="'@(SelectedFiles)' != ''">

    <YASM

       Remove="@(YASM)"

       Condition="'%(Identity)' != '@(SelectedFiles)'" />

  </ItemGroup>

  <ItemGroup>

    <YASM_tlog

       Include="%(YASM.ObjectFileName)"

       Condition="'%(YASM.ObjectFileName)' != '' and '%(YASM.ExcludedFromBuild)' != 'true'">

      <Source>@(YASM, '|')</Source>

    </YASM_tlog>

  </ItemGroup>

 

Please take a look of this connect bug on the version of custom build rules that can work with YASM tools.

 

You can also find a version of VS2010 custom build rule here that can work with the version of YASM executable (VSYASM) that supports batching.

Escape characters

Another limitation is that quotes around files are not properly handled. You may get errors like the following:

 

                error MSB4023: Cannot evaluate the item metadata "%(Extension)". The item metadata "%(Extension)" cannot be applied to the path ""x64\Release\add_err1_n.obj"". Illegal characters in path.

 

You will need to manually remove the quotes in the project file and/or .props file.

 

No UI support to create Custom Build Rule

One major change is that there is no more UI support in VS2010 to help you create custom build rule, although all your previous custom build rules from VS2005 and VS2008 can be converted to the VS2010 format. We are investigating ways to ease the inconvenience of the lack of UI support. The purpose of the blog is also to help you understand some of the underlying design so that you can make some small changes if needed.

  • @Richard MacCutchan

    Could you send me a repro of your problem including the original rule file? I will be glad to take a look. My Email is LISHAO AT Microsoft Dot COM

    Li Shao, C++ Project and Build Team

  • Hi,

    I have a similar issue as described by Richard MacCutchan.

    Basically I have an .mc file which generates a header file with the following rule:

    mc.exe -v -c -s -x .\ -U "$(InputPath)"

    which works perfectly for VS2005 and VS2008

    But, when I converted the project to VS2010

    I now have the following rule in the custom build step of the mc file:

    mc.exe -v -c -s -x .\ -U "%(FullPath)"

    Now when I build this project, it fails with the following message:

    1>mc : error : 0x3 trying to open file <.\>.

    I then removed the -x option from the command line.It now builds successfully, but does not generate the header file. We are now unable to proceed with our conversion process. Please let me know if this is a bug and if there is any workaround. Thanks.

  • Thank you for reporting the issue. The custom build step is converted correctly as far as conversion is concerned. If you take a look of the  VS2008 and VS2010 MC command line that are generated by the custom build step, they are identical.  $(InoutPath) maps to %(FullPath) in VS2010 if it is set on the file level.

    I see the same error when running the command line in the VS2010 command prompt and no errors when running the same command line in VS2008 command prompt. So this seems to be a behavior change in MC tool itself. I will find out more about the MC change from the team who owns MC and get back to you.

    Li Shao, C++ Project and Build Team

  • @ dev Sorry for the delay in replying. The forum was undergoing reconstruction so I could not post my answer.

    I checked with the people who owns the MC tool. It looks like this is a design change for mc.exe in recent WinSDKs.

    Here is what it says in this MSDN link: msdn.microsoft.com/.../aa385638(VS.85).aspx:

    -spath

    Use this argument to create a baseline of your instrumentation. Specify the path to the folder that contains your baseline manifest files. For subsequent releases, you would then use the -t argument to check the new manifest against the baseline for compatibility issues.

    Prior to MC version 1.12.7051:  Not available

    -s option was simply ignored in MC.exe that came with WInSDK 6.0a in VS2008.

    In VS2010, to accommodate this mc change, you can fix the commandline by passing path to -s

    mc.exe -v -c –s .\ -x .\ -U <.mc>

    Li Shao, C++ Project and Build Team

  • this is horrible, I am not even able to read all this stuff, I gues I will tather live without custom rules...

  • @ plvle Yes, we understand not having the custom build rule UI can cause some inconvenience. What particular problems you have?

    Li Shao, Project and Build Team

  • I am facing the same issue.

    How can I do a custom mc.exe step without the -s option?

  • @Poiuy I think with the latest mc, you have to supply the -s option. This is required by the MC tool. You can set the path the same as the one that you pass to -x option.

    Li Shao, Project and Build Team

  • I can find no information about how to create a new custom build rule. How is this done? This seems quite important, and I can't find any information about how to do this. Do we have to edit all three files?

  • Unfortunately, there is no UI support to create custom build rule in VS2010. The decision was made due to resource constrain and the customer survey results indicated that custom build rule was not widely used. If you have a rule created with VS2008, VS2010 can convert it for you. What type of custom build rule you would like to create?

    Li Shao, MSFT

Page 3 of 3 (40 items) 123