Quick Help on VS2010 Custom Build Rule

Quick Help on VS2010 Custom Build Rule

Rate This
  • Comments 40

Hi my name is Li Shao. I am a Software Design Engineer in Test for the C++ team. In my earlier blog “Visual Studio 2010 C++ Project Upgrade Guide”, I have mentioned about the change of Custom build Rules in VS2010. In this post, I would like to explain in more detail of the new format of the Custom Build Rules and some of its limitations in VS2010.

Custom build rule is a build feature introduced in VS2005. It provides the ability for the users to easily Plug-In third party tools to use in their build process. The custom build rule is defined by “.rules” files. Rules for MASM (Microsoft Micro Assembler) and LIC (License Compiler) are shipped in the box for VS2005 and VS2008.  

In VS2010, due to the migration to MSBuild, the information in the rules file is represented by three files: .XML, .props and .targets files. The XML file defines the property and switch mapping. The props file initializes the property values and the targets file generates the task based on the XML file and has the build logic. MASM and LIC custom build rules are also shipped in VS2010 with this new format. In the next couple of paragraphs, I will use MASM rule as an example to explain some important parts of the new custom build rule format and hope to give you some ideas on how to read custom build rule files if you need to.

 MASM custom build rule is available at %ProgramFiles%\MSBuild\Microsoft.Cpp\v4.0\BuildCustomizations or %ProgramFiles(x86)%\MSBuild\Microsoft.Cpp\v4.0\BuildCustomizations on x64 machine. You have the following in the MASM.XML file. 

<BoolProperty

      Name="GenerateDebugInformation"

      HelpUrl="http://msdn.microsoft.com/library/default.asp?url=/library/en-us/vcmasm/html/vclrfml.asp"

      DisplayName="Generate Debug Information"

      Description="Generates Debug Information.     (/Zi)"

      Switch="/Zi" />

 

With this XML file definition, “GenerateDebugInformation” is mapped to /Zi

In the file MASM.Props, “GenerateDebugInformation” is initialized to “True” (see highlighted line below)

<ItemDefinitionGroup>

  <MASM>

    <NoLogo>true</NoLogo>

    <GenerateDebugInformation>true</GenerateDebugInformation>

    <ObjectFileName>$(IntDir)%(FileName).obj</ObjectFileName>

    <PreserveIdentifierCase>0</PreserveIdentifierCase>

    <WarningLevel>3</WarningLevel>

    <PackAlignmentBoundary>0</PackAlignmentBoundary>

    <CallingConvention>0</CallingConvention>

    <ErrorReporting>0</ErrorReporting>

    <CommandLineTemplate Condition="'$(Platform)' == 'Win32'">ml.exe /c [AllOptions] [AdditionalOptions] /Ta[Inputs]</CommandLineTemplate>

    <CommandLineTemplate Condition="'$(Platform)' == 'X64'">ml64.exe /c [AllOptions] [AdditionalOptions] /Ta[Inputs]</CommandLineTemplate>

    <CommandLineTemplate Condition="'$(Platform)' != 'Win32' and '$(Platform)' != 'X64'">echo MASM not supported on this platform</CommandLineTemplate>

    <ExecutionDescription>Assembling [Inputs]...</ExecutionDescription>

  </MASM>

</ItemDefinitionGroup>

 

The MASM.XML file is imported by the MASM.targets file. MSBuild relies on the Task Generator to generate the task for the custom build rule.

<UsingTask

    TaskName="MASM"

    TaskFactory="XamlTaskFactory"

    AssemblyName="Microsoft.Build.Tasks.v4.0">

  <Task>$(MSBuildThisFileDirectory)$(MSBuildThisFileName).xml</Task>

</UsingTask>

 

The values initialized in MASM.props file is also passed to the targets:

<MASM

      GenerateDebugInformation="%(MASM.GenerateDebugInformation)"

      Inputs="%(MASM.Identity)" />

 

The build logic is in the .targets file as well. For example, these lines define the inputs and outputs for MASM:

Outputs="%(MASM.ObjectFileName)"

Inputs="%(MASM.Identity);%(MASM.AdditionalDependencies);$(MSBuildProjectFile)"

This defines what outputs from MASM need to be passed to Linker/Lib:

<Target

    Name="ComputeMASMOutput"

    Condition="'@(MASM)' != ''">

  <ItemGroup>

    <Link Include="@(MASM->Metadata('ObjectFileName')->Distinct()->ClearMetadata())" Condition="'%(MASM.ExcludedFromBuild)' != 'true'"/>

    <Lib Include="@(MASM->Metadata('ObjectFileName')->Distinct()->ClearMetadata())" Condition="'%(MASM.ExcludedFromBuild)' != 'true'"/>

  </ItemGroup>

</Target >

 

MASM.props and MASM.targets file are imported by project files (.vcxproj) if the custom build rule for MASM is enabled.

<ImportGroup Label="ExtensionSettings">

    <Import Project="$(VCTargetsPath)\BuildCustomizations\masm.props" />

</ImportGroup>

 

<ImportGroup Label="ExtensionTargets">

    <Import Project="$(VCTargetsPath)\BuildCustomizations\masm.targets" />

</ImportGroup>

Limitations and workarounds

VS2010 supports automatic conversion of the .rules file to .xml, .props and .targets files. The new build rules, in general, can be built without any modification. However, there are a few limitations that may require additional modification to the rule files. I will use custom build rules comes with CUDA tool kit and custom build rules that work with YASM tool as examples.

Environment variables as part of the switches

One major limitation is that MSBuild does not support expanding environment variables if they are part of the switches when generating tasks. Take the CUDA rule, for example. You have the following in your original rule file:

<EnumProperty

     Name="NvccCompilation"

     DisplayName="NVCC Compilation Type"

     Description="Select desired output of NVCC compilation (-c/-compile, -cuda, -gpu, -cubin, -ptx)"

     >

  <Values>

    <EnumValue

     Value="0"

     Switch="--compile -o &quot;$(IntDir)\$(InputName).cu.obj&quot;"

     DisplayName="Generate hybrid object file  (--compile / -c)"

      />

    <EnumValue

     Value="1"

     Switch="-cuda -o &quot;$(IntDir)\$(InputName).cu.c&quot;"

     DisplayName="Generate hybrid .c file  (-cuda)"

      />

    />

 

They will be converted into:

<EnumProperty

      Name="NvccCompilation"

      HelpContext="0"

      DisplayName="NVCC Compilation Type"

      Description="Select desired output of NVCC compilation (-c/-compile, -cuda, -gpu, -cubin, -ptx)">

  <EnumValue

    Name="0"

    DisplayName="Generate hybrid object file  (--compile / -c)"

    Switch="--compile -o &quot;$(IntDir)\$(InputName).cu.obj&quot;" />

  <EnumValue

    Name="1"

    DisplayName="Generate hybrid .c file  (-cuda)"

    Switch="-cuda -o &quot;$(IntDir)\$(InputName).cu.c&quot;" />

</EnumProperty>

 

The $(IntDir) and $(IntputName), however, will not be expanded during compile due to the limitation of the task generated by the task generator. As a result, you will see the following command line that $(IntDir) and $(InputName) are not expanded.

-arch sm_10 -ccbin "C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\bin"    -Xcompiler "/EHsc /W3 /nologo /O2 /Zi   /MT  " -I -maxrregcount=32  --compile -o "$(IntDir)\$(InputName).cu.obj" "C:\cuda\cuda\cuda2.cu"

 

To work around this, you need to redefine the value in the converted targets file as follows:

 

<ItemGroup>

  <CUDA_Build_Rule Condition="'%(CUDA_Build_Rule.NvccCompilation)' == '0'">

    <NvccCompilation>-c -o &quot;$(IntDir)\%(Filename).cu.obj&quot;</NvccCompilation>

  </CUDA_Build_Rule>

  <CUDA_Build_Rule Condition="'%(CUDA_Build_Rule.NvccCompilation)' == '1'">

    <NvccCompilation>-cuda -o &quot;$(IntDir)\%(Filename).cu.c&quot;</NvccCompilation>

  </ItemGroup>

 

You can take a look of this post on the modified converted CUDA rule files.

MSBuild evaluation sequences

MsBuild evaluates properties values in sequential order or early evaluation, versus late evaluation in the old VCBuild system. Here is an example of early evaluation: if you have property a= property b; property b= property c; a, however, is not necessary equal to c, as MSBuid will not go back to reevaluate the value of a.

 

As a result of early evaluation, some manual modification of the converted rule files may be needed in certain circumstances. Take YASM rule for example, if your object files are not in the default $(IntDir)\%(Filename).obj defined by the rule file,, but $(IntDir)mpn\%(Filename).obj, you may need to change “Outputs” to “ObjectFileName” in the targets file for custom build rules. For example, in YASM.props, you have this defined:

 

<ObjectFileName>$(IntDir)%(FileName).obj</ObjectFileName>

<PreProc>0</PreProc>

<Parser>0</Parser>

<CommandLineTemplate>yasm -Xvc -f $(Platform) [AllOptions] [AdditionalOptions] [Inputs]</CommandLineTemplate>

<Outputs>%(ObjectFileName)</Outputs>

 

As a result, “Outputs” value is “$(IntDir)%(FileName).obj”.

 

This YASM.props is imported into project file. After the import, the project defines the value for ObjectFileName as such:

 

<YASM>

  <Defines>_WIN64_ABI</Defines>

  <IncludePaths>$(ProjectDir)\</IncludePaths>

  <ObjectFileName>$(IntDir)mpn\%(Filename).obj</ObjectFileName>

  <Debug>true</Debug>

</YASM>

 

At this time, the value of “ObjectFileName”, which is “$(IntDir)mpn\%(Filename).obj” is out of sync with the value of “Outputs”, which still has the value of “$(IntDir)%(FileName).obj”.

 

This is what you have in the targets file after conversion:

 

<Target

    Name="_YASM"

    BeforeTargets="$(YASMBeforeTargets)"

    AfterTargets="$(YASMAfterTargets)"

    Condition="'@(YASM)' != ''"

    DependsOnTargets="$(YASMDependsOn);ComputeYASMOutput"

    Outputs="@(YASM-&gt;Metadata('Outputs')-&gt;Distinct())"

    Inputs="@(YASM);%(YASM.AdditionalDependencies);$(MSBuildProjectFile)">

  <ItemGroup

    Condition="'@(SelectedFiles)' != ''">

    <YASM

      Remove="@(YASM)"

      Condition="'%(Identity)' != '@(SelectedFiles)'" />

  </ItemGroup>

  <ItemGroup>

    <YASM_tlog

      Include="%(YASM.Outputs)"

      Condition="'%(YASM.Outputs)' != '' and '%(YASM.ExcludedFromBuild)' != 'true'">

      <Source>@(YASM, '|')</Source>

    </YASM_tlog>

  </ItemGroup>

 

Since Outputs="@(YASM-&gt;'%(Outputs)')" are what are being passed to the linker, as a result, nothing generated from YASM will be passed. The fix is to change “Outputs” to “ObjectFileName”.

 

<Target

      Name="_YASM"

      BeforeTargets="$(YASMBeforeTargets)"

      AfterTargets="$(YASMAfterTargets)"

      Condition="'@(YASM)' != ''"

      DependsOnTargets="$(YASMDependsOn);ComputeYASMOutput"

      Outputs="@(YASM-&gt;Metadata('ObjectFileName') -&gt;Distinct())"

      Inputs="@(YASM);%(YASM.AdditionalDependencies);$(MSBuildProjectFile)">

  <ItemGroup

   Condition="'@(SelectedFiles)' != ''">

    <YASM

       Remove="@(YASM)"

       Condition="'%(Identity)' != '@(SelectedFiles)'" />

  </ItemGroup>

  <ItemGroup>

    <YASM_tlog

       Include="%(YASM.ObjectFileName)"

       Condition="'%(YASM.ObjectFileName)' != '' and '%(YASM.ExcludedFromBuild)' != 'true'">

      <Source>@(YASM, '|')</Source>

    </YASM_tlog>

  </ItemGroup>

 

Please take a look of this connect bug on the version of custom build rules that can work with YASM tools.

 

You can also find a version of VS2010 custom build rule here that can work with the version of YASM executable (VSYASM) that supports batching.

Escape characters

Another limitation is that quotes around files are not properly handled. You may get errors like the following:

 

                error MSB4023: Cannot evaluate the item metadata "%(Extension)". The item metadata "%(Extension)" cannot be applied to the path ""x64\Release\add_err1_n.obj"". Illegal characters in path.

 

You will need to manually remove the quotes in the project file and/or .props file.

 

No UI support to create Custom Build Rule

One major change is that there is no more UI support in VS2010 to help you create custom build rule, although all your previous custom build rules from VS2005 and VS2008 can be converted to the VS2010 format. We are investigating ways to ease the inconvenience of the lack of UI support. The purpose of the blog is also to help you understand some of the underlying design so that you can make some small changes if needed.

  • Thanks for answering my questions with questions. You were absolutely no help to me. Here I am trying to learn how MSBuild works so I can create more rules in the future that are even more advanced; but since no information is being divulged and there is hardly any documentation on the schemas of MSBuild 4.0, the whole system is kind of useless.

  • @ Craig, based on your response, it looks like you are trying to create some XML files based on the syntax that we have been using for other tools, such as CL, to generate the task on the fly. Therotically, this is supported. However, due to resource constrain, we did not fix all the issues associated with certain syntax. The example you had with "Argument" is one of the cases that did not get fixed and we had an internal bug on this. If you like, you can also create a connect bug so to raise the priority of the issue.

    I totally agree with you that making this system robust can help people create more advanced tasks.

    Li Shao, C++ Project and Build Team

  • I have a custom build step which simply copies the compiled .dll and .lib file to another folder.  This worked great in VS2005 and VS2008 however now in VS2010, I get a 'LNK1181: cannot open input file' error.

    If I remove the copy command for the .lib file and only copy the .dll file, it compiles without any problems.

    Any tips as to what I'm doing wrong here?  Thanks.

    As an example, I used the DLLScreenCap VC++ sample project.  Then added the following to the Custom Build Step,

    Command Line:

    echo copy .\$(Configuration)\$(TargetName).dll ..\Publish\$(Configuration)\$(TargetName).dll

    copy .\$(Configuration)\$(TargetName).dll ..\Publish\$(Configuration)\$(TargetName).dll

    echo copy .\$(Configuration)\$(TargetName).lib ..\Publish\$(Configuration)\$(TargetName).lib

    copy .\$(Configuration)\$(TargetName).lib ..\Publish\$(Configuration)\$(TargetName).lib

    Outputs:

    ..\Publish\$(Configuration)\$(TargetName).dll

    ..\Publish\$(Configuration)\$(TargetName).lib

    Build log:

    1>------ Rebuild All started: Project: DllScreenCap, Configuration: Release Win32 ------

    1>Build started 29/04/2010 6:31:59 PM.

    1>_PrepareForClean:

    1>  Deleting file "Release\DllScreenCap.lastbuildstate".

    1>InitializeBuildStatus:

    1>  Touching "Release\DllScreenCap.unsuccessfulbuild".

    1>ClCompile:

    1>  stdafx.cpp

    1>  _WIN32_WINNT not defined. Defaulting to _WIN32_WINNT_MAXVER (see WinSDKVer.h)

    1>  ConfigDlg.cpp

    1>  _WIN32_WINNT not defined. Defaulting to _WIN32_WINNT_MAXVER (see WinSDKVer.h)

    1>  DllScreenCap.cpp

    1>  _WIN32_WINNT not defined. Defaulting to _WIN32_WINNT_MAXVER (see WinSDKVer.h)

    1>  Exports.cpp

    1>  _WIN32_WINNT not defined. Defaulting to _WIN32_WINNT_MAXVER (see WinSDKVer.h)

    1>  Generating Code...

    1>ComputeCustomBuildOutput:

    1>  Creating directory "C:\DllScreenCap\Publish\Release\".

    1>LINK : fatal error LNK1181: cannot open input file '..\Publish\Release\DllScreenCap.lib'

    1>

    1>Build FAILED.

    1>

    1>Time Elapsed 00:00:16.09

    ========== Rebuild All: 0 succeeded, 1 failed, 0 skipped ==========

  • @GC As a first check, did you find DllScreenCap.lib in ..\Publish\Release directory?

    Li Shao, C++ Project and Build Team

  • @GC. The reason this is happening is because in VS2008/VS2005 the custom build rules would create the \Publish\Release directory if it did not exist on disk. However in VS2010 we don't create those directories for custom build rules. To resolve the issue please make sure the directories where the files are to be created exist on disk.

    Thanks,

    Amit Mohindra

    Visual C++ Team

  • @Li Shao, No the lib file never made it to the ..\Publish\Release directory.  If I only have the command to copy the DLL, the DLL is copied correctly.

    @Amit Mohindra, Strange that you say that because if I only have the command to copy the DLL, VS2010 happily creates the directory, copies the DLL without any errors.

    I had another try and made sure the ..\Publish\Release directory existed before compiling however I still get the LNK1181 error when trying to copy the lib file.

    Thanks for your quick responses.

  • @GC In which directory is the .lib file generated? It may not be in the source directory as you expected $(Configuration)\$(TargetName).lib.

    You can take a look of this blog on some changes regarding OutputPath: http://blogs.msdn.com/vcblog/archive/2010/03/02/visual-studio-2010-c-project-upgrade-guide.aspx

    -If your project produces Import Library (Linker -> Advanced -> Import Library), you may need to change the Output folder of the Import Library as well after conversion if the Linker output directory is not the default output directory. Otherwise, the generated import lib maybe in a different directory than the linker output.

    Li Shao, Project and Build Team

  • @Li Shao, the (Linker -> Advanced -> Import Library) value is $(OutDir)DllScreenCap.lib which is equivalent to Release\DllScreenCap.lib

    $(Configuration)\$(TargetName).lib is also the same (Release\DllScreenCap.lib) for a Release build.  So as far as I can tell, it should have worked.

    Would you mind downloading the project and trying it at your end?  DllScreenCap can be found in http://code.msdn.microsoft.com/vcsamplesmfc

    Thanks for your time.

  • @Li Shao, the (Linker -> Advanced -> Import Library) value is $(OutDir)DllScreenCap.lib which is equivalent to Release\DllScreenCap.lib

    $(Configuration)\$(TargetName).lib is also the same (Release\DllScreenCap.lib) for a Release build.  So as far as I can tell, it should have worked.

    Would you mind downloading the project and trying it at your end?  DllScreenCap can be found in http://code.msdn.microsoft.com/vcsamplesmfc

    Thanks for your time.

  • @GC Thank you for the additional information. I was able to repro your problem. It looks like this is a bug on our side. We are trying to pass the custom build step output that we think is relevant to the linker command line, and thus the failure. In general, this is fine and it is the right thing to do. In this case, it happens that custom build build output is a lib file that is being generated by the linker. Please open a connect bug on it.

    To workaround the problem, please use "Post Build Step". You can use the following command to copy the files:

    copy $(OutDir)\$(TargetName).dll $(SolutionDir)\Publish\$(Configuration)\$(TargetName).dll

    copy $(OutDir)\$(TargetName).lib $(SolutionDir)\Publish\$(Configuration)\$(TargetName).lib

    Li Shao, C++ Project and Build Team

  • @Li Shao, kudos for following through so detaield with this case.

    Still, the limitations seem a big step backwards, e.g. I've relied a lot on the lazy evaluation in the past.

  • @peterchen Thank you for your feedback. I hope using "post build event" can help you workaround the problem.

    Li Shao, C++ Project and Build Team

  • @Li Shao, thanks for spending your time on this.  As requested, I have created a Connect bug report.

    https://connect.microsoft.com/VisualStudio/feedback/details/557340/vc-in-vs2010-fails-if-there-is-a-command-to-copy-lib-file-in-custom-build-step

  • Thanks GC for reporting the issue to us!

  • I have a rule file that I used to generate message files (via MC) in VC2005/2008 and that works OK. However when I upgraded to VS2010 the conversion failed, but gave no clue as to why. I have tried to convert it manually using MASM and LC as templates but the compile phase does nothing. Any suggestions as to how I can convert or otherwise create a valid props/targets/xml group for what was a very simple file?

Page 2 of 3 (40 items) 123