A first hand look from the .NET engineering teams
Note: This post is now also part of the official NuGet documentation.
In this post, I’ll provide a walkthrough on how to use package restore with Team Foundation Build with both, git as well TF version control.
In our previous blog post on how we improved package restore, I showed that NuGet 2.7 and Microsoft.Bcl.Build made restoring packages a lot easier. However, I didn’t explain in great detail on how you can restore packages during the build. In this post, I’ll close that gap by providing a more detailed walkthrough.
Although this walkthrough is specific for the scenario of using Team Foundation Service, the concepts also apply to other version control- and build systems.
An advantage of using a package manager like NuGet is that you can use it to avoid checking in binaries to your version control system.
This is especially interesting if you are using a distributed version control system like git because developers need to clone the entire repository, including the full history, before they can start working locally. Checking in binaries can cause significant repository bloat as binary files are typically stored without delta compression.
NuGet has supported restoring packages as part of the build for a long time now. The previous implementation had a chicken-and-egg problem for packages that want to extend the build process because NuGet restored packages while building the project. However, MSBuild doesn’t allow extending the build during the build; one could argue that this an issue in MSBuild but I would argue that this an inherent problem. Depending on which aspect you need to extend it might be too late to register by the time your package is restored.
The cure to this problem is making sure that packages are restored as the first step in the build process. NuGet 2.7 makes this super easy via a simplified command line. In order to restore packages for an entire solution all you need is to execute a command line like this:
nuget.exe restore path\to\solution.sln
When your build process restores packages before building the code, you don’t need to check-in .targets files.
Note: Packages must be authored to allow loading in Visual Studio (our packages now all behave like this). Otherwise, you may still want to check in .targets files so that other developers can simply open the solution without having to restore packages first.
The following demo project shows how to set up the build in such a way that the packages folders and .targets files don’t need to be checked-in. Finally, I’ll show how you can setup an automated build on the Team Foundation Service for this sample project.
Our demo project is a simple command line tool that uses the command line argument to query Bing. It targets the .NET Framework 4 and uses many of our packages (Microsoft.Net.Http, Microsoft.Bcl, Microsoft.Bcl.Async, and Microsoft.Bcl.Build).
The structure of the repository looks as follows:
<Project> │ .gitignore │ .tfignore │ build.proj │ ├───src │ │ BingSearcher.sln │ │ │ └───BingSearcher │ │ App.config │ │ BingSearcher.csproj │ │ packages.config │ │ Program.cs │ │ │ └───Properties │ AssemblyInfo.cs │ └───tools └───NuGet NuGet.exe
You can see that we haven’t checked-in the packages folder nor any .targets files.
We have, however, checked-in the nuget.exe as it’s needed during the build. Following widely used conventions we’ve checked it in under a shared tools folder.
The source code is under the src folder. Although our demo only uses a single solution, you can easily imagine that this folder contains more than one solution.
In order to communicate to the version control that we don’t intent to check-in the packages folders, we’ve also added ignore files for both git (.gitignore) as well as TF version control (.tfignore). These files describes patterns of files you don’t want to check-in.
The .gitignore file looks as follows:
The .gitignore file is quite powerful. For example, if you want to generally not check-in the contents of the packages folder but want to go with our previous guidance of checking in the .targets files you could have the following rule instead:
This will exclude all packages folders but will re-include all contained .targets files. By the way, you can find a template for .gitignore files that is specifically tailored for the needs of Visual Studio developers here.
TF version control supports a very similar mechanism via the .tfignore file. The syntax is virtually the same:
For our demo, we keep the build process fairly simple. We’ll create an MSBuild project that builds all solutions while making sure that packages are restored before building the solutions.
This project will have the three conventional targets Clean, Build and Rebuild as well as a new target RestorePackages.
The result looks as follows:
<?xml version="1.0" encoding="utf-8"?><Project ToolsVersion="4.0" DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<PropertyGroup> <OutDir>$(MSBuildThisFileDirectory)bin</OutDir> <Configuration>Release</Configuration> <ProjectProperties> OutDir=$(OutDir); Configuration=$(Configuration); </ProjectProperties> </PropertyGroup>
<ItemGroup> <Solution Include="$(MSBuildThisFileDirectory)src\*.sln" /> </ItemGroup>
<Target Name="RestorePackages"> <Exec Command=""$(MSBuildThisFileDirectory)tools\NuGet\NuGet.exe" restore "%(Solution.Identity)"" /> </Target>
<Target Name="Clean"> <MSBuild Targets="Clean" Projects="@(Solution)" Properties="$(ProjectProperties)" /> </Target>
<Target Name="Build" DependsOnTargets="RestorePackages"> <MSBuild Targets="Build" Projects="@(Solution)" Properties="$(ProjectProperties)" /> </Target>
<Target Name="Rebuild" DependsOnTargets="RestorePackages"> <MSBuild Targets="Rebuild" Projects="@(Solution)" Properties="$(ProjectProperties)" /> </Target>
Team Build offers various process templates. For this demonstration, I’m using the Team Foundation Service. On premise installations of TFS will be very similar though.
Git and TF Version Control have different Team Build templates, so the following steps will vary depending on which version control system you are using. In both cases, all you need is selecting the build.proj as the project you want to build.
First, let’s look at the process template for git. In the git based template the build is selected via the property 1. Solution to build:
Please note that this property is a location in your repository. Since our build.proj is in the root, we simply used build.proj. If you place the build file under a folder called tools, the value would be tools\build.proj.
In the TF version control template the project is selected via the property 1. Projects:
In contrast to the git based template the TF version control supports pickers (the button on the right hand side with the three dots). So in order to avoid any typing errors I suggest you use them to select the project.
Using NuGet package restore is a great way to avoid checked-in libraries. In this walkthrough I’ve demonstrated how you can set up a repository structure and a build process for a project that avoids this.
Please let us know what you think!
Solution looks fine to me.
I just have a question: what does "packages must be authored" mean?
@Mickaël: Sorry for the confusion. What I meant was that package owners need to author their packages such that consuming projects can be loaded in VS even when packages are still missing. For example, our previous packages added unconditional imports to targets files. When packages are missing, those target files are missing which means that all the consuming projects have an import to a non existing file which in turn means that the projects will not load successfully in MSBuild/Visual Studio. For more details, see our (now outdated) post on this issue: blogs.msdn.com/.../nuget-package-restore-issues.aspx.
Does this make more sense?
We're having a problem with NuGet 2.7 in VS 2013/TFS 2013 automatically adding files to TFS.
I tried adding .tfignore to the solution root with the line "packages" in the file. However, when I delete the packages folder and allow VS to restore the packages on a rebuild, it still adds the "packages" folder and all of the packages underneath it as pending changes to TFS. Highly annoying.
We are using server workspaces (not local workspaces) so I don't know if .tfignore is even applicable. We want to get away from the "Enable NuGet Package Restore" since this new method is much cleaner, just have this issue we need to fix.
Do you have a solution? Thank you.
NuGet still supports the switch in the .nuget\nuget.config file for disabling source control integration of the packages folder.
Refer to docs.nuget.org/.../nuget-config-settings and apply the setting for disableSourceControlIntegration.
Your answer to Craig really doesn't help me. My TFS has been broken ever since i upgraded to 2.7, and my solution structure looks nothing like yours so I'd rather just disable it entirely as well. By default i already have in nuget.config <add key="disableSourceControlIntegration" value="true" /> . Changing this to true or false still gives me the same error on build:
The build restored NuGet packages. Build the project again to include these packages in the build. For more information, see go.microsoft.com/fwlink.
Repeated builds do nothing
Never mind, I missed the suggestion here
to include the BCL.Build targets file. it works now.
We have lots of projects on our TFS 2012 build server. For us this is a fairly tedious workaround for something that Visual Studio can do out of the box by simply checking the appropriate options.
When will Team Build have the option to perform automatic package restore similar to how Visual Studio does by checking the appropriate options?
@Scott McFadden: Visual Studio Online (FKA as hosted TFS or Team Foundation Service) already has support for automatic package restore. Please make sure to use the latest build template. I'm not sure about on prem TFS but I've asked the team. Stay tuned.
@Scott McFadden: Turns out that on prem Team Build (starting with TFS 2013) also supports automatic package restore. For TFS 2013 you don't need to configure anything when creating new builds. For builds that were created with older version of TFS you'll need to update the build process template. So this guidance, which is now also part of the official NuGet documentation under
only applies to Team Foundation Build 2012 and earlier, or to Build Process Templates that have been customized and had the NuGet Restore functionality removed.
Does this help?
I'm probabaly barking up the wrong tree here:
My TFS build runs into the error "The build restored NuGet packages. Build the project again to include these packages in the build." Which is fine if your running VS, but not when using a buid service.
It appears I need to include the nuget restore in my build process template. Your screenshots show ".proj" files, but mine just show solution files. I can't find where to add the nuget.exe step into the template.
Thanks in advance,
Will de Haan
@galletto: If your TFS doesn't support automatic package restore then I'd apply the pattern I described in this post.
Whether you project includes *.proj or *.sln doesn't matter. In both cases you add can add the build.proj file I showed above. It will simply wrap your .sln file. The screenshot shows how you can configure Team Build to use the new build.proj instead of directly building your solution.