Software teams need to continually check the current condition of their bits in order to ensure they are in at least a testable if not yet shippable state. Nearly every day, one or more people on the team seek answers to a few questions that are easily asked. “How good is that build?” “What was the last good build?” “What changes were made and what bugs were fixed in that build?”
Easily asked, but not necessarily easily answered. In this article I hope to help you quickly understand the quality indicators available in Team Foundation Build and how you can use them to answer your own questions about the quality of your completed builds. I also aim to begin filling a subtle gap in the recently published details about the AssociateChangesetsAndWorkItems activity.
Your Team Foundation Build system provides three primary indicators about how good a completed build is:
Before we explore all three indicators in detail, I think it will help for you to first see how the system generally evaluates the two automatic indicators:
There are a few exceptions that can cause the build to be evaluated differently than is shown in the above table. They are covered in the details below.
Build status is the most prominent indicator of build quality in your build system. You see it in both the build explorer and the build results window.
Build status is evaluated in the Default Template (DefaultTemplate.xaml). In a build based on DefaultTemplate.xaml, build status is determined as follows:
If you develop a custom build process template, you can use the SetBuildProperties activity to force the build status to be whatever value you want based on your own workflow logic. If you use this approach, then the logic described above is disabled for any builds based on your custom build process template.
The build status for each build is stored in the data warehouse via the IBuildDetail Status property.
Build status is displayed in the build results window and in Build Explorer. This indicator is used by the system to determine which retention policy to apply to a completed build. For more information, see Create a Basic Build Definition.
I actually made up the phrase, “build goodness” because nowhere in the product does this phrase appear. But as you will see, the concept behind build goodness is important.
Build goodness is evaluated by Team Foundation Server internal logic, and thus cannot be customized. The rule is simple. The build is “good” if both the IBuildDetail CompilationStatus property and the IBuildDetail TestStatus property are set to “Succeeded”. Otherwise, the build is “bad”.
Unlike build status, there is no attribute of a completed build in which build goodness is indicated. Rather, a reference to last good build is recorded in the IBuildDefinition LastGoodBuildLabel and LastGoodBuildUri properties.
Build goodness has one very important effect, which is that it is used by the AssociateChangesetsAndWorkItems activity to determine which completed builds are associated with recent changesets and their associated work items. Specifically, the AssociateChangesetsAndWorkItems activity:
Build quality is simply a string that someone on your team uses to characterize the quality of the completed build.
You manually set this value, as explained in Rate the Quality of a Completed Build. You can also customize the list of build quality values, as explained in Add or Remove Build Quality Values.
Build quality is displayed at the top of the build results window of a completed build. It is also displayed and can be used to filter the view in Build Explorer.
This post is the result of one of several key concept article ideas that crystalized in my mind as I researched and wrote Team Foundation Build Activities. I also want to express my thanks to Ed Blankenship, whose comment inspired me to write this blog post. He promptly pointed out a subtle gap in the section on AssociateChangesetsAndWorkItems, a gap that I hope this article partly fills.
I aim to blog additional experimental drafts such as this one, and then eventually publish the material on MSDN. Is this kind of information helpful? Is the above explanation of how the AssociateChangesetsAndWorkItems activity works sufficient? Or would you like to see the behavior laid out in more detail with examples and graphics? As always, I welcome your feedback.