Patrick's Blog

TFS 2010 - An Introduction to Gated Check-in

In TFS 2008 we made quite a few improvements to build, such as scheduled builds and continuous integration. The continuous integration feature is nice for detecting failures in the build as soon as possible to ensure minimal downtime and reduce the chance of generating a bad nightly build. However, it’s not a perfect solution since even when the build fails it is possible for others to submit unrelated changes and keep the build broken. As a first measure to aid this scenario, Buck Hodges wrote a check-in policy that would determine which, if any, build definitions would be affected by the changes being checked in – if the last build for the definition failed, the policy would not allow the user to check-in without overriding it. But there are quite a few issues with this approach, the main one being that it requires the users to install the policy and only override it when honestly submitting fixes for the build break. At this point you may be asking yourself “what if there were a way to stop people from submitting broken source code into the repository on the server?” This is exactly what gated check-in offers, and I intend to explore the features around gated check-in in the following paragraphs.

Setting Up Gated Check-in

Gated check-in is merely an extension of the check-in trigger for a build definition, so it should be easy to setup for those familiar continuous integration. In TFS 2008, we provided ‘Manual’, ‘Continuous Integration’, ‘Rolling builds’, and ‘Schedule’ (along with the option to force builds on that schedule even if no changes occurred since the last build). When viewing the build definition properties in TFS 2010, you will see the same options (with the addition of ‘Gated Check-in’):

When this trigger is selected for a build definition, any check-in that is made to a file that is mapped in the associated workspace will trigger a verification build. If the build succeeds, the changes will be submitted to the repository. If the build fails, the changes are not allowed to be submitted and must be fixed and resubmitted.

Submitting a Gated Check-in

When a user attempts to submit changes to a version control path that is mapped by a build definition with the ‘Gated Check-in’ trigger, they will be presented with the following dialog:

The first thing to point out here is that the check-in been automatically converted to a shelveset. In this particular scenario, the shelveset has been named ‘Gated_2009-06-29_02.49.00.9801’. The check-in notes, associated work items, etc., are all replicated into the shelveset and will be included in the changeset upon a successful build. The second piece of the dialog to pay attention to is the build definition selection. If there were only a single build definition that was considered affected by the changes, the combo box would be disabled and you would not be provided any options. If, however, there is more than one build definition that maps the changes being submitted, the user you will be presented with the list of build definitions as shown in the following image:

It’s important to note here that the actual definition selected to perform the check-in does not matter here, as both have been determined to map at least some subset of changes included in the check-in. However, it’s possible that ‘Gated Definition 1’ only maps a single file in the check-in, while ‘Gated Definition 2’ could map all the files or a larger subset. Currently we do not provide any indication as to what percentage of the changes are mapped by any given build definition. Once you have selected a build definition, you have the ability to specify some extra options. For example, it is entirely possible that you are a build administrator and wish to submit a fix directly rather than waiting for the verification build. However, this option will only be enabled for a selected build definition if you have the appropriate permission, ‘Override Check-in Validation by Build’. If you have the appropriate permission, we have provided the ability to bypass build verification as shown in the following image (although it’s disabled in this particular shot since no build definition is currently selected):

The other option available in the ‘Show Options’ section of the dialog is ‘Preserve my pending changes locally’. This allows the user to decide whether or not the changes being submitted should be left in the workspace or undone once the ‘Build Changes’ button is clicked (if the changes are left in the workspace, there is an entry point that we will explore later in the article to reconcile the submitted changes with those in your workspace). Once you are satisfied with the options on this dialog and ‘Build Changes’ is clicked, you will be redirected to the build explorer where the build just queued will be highlighted for easy discovery as shown here:

Now I know there is only one build in this particular queue so it wouldn’t be very difficult to locate the build, but imagine there is a team of 50-60 people all submitting changes to the same build definition, or even multiple definitions all sharing the same build controller. :-) Anyway, moving along to the successful gated check-in, you can see from the following build report that the submitted changeset is associated with the gated build, just like changesets would be associated with a non-gated build by comparing the labels and analyzing the differences.

Now that the build has completed successfully, you may have some further actions to perform if you chose not to preserve pending changes in your workspace. The entry point for this functionality is in build explorer, which is available via the context menu of the build detail entry in the ‘Completed Builds’ tab or the queued build entry in the ‘Queued Builds’ tab. If you only have a single workspace on the current machine then a workspace will be automatically selected for you. If, however, you have more than one workspace on the machine, you will be prompted to select the workspace which you would like to reconcile changes with. We currently do not support storing the source workspace from which the shelveset originated, so we cannot automatically determine the workspace used to create the shelveset. However, this feature is on a backlog and we are looking at including this functionality in a future version. The second entry point is from the gated check-in notification window in the build notification system tray application, which is included with the product in TFS 2010.

This has been a very targeted and simple walk-through of the new gated check-in functionality available in TFS 2010. I will be covering more advanced scenarios surrounding permissions, integration with the build notification system tray application, and more in future postings. Stay tuned!

The access control list is not canonical

Developing a program that does not require administrative privileges requires quite a bit of work up-front during the install. You need to make sure you do things like write registry keys to HKLM, create file system entries, and most importantly ensure that your ACLs (Access Control List) are correct to ensure smooth operation of your program while running under that non-admin account. However, sometimes this last step fails with a rather cryptic error, mentioning something along the lines of "The Access Control List is not canonical". After searching the web for solutions to this error, I couldn't find a definitive answer. One thing I did notice is if you open the 'security' dialog of the object that exhibits this issue, the Windows UI will actually recognize that there is an issue and ask if you would like to correct it. So obviously there is a way to fix this problem, but for my scenario requiring user intervention was just not good enough .. so I returned to searching. I finally came across an algorithm, buried in the depths of MSDN, which described how to fix the ordering of a DACL (Discretionary Access Control List). There was only one problem: it was written in VB. So, after a lot of hard work converting the code (just kidding, it was quite simple to convert), I produced the following helper method to attempt fixing the order of a DACL.

   internal static void CanonicalizeDacl(NativeObjectSecurity objectSecurity)
   {
       if (objectSecurity == null)
       {
           throw new ArgumentNullException("objectSecurity");
       }

       if (objectSecurity.AreAccessRulesCanonical)
       {
           return;
       }

       // A canonical ACL must have ACES sorted according to the following order:
        //   1. Access-denied on the object
        //   2. Access-denied on a child or property
        //   3. Access-allowed on the object
        //   4. Access-allowed on a child or property
        //   5. All inherited ACEs 
        RawSecurityDescriptor descriptor = new RawSecurityDescriptor(objectSecurity.GetSecurityDescriptorSddlForm(AccessControlSections.Access));

       List<CommonAce> implicitDenyDacl = new List<CommonAce>();
       List<CommonAce> implicitDenyObjectDacl = new List<CommonAce>();
       List<CommonAce> inheritedDacl = new List<CommonAce>();
       List<CommonAce> implicitAllowDacl = new List<CommonAce>();
       List<CommonAce> implicitAllowObjectDacl = new List<CommonAce>();

       foreach (CommonAce ace in descriptor.DiscretionaryAcl)
       {
           if ((ace.AceFlags & AceFlags.Inherited) == AceFlags.Inherited)
           {
               inheritedDacl.Add(ace);
           }
           else
            {
               switch (ace.AceType)
               {
                   case AceType.AccessAllowed:
                       implicitAllowDacl.Add(ace);
                       break;

                   case AceType.AccessDenied:
                       implicitDenyDacl.Add(ace);
                       break;

                   case AceType.AccessAllowedObject:
                       implicitAllowObjectDacl.Add(ace);
                       break;

                   case AceType.AccessDeniedObject:
                       implicitDenyObjectDacl.Add(ace);
                       break;
               }
           }
       }

       Int32 aceIndex = 0;
       RawAcl newDacl = new RawAcl(descriptor.DiscretionaryAcl.Revision, descriptor.DiscretionaryAcl.Count);
       implicitDenyDacl.ForEach(x => newDacl.InsertAce(aceIndex++, x));
       implicitDenyObjectDacl.ForEach(x => newDacl.InsertAce(aceIndex++, x));
       implicitAllowDacl.ForEach(x => newDacl.InsertAce(aceIndex++, x));
       implicitAllowObjectDacl.ForEach(x => newDacl.InsertAce(aceIndex++, x));
       inheritedDacl.ForEach(x => newDacl.InsertAce(aceIndex++, x));

       if (aceIndex != descriptor.DiscretionaryAcl.Count)
       {
           System.Diagnostics.Debug.Fail("The DACL cannot be canonicalized since it would potentially result in a loss of information");
           return;
       }

       descriptor.DiscretionaryAcl = newDacl;
       objectSecurity.SetSecurityDescriptorSddlForm(descriptor.GetSddlForm(AccessControlSections.Access));
   }

I hope this code saves someone out there a lot of trouble and time. Until next time …

Hello, World

What software developer can start anything new without the proverbial "Hello, World". Since I'm new to this side of things, this will serve as my introduction. My name is Patrick Carnahan, and I'm currently a developer on a product called Team Foundation Server, of which I focus on the Build Automation functionality. More specifically, I mainly design and work on the server code, including web services, data access, and SQL. This isn't where I started my almost 4 year career at Microsoft, however. I was originally hired on to the Version Control team within the same server product in June of 2005. At that point in time I knew nothing about .NET application development, ASP.NET web services, and very little about SQL development. This product has allowed me to interact with these technologies much more than I ever would have working on projects for myself. I have become proficient in SQL, including query performance analysis and debugging. My knowledge of .NET libraries continues to grow every day. I deal with distributed programming models all the time, which keeps my job extremely interesting.  

I am very passionate about the product that I work on and hope to deliver compelling releases in the future. Hopefully this spot will serve as a more general discussion of programming topics, but I will most likely include targeted posts helping users of Team Foundation Server get the most out of the product. I have learned quite a bit since starting my tenure with this company, and hopefully I can begin to share my knowledge with others.