Buck Hodges

Visual Studio Online, Team Foundation Server, MSDN

Posts
  • Buck Hodges

    TFS 2012 Update 1 Issue: Error while configuring build

    • 6 Comments

    [Update 2/1/13] A fix for the issues is now available.

    [Update 1/14/13] See this post for the latest on a fix with attaching collections.

    ---

    We’ve discovered an issue in TFS Update 1 that may result in errors when you try to configure a build agent or controller. You will see an error in the Build Service Configuration Wizard (configuring a new agent/controller) or in the TFS Administration Console (viewing the properties of a build controller/agent).

    In the event log on the server, you’d find an error about the database not being configured correctly.

    TF53010: The following error has occurred in a Team Foundation component or extension:

    <…>

    Detailed Message: TF30065: An unhandled exception occurred.

    <…>

    Exception Message: TF30040: The database is not correctly configured. Contact your Team Foundation Server administrator. (type DatabaseConfigurationException)

    Exception Stack Trace:    at Microsoft.TeamFoundation.Framework.Server.TeamFoundationSqlResourceComponent.TranslateException(Int32 errorNumber, SqlException sqlException, SqlError sqlError)

    <…>

    Exception Message: Operand type clash: dbo.typ_BuildControllerTableV2 is incompatible with dbo.typ_BuildControllerTable (type SqlException)

    The workaround is to run the following commands.

    • iisreset
    • net stop tfsjobagent
    • net start tfsjobagent

    Grant Holliday posted more details here.

    [Update 11/30/12] Note that this is a race condition that you may not hit and that the app pools for TFS are set to recycle every 29 hours (the Regular Time Interval setting in IIS), which will also fix this issue.

    Follow me at twitter.com/tfsbuck

  • Buck Hodges

    TFS and VS 2012 Update 1 now available

    • 18 Comments

    [Update 2/1/13] A fix for the issues is now available.

    [Update 1/14/13] See this post for the latest on issues with attaching a collection.

    As announced on Soma’s blog, Update 1 for Visual Studio and Team Foundation Server 2012 is now available. Over at the ALM blog, you can find more details on what’s new. For those using the Team Foundation Service at tfs.visualstudio.com, you are already familiar with the new features, as we update the service every three weeks.

    In addition to fixing bugs that were discovered after RTM, here are the new Features in Team Foundation Server in Visual Studio 2012 Update 1, which I’ve copied here from the ALM blog.

    Note: The Version control warehouse still has a 260 character limit and you need to have this update applied to both your Team Foundation Server and Visual Studio client.

    Build

    One caveat I want to mention is with upgrading your build computers. After you install TFS Update 1 on your build computers to update them, you will need to go through the configuration again, including choosing the collection, setting the service account, etc. If you have settings you want to preserve, be sure to look them up in the TFS Administration Console on your build computer and write them down for use after you upgrade your build computer. We hope to have this fixed for Update 2.

    Power Tools

    The Team Foundation Server Power Tools will be updated for Update 1 as well. There were significant changes to the APIs in the server DLLs (not the web services – the .NET assemblies), so tools like the backup/restore tool for the server had to be updated. The update 2012 power tools should be available later this week, barring any last-minute issues.

    Server Installation

    I’ve seen some questions about how to install Team Foundation Server 2012 Update 1. Because it is a server and we need to do things like modify the database as part of the installation, we designed TFS 2012 to use the regular installer to install the updates. You do not need to uninstall anything. Just run the installer, and it will take care of updating your TFS. After the installer completes, it launches the upgrade wizard.

    So, when you go to http://www.microsoft.com/visualstudio/eng/downloads you’ll see two choices for TFS 2012, Install Now and Download Now.

    image

    • Install Now will use the web installer and will download what is needed in order to install TFS 2012 Update 1
    • Download Now will download an entire layout, which you can use to install on a machine that doesn’t have internet access or if you need to update more than one and want to avoid the downloads by the web installer.

    After you run either one and the installation phase is complete, you will see the upgrade wizard.

    image

     

    You then must confirm that you have a backup.

    image

    Then select your database by setting the SQL Server Instance if the default isn’t correct and then using List Available Databases to see a list of all of the configuration DBs (usually there is just one).

    image

    At that point it becomes just a matter of clicking the Next button a few times, watching the upgrade run (okay, only for a small DB – you may want to grab a bite to eat while it runs for a bigger DB), and you’re done!

    image

    image

    image

     

    Note: If you had a browser open with the web UI for TFS 2012, you may get something garbled after the upgrade if you click on a link that doesn’t do a page refresh. The reason is that it’s a mix of the old and new web code. Just click Refresh in your browser to fix that.

    After clicking a link after upgrade (browser was open with the TFS web interface prior to upgrade):

    image

    After clicking Refresh in the browser:

    image

     

    Enjoy!

    Follow me on Twitter at twitter.com/tfsbuck

  • Buck Hodges

    New tool to upgrade from Visual SourceSafe to Team Foundation Server or Service

    • 16 Comments

    Today we have released an upgrade tool for users of VSS to upgrade to either on-premises TFS or the Team Foundation Service in the cloud! It provides a wizard-based UI for upgrading a VSS repository to TFS 2010, 2012, or the service.

    Compared to the old VSS conversion experience, there are quite a few improvements.

    • Wizard-based UI
    • VSS DLLs are bundled with the tool (no need to have VSS installed)
    • No manual mapping or creation of XML files
    • Lots of bug fixes
    • Support for Team Foundation Service

    With mainstream support having ended back in July for VSS 2005, now is a great time to move.

    You can download it now and find the documentation here.

    [UPDATE 11/14/12] We will be changing the docs to indicate that you are not required to have an empty team project collection or team project for this tool to work. However, we do not validate that there will not be a collision before the migration starts, so you will need to make sure that there is no path in the destination in TFS that will be the same as what you are migrating from VSS.

    Follow me on Twitter at twitter.com/tfsbuck

  • Buck Hodges

    Windows Phone SDK now available in Build for Team Foundation Service

    • 0 Comments

    Brian mentioned this in a blog post last week, but I think it may have gotten lost in everything else that was going on. The Team Foundation Service’s build feature (i.e., build in the cloud) has the Windows Phone SDK installed on it now so that you can build you Windows Phone 8 (and 7) apps. The blocker here had been that the SDK could not be installed on the VMs we use because they run Windows Server 2008 R2, and that was fixed with the SDK released last week.

    Follow me on Twitter at twitter.com/tfsbuck

  • Buck Hodges

    Video from 2012 Build Conference: Evolution of TFS into a Service

    • 1 Comments

    This year at //build/ conference I gave a presentation on the evolution of Team Foundation Server from a box product to a service that has a box product. It’s been a fun journey, and I enjoyed telling the story of what our team has accomplished over the last couple of years. I start with the beginning of the 2012 cycle and go over the changes we’ve made to our process and engineering system to be able to ship every three weeks. Along the way I answer questions from the audience and wrap up with some of the lessons we’ve learned. I hope you enjoy it.

    Developing Continuous Services: Real world experiences of the Team Foundation Service engineering team

    Downloads:

     

    Other Build conference presentations related to TFS:

  • Buck Hodges

    DevDiv is now running on TFS 2012 Update 1 final CTP

    • 9 Comments

    Back in June, I wrote about the upgrade of the main Developer Division server to TFS 2012 RC. That post described both the scale of that server as well as the issues that we found in the process of upgrading it and the for the first week of running it (we found most all of the issues in the first week). Last month we upgraded that server to TFS 2012 RTM.

    Now this past Friday, we upgraded it to the final CTP for TFS 2012 Update 1. While the CTP is not licensed for production use, we wanted to use this opportunity to find the issues that you only find in production on such a large server. We’ve been upgrading our smaller internal “dogfood” server called pioneer every three weeks with every CTP we’ve released, and of course we put the same update on tfspreview.com every three weeks. However, the developer division server scale is different with a few thousand users all working on the same project.

    Just like we did with the 2012 RC, we found some issues. We are going to have these addressed for the final release of TFS Update 1, so you’ll be able to use it knowing that we have put it into production on our mission-critical server.

    • Web.config settings aren’t being maintained (affected both build and version control cache directory)
    • Circular group memberships are causing performance problems

    Brian has listed the features in Visual Studio Update 1 and TFS Update 1.

    Follow me on Twitter at twitter.com/tfsbuck

  • Buck Hodges

    Upgrading TFS build from 2010 to 2012 – issues and workarounds

    • 3 Comments

    Jason Prickett has written a blog post summarizing issues and workarounds for problems encountered with TFS build when upgrading from TFS 2010 to TFS 2012.

    Upgrading your build definitions from TFS2010 to TFS2012

    Follow me on Twitter at twitter.com/tfsbuck

  • Buck Hodges

    Internal error loading the Changeset Comments checkin policy

    • 24 Comments

    [Update 11/26/12] You can get the fix by installing Update 1 (or newer) for Visual Studio 2012: http://www.microsoft.com/visualstudio/eng/downloads.

    Some customers, after starting to use Visual Studio 2012 with their existing TFS deployment, have been receiving check-in policy errors having to do with the Changeset Comments policy. The errors look like:

    Internal error in Changeset Comments Policy. Error loading the Changeset Comments Policy policy (The policy assembly 'Microsoft.TeamFoundation.PowerTools.CheckinPolicies.ChangesetComments, Version=9.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' is not registered.). Installation instructions: To install this policy, follow the instructions in CheckForComments.cs.

    The version number may vary slightly, but for this particular problem, it's always going to start with an 8 or a 9.

    Cause

    With VS 2005 through 2010, to get the Changeset Comments policy, you had to download and install the Team Foundation Power Tools. With VS 2012, the policy is included in the box and requires no additional download. This problem is a bug that was introduced as a part of moving that check-in policy into the product.

    For this particular bug, only users using Visual Studio 2012 will be affected. If you have other users in your organization connecting to the same TFS server with VS 2005, 2008, or 2010, then the Changeset Comments policy should be working fine for them.

    Workaround

    There is also a simple workaround that you can put in place immediately, as long as you have administrative permissions on your team project. Using a Visual Studio 2010 or 2012 client, navigate to the Team Project Settings for the Team Project that has the Changeset Comments policy configured. Remove the check-in policy from the Team Project, and then immediately re-add it. The fact that you performed this step from a Visual Studio 2010 or 2012 client will re-register the policy on the server as the "10.0.0.0" version, which fixes the problem. Now any client (VS 2005 through VS 2012) will be able to load the policy successfully.

    Fix

    We are including a fix for this problem in the final version of Visual Studio 2012 Update 1. You can read more about Update 1 in Brian's blog post, but the currently available preview release of that update doesn't include this fix.

    We apologize for the inconvenience!

    Follow me on Twitter at twitter.com/tfsbuck

  • Buck Hodges

    Standalone installer for the TFS client object model

    • 18 Comments

    [UPDATE 12/13/12] The TFS 2012 Object Model Installer is now available for download.

    This past week we released an installer for the Team Foundation Server 2010 SP1 client object model.  Up to this point the only way to get it was to install Team Explorer.  For folks writing tools that access TFS, this was a pain, as having to install TE was overkill when all that was needed was a small set of DLLs.  Lots of folks have asked for this, and it’s finally available.  We’ll be shipping a TFS 2012 client object model installer in the RTM timeframe as well.

    Team Foundation Server 2010 SP1 Object Model Installer

    This is the stand-alone Object Model Installer for TFS 2010 (Sp1).  This may be used for developing applications that integrate with TFS 2010.

    This is the same Object Model included with the Visual Studio & Team Explorer SKU, in a stand-alone package.  This OM will allow applications to connect to a TFS 2010 server, without the need for a Visual Studio or Team Explorer being installed on the machine. (Proper licensing / CALs are still required)

    Download

    Enjoy!

    Follow me on Twitter at twitter.com/tfsbuck

  • Buck Hodges

    Developer Division is running on TFS 2012 RC!

    • 11 Comments

    This past weekend we upgraded the TFS server used by the Developer Division and other teams who deliver Visual Studio 2012.  It is now running TFS 2012 RC!

    Back in the beginning, the DevDiv server was our dogfood server.  Then as all of the folks shipping products in Visual Studio, there were too many critical deadlines to be able to put early, sometimes raw, builds on the server.   So we dogfood TFS on a server called pioneer, as described here.  Pioneer is used mostly by the teams in ALM (TFS, Test & Lab Management, and Ultimate), and we’ve been running TFS 2012 on it since February 2011, which was a full year before beta. Never before have we been able to use TFS so early in the product cycle, and our ability to get that much usage on early TFS 2012 really showed in the successful upgrade of the DevDiv server.

    We also run TFS 2012 in the cloud at http://tfspreview.com, and that’s been running for a year now.  While that’s not a dogfood effort, it’s helped us improve TFS 2012 signficantly. The other dogfooding effort leading up to this upgrade was Microsoft IT.  They upgraded a TFS server to TFS 2012 Beta, and we learned from that as well.

    The scale of the DevDiv server is huge, being used by 3,659 users in the last 14 days.  Nearly all of those users are working in a single team project for the delivery of Visual Studio 2012.  Our branches and workspaces are huge (a full branch has about 5M files, and a typical dev workspace 250K files).  As a result, we always find product issues when we upgrade it.  For the TFS 2010 product cycle, we did not upgrade the server until after RTM.  Having been able to do this upgrade with TFS 2012 RC, the issues we find will be fixed in the RTM release of TFS 2012!

    Here’s the topology of the DevDiv TFS deployment, which I’ve copied from Grant Holliday’s blog post on the upgrade to TFS 2010 RTM two years ago.  I’ll call out the major features.

    • We use two application tiers behind an F5 load balancer.  The ATs will each handle the DevDiv load by themselves, in case we have to take one offline (e.g., hardware issues).
    • There are two SQL Server 2008 R2 servers in a failover configuration.  We are running SP1 CU1.  TFS 2012 requires an updated SQL 2008 for critical bug fixes.
    • SharePoint and SQL Analysis Services are running on separate computer in order balance the load (cube processing is particularly intensive).
    • We use version control caching proxy servers both in Redmond and for remote offices.

    These statistics will give you a sense of the size of the server.  There are two collections, one that is in use now and has been used since the beginning of the 2012 product cycle (collection A) and the original collection which was used by everyone up through the 2010 product cycle (collection B).  The 2010 collection had grown in uncontrolled ways, and there were more than a few hacks in it from the early days of scaling to meet demand.  Since moving to a new collection, we’ve been able pare back the old collection, and the result of those efforts has been a set of tools that we use on both collections (we’ll eventually release them).  Both collections were upgraded.  The third column is a server we call pioneer. 

    Grant posted the queries to get the stats on your own server (some need a little tweaking because of schema changes, and we need to add build).  Also, the file size is now all of the files, including version control, work item attachments, and test attachments, as they are all stored in the same set of tables now.

    Collection A Collection B Pioneer
    Recent Users 3659 1516 1,143
    Build agents and controllers 2,636 284 528
    Files 16,855,771 21,799,596 11,380,950
    Uncompressed File Size (MB) 14,972,584 10,461,147 6,105,303
    Compressed File Size (MB) 2,688,950 3,090,832 2,578,826
    Checkins 681,004 2,294,794 133,703
    Shelvesets 62,521 12,967 14,829
    Merge History 1,512,494,436 2,501,626,195 162,511,653
    Workspaces 22,392 6,595 5,562
    Files in workspaces 4,668,528,736 366,677,504 406,375,313
    Work Items 426,443 953,575 910,168
    Areas & Iterations 4,255 12,151 7,823
    Work Item Versions 4,325,740 9,107,659 9,466,640
    Work Item Attachments 144,022 486,363 331,932
    Work Item Queries 54,371 134,668 28,875

    The biggest issue we faced after the upgrade was getting the builds going again.  DevDiv (collection B) has 2,636 build agents and controllers, with about 1,600 being used at any given time.  On pioneer, we didn’t have nearly that many running.  The result was that we hit a connection limit, and the controllers and agents would randomly go online and offline.  Working with WCF team, we now understand the connection settings, and this will be fixed for RTM.  We’ve also now got a test in place that finds the issue and verifies the fix.

    Here’s a subset of the issues we’ve found, including all of the ones we found the first day.  There are probably another dozen that we’ve found during the week that I haven’t listed.

    1. Build machine connections are limited to 128 for some reason.
    2. Build config wizard iterates through every build machine and resolves its DNS entry.
    3. Code to handle file attachments not yet upgraded fails. prc_RetrieveFIle was calling prc_MigrateFile incorrectly.
    4. Issue with leading key transfer when there are tables with duplicate index names in the database.
    5. Warehouse upgrade timed out on dropping primary keys.
    6. Creating a build label times out.  Bad query plan on prc_iiUpdateBuildInformation
    7. DevDiv Upgrade: File Migration Job is too slow
    8. Command-line only users will not have their location service cache updated on certain commands which leads to confusion on available services.
    9. Unable to access work items page.  MaxJsonLength should be configurable (in web.config) for JSON action results.
    10. Updating build controllers/agents makes many calls to the server and suffers lock contention.
    11. QueryBuilds is slow on vstfdevdiv.
    12. Browser cache needs to be cleared for some users after upgrade.
    13. Work item template links created when the server was 2010 no longer work.
    14. SOAP notification breaks compat with previous releases.
    15. Get with a workspace version used by tfpt scorch is slow.

    The upgrade to TFS 2012 RC was a huge success, and it was a very collaborative effort across TFS, central engineering, and IT.  We quickly got everything to full scale and running smoothly.  As a result of this experience and our experience on pioneer, TFS 2012 is not only a great release with an incredible set of features, but it’s also running at high scale on a mission critical server!

    I hope you upgrade your server to TFS 2012.  You are going to love it!

     

    Follow me at twitter.com/tfsbuck

  • Buck Hodges

    How to customize the build summary page in TFS 2012 without a plug-in

    • 0 Comments

    Jason Prickett has been writing some great posts about the new features in Build for TFS 2012.  One of my favorites is the ability for you to add sections to the build summary page without having to write any plug-ins.  You just use the WriteCustomSummaryInformation activity in your workflow or make the equivalent OM calls from a workflow activity.  The result is that you can display results, hyperlinks, and more on the build summary page.

    TFS11 RC – Simple Build Summary Customization

    So, I did a few posts when TFS 2010 released on how developers could customize the Visual Studio Build Details View. However, it required creating a VS Add-In or Package and then deploying that to all your clients. Well, with the RC release of TFS 11 and Visual Studio, there is a much easier way to add simple content to your Build Details Summary view.

    more…

    Here’s a screen shot showing a “My New Section” that he added.  Enjoy!

     

  • Buck Hodges

    Visual Studio 2012 features enabled by using a TFS 2012 server

    • 14 Comments

    While many of the features in Visual Studio 2012 are available to users using TFS 2008 or 2010, there are some features that are only available when using a TFS 2012 server.  Also, the web experience in TFS 2012 has been rebuilt entirely from the ground up, and the result is a fast, fluid experience that also includes new experiences tailored to Scrum.

    Once you’ve upgraded your server to TFS 2012, installed your first TFS server, or are using the Team Foundation Service Preview, here are some of the features you are now able to use.  The goal here isn’t an exhaustive list but to get you started.

    Version Control

    • Async checkout – We have added a new TFS 2012 feature so that VS 2012 will do checkouts in the background for server workspaces.  That eliminates the pause when you start typing and VS checks out the file.  Turning it on turns off checkout locks, but you can still use checkin locks.  You can find out how to turn it on here.
    • Merge on Unshelve – Shelvesets can now be unshelved into a workspace even if there are local changes on files in the shelveset.  Conflicts will be created for any items modified both locally and in the shelveset, and you will resolve them as you would any other conflict.
    • Local workspacesLocal workspaces allow many operations to be done offline (add, edit, rename, delete, undo, diff) and are recommended only for workspaces with fewer 50,000 files.  Local workspaces are now the default with TFS 2012, but you can control that if you want server workspaces to be the default (setting is in the dialog shown here).

    Alerts editor – Replacing the power tool, there’s now an even better experience built into the web UI for managing your email notifications.  You will be able to see and manage all of your alerts in one place.  To get to it, go to a project, click on your name in the upper right, and choose Manage Alerts.  Note that you can only get to it if your administrator has configured your server to send email.

    Retry build – If your build fails for reasons unrelated to your changes, you can now right-click and retry it.

    My Work – This is a new feature of Team Explorer that allows you to suspend and resume work quickly and easily.  This feature is only available in Premium or Ultimate.

    Code Review – You will be able to use the new code review experience.  You can start a code review from the Pending Changes page in Team Explorer or from the History window by right clicking on a changeset.  Your list of code reviews is shown in the My Work window.  This feature also requires Premium or Ultimate.

    Agile/Scrum – We’ve added a first-class experience for Scrum in TFS 2012 in our web UI.  To use these, teams will need to be created.  You can learn more here.

    • Team – We now have teams in TFS!
    • Task Board
    • Product Backlog and Sprint Planning – these require Premium or Ultimate (see this post about enabling via the licensing feature)
    • Team Home Page with a burn down chart and customizable tiles based on team favorites.  To add tiles to a team’s home page, create team favorites in work item tracking, build and version control.

    Feedback – You’ll be able to use this feature to request feedback on your work, and users will be able to use the Microsoft Feedback Client (including accessing it from Microsoft Test Manager).

    Storyboarding – This is available in VS Premium or Ultimate and in Test Pro, and TFS 2012 adds a Storyboard tab to work items such as User Story.  You can learn more here and find additional shapes here.

    You can also use the new unit testing features to run tests via Nunit, xunit, qunit, and more in your TFS build (aka team build).

    Microsoft Test Manager – there are a few features only available when using MTM 2012 with TFS 2012

    • Inline Reports/Test Results (Testing Center -> Plan Activity -> Results)

    clip_image001

    • Exploratory feature (Testing Center->Test Activity->Do Exploratory Testing, Testing Center->Test Activity->View Exploratory Test Sessions)

    clip_image001[4]

    • Support for Standard Environments (Lab Center->Lab Activity->Environments

    clip_image001[6]

    For a more detailed list of the features in ALM for 2012, see the ALM 2012 section or download the product guide.  There’s also a list of what’s new in VS 2012.

    [UPDATE 6/20/2012]  I’ve added more details around MTM requested by Neno and supplied by Ravi.  I’ve also fixed the broken product guide link (thanks, Mayur).

    [UPDATE 6/7/2012]  TFS Express does not include any of the Agile features.  It is really focused on source control, build, and bug tracking.  You can read more about it here.

    Follow me at twitter.com/tfsbuck

  • Buck Hodges

    Team Explorer 2012 keyboard shortcuts

    • 9 Comments

    Team Explorer 2012 has the following keyboard shortcuts for the built-in pages.  We are adding some shortcuts within the pages for RTM.

    UPDATE 08/20/12:  The last seven shortcuts were added for RTM.

    Shortcut Team Explorer Page
    Ctrl+' Search
    Ctrl+0,H Home
    Ctrl+0,P Pending Changes
    Ctrl+0,M My Work
    Ctrl+0,W Work Items
    Ctrl+0,B Build
    Ctrl+0,R Reports
    Ctrl+0,D Documents
    Ctrl+0,S Settings
    Ctrl+0,A Web Access – team home page
    F5 Refresh
    Ctrl+Up Arrow Move focus to the previous visible section header
    Ctrl+Down Arrow Move focus to the next visible section header
    Alt+Left Navigate backward
    Alt+Right Navigate forward
    Alt+Home Focus the navigation control
    Alt+0 Focus the page top level content
    Alt+[1-9] Focus the visible section [1-9] level content
    Alt+Up Focus the previous visible section content
    Alt+Down Focus the next visible section content

    Follow me at twitter.com/tfsbuck

  • Buck Hodges

    After upgrading: Updating licensing and enabling async checkout for server workspaces

    • 1 Comments

    The other day I wrote a post with a summary of links for upgrading team project settings to turn on all of the new features, such as code review, task board, and more.

    There are a couple more things I want to point out that you’ll want to tweak after the upgrade.

    First is licensing for the feature set.  The default setting is Standard.  You can choose to set the default to Full if all of your users are licensed for Premium, Ultimate, or Test Professional 2012 (see the editions page).  If not, you can add the people or groups who are licensed for full.  Only users with Full access will get the following features.

    • Backlog and Sprint Planning Tools
    • Request and Manage Feedback

    Without adding licensing access to the feature set, you will get "TF400409: You do not have licensing rights to access this feature: Backlog and Sprint Planning Tools" (you may see other features listed, depending on what you try access).

    From the Welcome Page, click on Administer Team Foundation Server and then click on the Web Access tab.  To change the default, click on Full and then click on “Set as default web access” to make it the default.  The screen shot shows what you’ll see after making the change.  Note that the “Limited” setting is for users who don’t have a CAL and that you want to grant access to see their work items and file new ones (also known as work item only view in the TFS 2010).

    licensing

    Second, while we have a great local workspaces feature that is new for TFS 2012 and you can easily switch a server workspace to a local workspace, you may want to keep your workspace a server workspace if it is extremely large (about 50,000 files for a spinning disk, higher for SSD).  Internally, our dev workspaces are often 250,000 files or more (getting them trimmed down is an ongoing effort).  We’ve introduced a feature in TFS 2012 where checkouts in VS 2012 are completely asynchronous if you enable the setting on the TFS 2012 server.  That means that there’s never a pause in VS when it checks out a file you are editing – it happens in the background.

    As a consequence of turning on this setting, TFS disables exclusive checkout (any existing ones are converted to exclusive checkin locks) and the get latest on checkout feature.  You can find the documentation here.  You’ll also see in this dialog that you can control whether newly created workspaces default to server or local (out of the box, the default is local workspaces, since we believe that’s what the vast majority of folks will want most of the time).

    To get to this dialog, choose Team –> Team Project Collection Settings from the main menu in VS, then choose Source Control.

    asynccheckout

    [UPDATE 6/4/2012]  I've included the TF400409 error message.

    Follow me at twitter.com/tfsbuck

  • Buck Hodges

    Use the wizard to upgrade team projects in TFS 2012

    • 1 Comments

    As you’ve seen from Brian’s post and a bunch of others, Visual Studio and Team Foundation Server 2012 RC are now available.  One of the consistent pain points in upgrading from one version of TFS to the next has been in upgrading the process for a team project.  With TFS 2012, we’ve taken our first steps toward making it dramatically better.  We now provide a wizard that is able to upgrade a team project’s process quickly and easily if you have made no customizations or only a limited set of customizations.  There are a number of resources available to help you through the process, and I wanted to provide a collection of links in one place.

    First, Tim Elhajj, who creates documentation on administering TFS, has posted a very easy to read article on how to use this new configuration upgrade wizard, and the end of that page has links to MSDN for resolving errors and dealing with the differences in the Agile process template as well as adding new shared queries that will be automatically created for new team projects.  If you are new to upgrading TFS, you can start at the beginning to walk through the upgrade process.

    Ewald Hofman, a program manager for work item tracking, has posted code for a tool to allow you to automate upgrading many team projects.  For those of you with lots of team projects, that’s really useful.  If you want to know more details on how the project upgrade process works, you’ll want to read his deep dive on it.  It’s called configuring features, but it is team project process upgrade.

    What we’ve provided in 2012 is not going to solve the problem of upgrading significantly customized or custom-built processes, but it should go a long ways toward helping so many folks who don’t make significant changes to the process template.

    Though not directly related to upgrading your team projects, you can get an overview of what all has changed VS and TFS 2012 on the What’s New in Visual Studio 2012 RC page.

    Follow me on twitter

  • Buck Hodges

    TFS 11: API for teams and understanding burn down charts

    • 0 Comments

    Ivan Popek has been spending a lot of time lately digging into TFS 11.  He has written a series of posts on the APIs for teams in TFS 11, and he explains the burn down chart trend lines.

     

    Follow me on Twitter at twitter.com/tfsbuck

  • Buck Hodges

    Perf counters in TFS 11

    • 0 Comments

    Jim Szubryt has written a post listing the perf counters available in all versions of TFS and calls out the new ones in TFS 11.  Check it out if you are looking at the performance of your server.

    New Perfmon Counters With TFS 11 Beta

    Whether you are a small company running TFS or a large Enterprise there may come a time for you too where performance of your TFS system may come into question and you will need to determine what to look at when “TFS seems to be running slow”. You should proactively be establishing a baseline of performance for your environment and Perfmon counters can help. Having this baseline will be extremely important in understanding the differences between what has changed from when the application was running properly and the state when it is not. I can attest to this from experiences I have had with when load testing a financial system a few years ago before it went live. We started capturing Perfmon counters prior to processing transactions and then ramped up the users/transactions created. We were able to identify and address memory and SAN issues before the system went live. Much better than doing this in a production environment and affecting real users.

    With the upcoming release of TFS there will be additional counters to assist in identifying what may be affecting TFS. My intention with this post is to raise your awareness of what options you have and not to guide you through determining which value is indicating a performance issue. That task is left to you to measure against what they were when the system was running smoothly and what they are when it is not. If you need some help in setting up counter collection you can take a look at Grant Holliday’s post on Querying Perfmon data from SQL. He has another good post on Large TFS Performance characteristics. They are a couple years old however, they are still very relevant.

    more…

    Follow me at twitter.com/tfsbuck

  • Buck Hodges

    TFS 11 Beta upgrade: Existing workflows that have custom workflow activities using the TFS client OM

    • 0 Comments

    Duat, a developer on Team Foundation Build, has written a blog post on how to resolve post-upgrade issues with your build workflow if your 2010 build workflow uses custom workflow activities that use the TFS client object model.

    Your custom assemblies need update, or else, redirecting

    Loading your custom assemblies in the TFS 11 build service process triggers loading TFS 2010 assemblies, and the Workflow Foundation (WF) services are not happy about that. Why?

    The build process template, for a particular reason, references the TFS OM assemblies by their simple names. When there are more than one version of the same assembly present in the process, the WF services, while loading and validating the build workflow, attempts to resolve the simple-name references with best effort, and in this case it decides to fail with a namespace ambiguity error.

    Here’s what you likely see when this issue occurs:

    TF215097: An error occurred while initializing a build for build definition \TFS\Build MAIN: The root element of the build process template found at $/TFS/BuildProcessTemplates/MainDefaultTemplate.xaml (version C182) is not valid. (The build process failed validation. Details: Validation Error: Compiler error(s) encountered processing expression "New Microsoft.TeamFoundation.Build.Workflow.Activities.BuildSettings()".'BuildSettings' is ambiguous in the namespace 'Microsoft.TeamFoundation.Build.Workflow.Activities'.

    more…

  • Buck Hodges

    Running only impacted tests in TFS 2010 Build

    • 2 Comments

    Richard Hundhausen pointed out this blog post recently, and I thought I’d mention it here.  Rob Maher walks through the process of setting up a build definition to run only the tests that involve the code that was changed.

    TFS 2010 Build - Only run impacted tests

    If your tests take a long time to run, you may wish to only run the tests that have been impacted by code changes checked in with the build (and of course run a full nightly build that executes all tests :) Unfortunately there is no out of the box feature to do this, so we need to edit our build xaml to do it.

    more…

  • Buck Hodges

    Team Foundation Version Control client API example for TFS 2010 and newer

    • 39 Comments

    Over six years ago, I posted a sample on how to use the version control API.  The API changed in TFS 2010, but I hadn’t updated the sample.  Here is a version that works with 2010 and newer and is a little less aggressive on clean up in the finally block.

    This is a really simple example that uses the version control API.  It shows how to create a workspace, pend changes, check in those changes, and hook up some important event listeners.  This sample doesn't do anything useful, but it should get you going.

    You have to supply a Team Project as an argument.

    The only real difference in this version is that it uses the TeamFoundationServer constructor (in beta 3, you were forced to use the factory class).

    You'll need to add references to the following TFS assemblies to compile this example.

    Microsoft.TeamFoundation.VersionControl.Client.dll
    Microsoft.TeamFoundation.Client.dll

    Code Snippet
    1. using System;
    2. using System.Collections.Generic;
    3. using System.Diagnostics;
    4. using System.IO;
    5. using System.Text;
    6. using Microsoft.TeamFoundation.Client;
    7. using Microsoft.TeamFoundation.VersionControl.Client;
    8.  
    9. namespace BasicSccExample
    10. {
    11.     class Example
    12.     {
    13.         static void Main(string[] args)
    14.         {
    15.             // Verify that we have the arguments we require.
    16.             if (args.Length < 2)
    17.             {
    18.                 String appName = Path.GetFileName(Process.GetCurrentProcess().MainModule.FileName);
    19.                 Console.Error.WriteLine("Usage: {0} collectionURL teamProjectPath", appName);
    20.                 Console.Error.WriteLine("Example: {0} http://tfsserver:8080/tfs/DefaultCollection $/MyProject", appName);
    21.                 Environment.Exit(1);
    22.             }
    23.  
    24.             // Get a reference to our Team Foundation Server.
    25.             TfsTeamProjectCollection tpc = new TfsTeamProjectCollection(new Uri(args[0]));
    26.  
    27.             // Get a reference to Version Control.
    28.             VersionControlServer versionControl = tpc.GetService<VersionControlServer>();
    29.  
    30.             // Listen for the Source Control events.
    31.             versionControl.NonFatalError += Example.OnNonFatalError;
    32.             versionControl.Getting += Example.OnGetting;
    33.             versionControl.BeforeCheckinPendingChange += Example.OnBeforeCheckinPendingChange;
    34.             versionControl.NewPendingChange += Example.OnNewPendingChange;
    35.  
    36.             // Create a workspace.
    37.             Workspace workspace = versionControl.CreateWorkspace("BasicSccExample", versionControl.AuthorizedUser);
    38.  
    39.             String topDir = null;
    40.  
    41.             try
    42.             {
    43.                 String localDir = @"c:\temp\BasicSccExample";
    44.                 Console.WriteLine("\r\n--- Create a mapping: {0} -> {1}", args[1], localDir);
    45.                 workspace.Map(args[1], localDir);
    46.  
    47.                 Console.WriteLine("\r\n--- Get the files from the repository.\r\n");
    48.                 workspace.Get();
    49.  
    50.                 Console.WriteLine("\r\n--- Create a file.");
    51.                 topDir = Path.Combine(workspace.Folders[0].LocalItem, "sub");
    52.                 Directory.CreateDirectory(topDir);
    53.                 String fileName = Path.Combine(topDir, "basic.cs");
    54.                 using (StreamWriter sw = new StreamWriter(fileName))
    55.                 {
    56.                     sw.WriteLine("revision 1 of basic.cs");
    57.                 }
    58.  
    59.                 Console.WriteLine("\r\n--- Now add everything.\r\n");
    60.                 workspace.PendAdd(topDir, true);
    61.  
    62.                 Console.WriteLine("\r\n--- Show our pending changes.\r\n");
    63.                 PendingChange[] pendingChanges = workspace.GetPendingChanges();
    64.                 Console.WriteLine("  Your current pending changes:");
    65.                 foreach (PendingChange pendingChange in pendingChanges)
    66.                 {
    67.                     Console.WriteLine("    path: " + pendingChange.LocalItem +
    68.                                       ", change: " + PendingChange.GetLocalizedStringForChangeType(pendingChange.ChangeType));
    69.                 }
    70.  
    71.                 Console.WriteLine("\r\n--- Checkin the items we added.\r\n");
    72.                 int changesetNumber = workspace.CheckIn(pendingChanges, "Sample changes");
    73.                 Console.WriteLine("  Checked in changeset " + changesetNumber);
    74.  
    75.                 Console.WriteLine("\r\n--- Checkout and modify the file.\r\n");
    76.                 workspace.PendEdit(fileName);
    77.                 using (StreamWriter sw = new StreamWriter(fileName))
    78.                 {
    79.                     sw.WriteLine("revision 2 of basic.cs");
    80.                 }
    81.  
    82.                 Console.WriteLine("\r\n--- Get the pending change and check in the new revision.\r\n");
    83.                 pendingChanges = workspace.GetPendingChanges();
    84.                 changesetNumber = workspace.CheckIn(pendingChanges, "Modified basic.cs");
    85.                 Console.WriteLine("  Checked in changeset " + changesetNumber);
    86.             }
    87.             finally
    88.             {
    89.                 if (topDir != null)
    90.                 {
    91.                     Console.WriteLine("\r\n--- Delete all of the items under the test project.\r\n");
    92.                     workspace.PendDelete(topDir, RecursionType.Full);
    93.                     PendingChange[] pendingChanges = workspace.GetPendingChanges();
    94.                     if (pendingChanges.Length > 0)
    95.                     {
    96.                         workspace.CheckIn(pendingChanges, "Clean up!");
    97.                     }
    98.  
    99.                     Console.WriteLine("\r\n--- Delete the workspace.");
    100.                     workspace.Delete();
    101.                 }
    102.             }
    103.         }
    104.  
    105.         internal static void OnNonFatalError(Object sender, ExceptionEventArgs e)
    106.         {
    107.             if (e.Exception != null)
    108.             {
    109.                 Console.Error.WriteLine("  Non-fatal exception: " + e.Exception.Message);
    110.             }
    111.             else
    112.             {
    113.                 Console.Error.WriteLine("  Non-fatal failure: " + e.Failure.Message);
    114.             }
    115.         }
    116.  
    117.         internal static void OnGetting(Object sender, GettingEventArgs e)
    118.         {
    119.             Console.WriteLine("  Getting: " + e.TargetLocalItem + ", status: " + e.Status);
    120.         }
    121.  
    122.         internal static void OnBeforeCheckinPendingChange(Object sender, ProcessingChangeEventArgs e)
    123.         {
    124.             Console.WriteLine("  Checking in " + e.PendingChange.LocalItem);
    125.         }
    126.  
    127.         internal static void OnNewPendingChange(Object sender, PendingChangeEventArgs e)
    128.         {
    129.             Console.WriteLine("  Pending " + PendingChange.GetLocalizedStringForChangeType(e.PendingChange.ChangeType) +
    130.                               " on " + e.PendingChange.LocalItem);
    131.         }
    132.     }
    133. }

  • Buck Hodges

    Updating a team project to use new features after upgrading to TFS 11 Beta

    • 3 Comments

    [UPDATE 3/12/12] Aaron pointed out Ewald's recent detailed blog post that walks through adding optional metadata beyond the MSDN article: http://blogs.msdn.com/b/visualstudioalm/archive/2012/03/06/get-your-agile-project-fixed-after-an-upgrade-from-tfs2010-to-tfs11-beta.aspx.

    Team Foundation Server 11 Beta can be used to upgrade your existing TFS 2010 or 2008 server and used in production.  We’ll be supporting upgrade from TFS 11 Beta to RC (you’ll very likely need to go from Beta to RC to RTM in order to follow the supported path).  As Jason said in his beta blog post, TFS 11 beta is “go-live,” which means you can use it in production and receive support (details).

    Once you upgrade your server, you’ll want to enable new features that require things like new work item types.  We are working on changes that will hopefully make it easier for RC.  Here’s the documentation you’ll need to make the necessary changes.

    Updating an Upgraded Team Project to Access New Features

    After you upgrade to Visual Studio 11 Team Foundation Server Beta, you can still access the data from team projects that you created in the previous release. By downloading a localized zip file, extracting the files, and running the provided batch file of commands, you can update your team project to use the following new tools for managing the application lifecycle:

    read more…

  • Buck Hodges

    Permission error with creating a team project from VS 2010 on TFS 2012

    • 28 Comments

    You must use the Visual Studio Team Explorer 2012 (included in all Visual Studio editions or may be separately downloaded) to create a team project on a TFS 2012 server.  If you use VS 2010, you will get an error about not having permission.  The error message is very misleading, because it’s not a problem with your permissions.

    ---------------------------
    Microsoft Visual Studio
    ---------------------------
    TF30172: You do not have permission to create a new team project.
    ---------------------------
    OK   Help  
    ---------------------------

  • Buck Hodges

    Listing the work items associated with changesets for a path

    • 2 Comments

    Philip wrote a simple app to list the work items associated with the changesets for a given path, and it’s in some ways an enhanced update of Naren’s post.

    Given an URL to a collection and a server path (e.g., $/myproject/coolthing), it will list the work items that are associated with the most recent 25 checkins.  This sample shows how to use the linking service to convert the work item artifact URIs that are stored with the changesets to get the core work item fields (ID, assigned to, state, type, and title).

    It will produce output like the following.

    Id: 352694 Title: Improve performance of queuing servicing jobs on Azure.

    You will need to reference the following DLLs to build this, all of which are found on the .NET tab of the Add Reference dialog in Visual Studio 2010.

    • Microsoft.TeamFoundation.Client.dll
    • Microsoft.TeamFoundation.Common.dll
    • Microsoft.TeamFoundation.VersionControl.Client.dll
    using System; 
    using System.Collections.Generic; 
    using System.Diagnostics; 
    using Microsoft.TeamFoundation; 
    using Microsoft.TeamFoundation.Client; 
    using Microsoft.TeamFoundation.VersionControl.Client; 
    
    namespace ListWorkItems 
    { 
        class Program 
        { 
            static void Main(string[] args) 
            { 
                if (args.Length < 2)
                { 
                    Console.WriteLine("Usage: listworkitems <URL for TFS> <server path>"); 
                    Environment.Exit(1); 
                } 
    
                TfsTeamProjectCollection tpc = new TfsTeamProjectCollection(new Uri(args[0]));
                VersionControlServer vcs = tpc.GetService<VersionControlServer>(); 
    
                // Get the changeset artifact URIs for each changeset in the history query
                List<String> changesetArtifactUris = new List<String>(); 
    
                foreach (Object obj in vcs.QueryHistory(args[1],                       // path we care about ($/project/whatever) 
                                                        VersionSpec.Latest,            // version of that path
                                                        0,                             // deletion ID (0 = not deleted) 
                                                        RecursionType.Full,            // entire tree - full recursion
                                                        null,                          // include changesets from all users
                                                        new ChangesetVersionSpec(1),   // start at the beginning of time
                                                        VersionSpec.Latest,            // end at latest
                                                        25,                            // only return this many
                                                        false,                         // we don't want the files changed
                                                        true))                         // do history on the path
                { 
                    Changeset c = obj as Changeset; 
                    changesetArtifactUris.Add(c.ArtifactUri.AbsoluteUri); 
                } 
    
                // We'll use the linking service to get information about the associated work items
                ILinking linkingService = tpc.GetService<ILinking>(); 
                LinkFilter linkFilter = new LinkFilter(); 
                linkFilter.FilterType = FilterType.ToolType; 
                linkFilter.FilterValues = new String[1] { ToolNames.WorkItemTracking };  // we only want work itms
    
                // Convert the artifact URIs for the work items into strongly-typed objects holding the properties rather than name/value pairs 
                Artifact[] artifacts = linkingService.GetReferencingArtifacts(changesetArtifactUris.ToArray(), new LinkFilter[1] { linkFilter });
                AssociatedWorkItemInfo[] workItemInfos = AssociatedWorkItemInfo.FromArtifacts(artifacts);
    
                // Here we'll just print the IDs and titles of the work items
                foreach (AssociatedWorkItemInfo workItemInfo in workItemInfos)
                { 
                    Console.WriteLine("Id: " + workItemInfo.Id + " Title: " + workItemInfo.Title); 
                } 
            } 
        } 
    
        internal class AssociatedWorkItemInfo
        { 
            private AssociatedWorkItemInfo() 
            { 
            } 
    
            public int Id 
            { 
                get 
                { 
                    return m_id; 
                } 
            } 
    
            public String Title 
            { 
                get 
                { 
                    return m_title; 
                } 
            } 
    
            public String AssignedTo 
            { 
                get 
                { 
                    return m_assignedTo; 
                } 
            } 
    
            public String WorkItemType 
            { 
                get 
                { 
                    return m_type; 
                } 
            } 
    
            public String State 
            { 
                get 
                { 
                    return m_state; 
                } 
            } 
    
            internal static AssociatedWorkItemInfo[] FromArtifacts(IEnumerable<Artifact> artifacts)
            { 
                if (null == artifacts)
                { 
                    return new AssociatedWorkItemInfo[0];
                } 
    
                List<AssociatedWorkItemInfo> toReturn = new List<AssociatedWorkItemInfo>(); 
    
                foreach (Artifact artifact in artifacts)
                { 
                    if (artifact == null)
                    { 
                        continue; 
                    } 
    
                    AssociatedWorkItemInfo awii = new AssociatedWorkItemInfo();
    
                    // Convert the name/value pairs into strongly-typed objects containing the work item info 
                    foreach (ExtendedAttribute ea in artifact.ExtendedAttributes)
                    { 
                        if (String.Equals(ea.Name, "System.Id", StringComparison.OrdinalIgnoreCase)) 
                        { 
                            int workItemId; 
    
                            if (Int32.TryParse(ea.Value, out workItemId))
                            { 
                                awii.m_id = workItemId; 
                            } 
                        } 
                        else if (String.Equals(ea.Name, "System.Title", StringComparison.OrdinalIgnoreCase)) 
                        { 
                            awii.m_title = ea.Value; 
                        } 
                        else if (String.Equals(ea.Name, "System.AssignedTo", StringComparison.OrdinalIgnoreCase)) 
                        { 
                            awii.m_assignedTo = ea.Value; 
                        } 
                        else if (String.Equals(ea.Name, "System.State", StringComparison.OrdinalIgnoreCase)) 
                        { 
                            awii.m_state = ea.Value; 
                        } 
                        else if (String.Equals(ea.Name, "System.WorkItemType", StringComparison.OrdinalIgnoreCase)) 
                        { 
                            awii.m_type = ea.Value; 
                        } 
                    } 
    
                    Debug.Assert(0 != awii.m_id, "Unable to decode artifact into AssociatedWorkItemInfo object."); 
    
                    if (0 != awii.m_id)
                    { 
                        toReturn.Add(awii); 
                    } 
                } 
    
                return toReturn.ToArray(); 
            } 
    
            private int m_id; 
            private String m_title; 
            private String m_assignedTo; 
            private String m_type; 
            private String m_state; 
        } 
    }
  • Buck Hodges

    How to get the TFS objects used in our own UI integration

    • 3 Comments

    Philip, a dev on version control, recently helped with a question on how to get the TFS objects we use in our UI.  I thought I’d post since others may find it useful.

    We recently had a request from a customer for a VS add-in that would be able to access the same TfsTeamProjectCollection and VersionControlServer objects that our own UI integration (such as the Team Explorer and Pending Changes toolwindow) are using. In this particular case the customer wanted to hook the BeforeCheckinPendingChange event from the VersionControlServer object and take a specific action when that occurred. But the framework shown in this piece of sample code is generic -- you can use it to get the very same VersionControlServer or WorkItemStore object that our integration is using to connect to TFS.

    The trick here is to hook the ProjectContextChanged event on the TeamFoundationServerExt extensibility object. While that extensibility point won't give you the TfsTeamProjectCollection object directly, we can ask the TfsTeamProjectCollectionFactory's static GetTeamProjectCollection method to retrieve it from a runtime cache. The cache is keyed by URI -- which (handily) is provided by TeamFoundationServerExt. By the time the ProjectContextChanged event fires, the ActiveProjectContext.DomainUri property has already been updated.

    All the services in the TFS client object model are owned by the TfsTeamProjectCollection. Once we have it, we can call GetService to request the VersionControlServer object. There's only one per TfsTeamProjectCollection; the same holds true for WorkItemStore, IBuildServer, or any of the other client object model services you may be familiar with.

    Happy extending!

    using System;
    using System.Diagnostics;
    using System.Windows.Forms;
    using Extensibility;
    using EnvDTE;
    using EnvDTE80;
    using Microsoft.TeamFoundation.Common;
    using Microsoft.TeamFoundation.Client;
    using Microsoft.TeamFoundation.VersionControl.Common;
    using Microsoft.TeamFoundation.VersionControl.Client;
    using Microsoft.VisualStudio.TeamFoundation;
    using Microsoft.VisualStudio.TeamFoundation.VersionControl;

    namespace MyAddin1
    {
        /// <summary>The object for implementing an Add-in.</summary>
        /// <seealso class='IDTExtensibility2' />
       
    public class Connect : IDTExtensibility2
        {
            /// <summary>Implements the constructor for the Add-in object. Place your initialization code within this method.</summary>
           
    public Connect()
            {
            }

            /// <summary>Implements the OnConnection method of the IDTExtensibility2 interface. Receives notification that the Add-in is being loaded.</summary>
            /// <param term='application'>
    Root object of the host application.</param>
            /// <param term='connectMode'>
    Describes how the Add-in is being loaded.</param>
            /// <param term='addInInst'>
    Object representing this Add-in.</param>
            /// <seealso class='IDTExtensibility2' />
           
    public void OnConnection(object application, ext_ConnectMode connectMode, object addInInst, ref Array custom)
            {
                _applicationObject = (DTE2)application;
                _addInInstance = (AddIn)addInInst;

                try
               
    {
                    m_tfsExt = _applicationObject.GetObject("Microsoft.VisualStudio.TeamFoundation.TeamFoundationServerExt") as TeamFoundationServerExt;

                    if (null != m_tfsExt)
                    {
                        m_tfsExt.ProjectContextChanged += new EventHandler(m_tfsExt_ProjectContextChanged);

                        if (null != m_tfsExt.ActiveProjectContext)
                        {
                            // Run the event handler without the event actually having fired, so we pick up the initial state.
                           
    m_tfsExt_ProjectContextChanged(null, EventArgs.Empty);
                        }
                    }
                }
                catch (Exception ex)
                {
                    MessageBox.Show(ex.Message);
                }
            }

            /// <summary>Implements the OnDisconnection method of the IDTExtensibility2 interface. Receives notification that the Add-in is being unloaded.</summary>
            /// <param term='disconnectMode'>
    Describes how the Add-in is being unloaded.</param>
            /// <param term='custom'>
    Array of parameters that are host application specific.</param>
            /// <seealso class='IDTExtensibility2' />
           
    public void OnDisconnection(ext_DisconnectMode disconnectMode, ref Array custom)
            {
                // Unhook the ProjectContextChanged event handler.
               
    if (null != m_tfsExt)
                {
                    m_tfsExt.ProjectContextChanged -= new EventHandler(m_tfsExt_ProjectContextChanged);
                    m_tfsExt = null;
                }
            }

            /// <summary>Implements the OnAddInsUpdate method of the IDTExtensibility2 interface. Receives notification when the collection of Add-ins has changed.</summary>
            /// <param term='custom'>
    Array of parameters that are host application specific.</param>
            /// <seealso class='IDTExtensibility2' />       
           
    public void OnAddInsUpdate(ref Array custom)
            {
            }

            /// <summary>Implements the OnStartupComplete method of the IDTExtensibility2 interface. Receives notification that the host application has completed loading.</summary>
            /// <param term='custom'>
    Array of parameters that are host application specific.</param>
            /// <seealso class='IDTExtensibility2' />
           
    public void OnStartupComplete(ref Array custom)
            {
            }

            /// <summary>Implements the OnBeginShutdown method of the IDTExtensibility2 interface. Receives notification that the host application is being unloaded.</summary>
            /// <param term='custom'>
    Array of parameters that are host application specific.</param>
            /// <seealso class='IDTExtensibility2' />
           
    public void OnBeginShutdown(ref Array custom)
            {
            }

            /// <summary>
            ///
    Raised by the TFS Visual Studio integration package when the active project context changes.
           
    /// </summary>
            /// <param name="sender"></param>
            /// <param name="e"></param>
           
    private void m_tfsExt_ProjectContextChanged(Object sender, EventArgs e)
            {
                try
               
    {
                    if (null != m_tfsExt.ActiveProjectContext &&
                        !String.IsNullOrEmpty(m_tfsExt.ActiveProjectContext.DomainUri))
                    {
                        SwitchToTfs(TfsTeamProjectCollectionFactory.GetTeamProjectCollection(new Uri(m_tfsExt.ActiveProjectContext.DomainUri)));
                    }
                    else
                   
    {
                        SwitchToTfs(null);
                    }
                }
                catch (Exception ex)
                {
                    MessageBox.Show(ex.Message);
                }
            }

            private void SwitchToTfs(TfsTeamProjectCollection tfs)
            {
                if (Object.ReferenceEquals(m_tfs, tfs))
                {
                    // No work to do; could be a team project switch only
                   
    return;
                }

                if (null != m_tfs)
                {
                    m_tfs.GetService<VersionControlServer>().BeforeCheckinPendingChange -= new ProcessingChangeEventHandler(VersionControlServer_BeforeCheckinPendingChange);
                    m_tfs = null;
                }

                if (null != tfs)
                {
                    m_tfs = tfs;
                    m_tfs.GetService<VersionControlServer>().BeforeCheckinPendingChange += new ProcessingChangeEventHandler(VersionControlServer_BeforeCheckinPendingChange);               
                }
            }

            private void VersionControlServer_BeforeCheckinPendingChange(Object sender, ProcessingChangeEventArgs e)
            {
                if (null != e.PendingChange &&
                    !String.IsNullOrEmpty(e.PendingChange.ServerItem))
                {
                    MessageBox.Show("About to check in: " + e.PendingChange.ServerItem);
                }
            }
           
            private DTE2 _applicationObject;
            private AddIn _addInInstance;

            private TeamFoundationServerExt m_tfsExt;
            private TfsTeamProjectCollection m_tfs;
        }
    }

  • Buck Hodges

    A tool to find duplicate copies in a build

    • 2 Comments

    As part of our builds, quite a few projects copy files to the binaries directory or other locations.  These can be anything from image files to test scripts.  To have our builds complete more quickly, we use the multi-process option (/maxcpucount) of msbuild to build projects in parallel.

    This all sounds normal, so what’s the problem?  In a large team, people will sometimes inadvertently add statements to different project files that copy files to the same destination.  When those project files have no references to each other, directly or indirectly, msbuild may build them in parallel.  If it does happen to run those projects in parallel on different nodes and the copies happen at the same time, the build breaks because one copy succeeds and one fails.  Since the timing is not going to be the same on every build, the result is random build breaks.  Build breaks suck.  They drain the productivity of the team and are frustrating.

    Whether the build is continuous integration or gated checkin, these breaks may happen randomly.  They are most likely to happen on incremental builds where the percentage of time spent during the build on doing copies is much higher than a clean build.  Tracking them down as they happen is painful.

    So, I wrote a simple tool to find cases in the log where the destination is the same for more than one copy operation.  The comment in the header explains what the code is looking for.  Running this on the normal verbosity msbuild logs from a clean build ensures that all of the copies are in the log for analysis.  We also build what we call partitions separately, resulting in the number of log files being a multiple of the number of partitions being built (a partition is a subset of the source and is typically a top-level directory in the branch).

    In our internal builds, we record multiple log files for our builds, including minimal, normal, and detailed.  When there’s a problem, you can start with the smaller build logs and increase to the more verbose logging as needed.

    I’m posting this for any of you who might run into the same thing.  Keep in mind that there are other things, such as antivirus software, that can interfere with the build process and result in errors for files being copied.

    using System;
    using System.Collections.Generic;
    using System.Linq;
    using System.IO;
    using System.Text;
    using System.Text.RegularExpressions;
    using System.Threading.Tasks;
    
    /*
     * This tool finds cases where more than one file is copied to the same target.  This will cause
     * build breaks when msbuild executes the copies in parallel because the copies are independent
     * (there are no dependencies).  This typically occurs in incremental builds because incremental
     * builds do a lot less work (not nearly as much to build), resulting in the copies being a much
     * higher percentage of the build activities and more likely to collide.  Gated checkin,
     * continuous integration, and developer/tester builds are almost always incremental, not clean.
     * These issues are still possible in regular clean builds, such as done nightly by the build lab.
     * 
     * These race conditions are difficult to debug manually.  Since msbuild records all of the copies
     * made via the copy task, we can use the log file to identify cases where the same destination
     * path is used in more than one copy.
     * 
     * Use the *.normal.* logs from a clean build with this tool.
     * 
     * The best thing to do is to ensure that each file copy to a particular destination is done by
     * one and only one project.  When that is the case, you are guaranteed not to have problems
     * with two copies colliding and breaking your build.
     * 
     * Here's example output from buildr.suitesrc.normal.log that shows a copy failure.  Here two
     * copies were executed in parallel and the second one failed, causing the build to fail.
     * 
        48>Project "D:\a1\dd\alm\tfs_core\Admin\Servicing\Data\dirs.proj" (48) is building "D:\a1\dd\alm\tfs_core\Admin\Servicing\Data\Azure\Microsoft.TeamFoundation.Data.Azure.csproj" (55) on node 8 (BuildLinked target(s)).
    
        55>_CopyOutOfDateSourceItemsToOutputDirectory:
             Copying file from "D:\a1\dd\alm\tfs_core\Admin\Servicing\Data\ReleaseManifest.xml" to "D:\a1\binaries.x86ret\bin\i386\ReleaseManifest.xml".
    
    
        48>Project "D:\a1\dd\alm\tfs_core\Admin\Servicing\Data\dirs.proj" (48) is building "D:\a1\dd\alm\tfs_core\Admin\Servicing\Data\SqlServer\Microsoft.TeamFoundation.Data.csproj" (53) on node 4 (BuildLinked target(s)).
    
        53>_CopyOutOfDateSourceItemsToOutputDirectory:
             Copying file from "D:\a1\dd\alm\tfs_core\Admin\Servicing\Data\ReleaseManifest.xml" to "D:\a1\binaries.x86ret\bin\i386\ReleaseManifest.xml".
    
        53>D:\a1\dd\tools\x86\managed\v4.5\Microsoft.Common.targets(3516,5): error MSB3021: Unable to copy file "D:\a1\dd\alm\tfs_core\Admin\Servicing\Data\ReleaseManifest.xml" to "D:\a1\binaries.x86ret\bin\i386\ReleaseManifest.xml". Access to the path 'D:\a1\binaries.x86ret\bin\i386\ReleaseManifest.xml' is denied. [D:\a1\dd\alm\tfs_core\Admin\Servicing\Data\SqlServer\Microsoft.TeamFoundation.Data.csproj]
      
     * 
     * Note that there may be multiple copies in a sequence.
     * 
        291>_CopyOutOfDateSourceItemsToOutputDirectoryAlways:
             Copying file from "D:\a1\dd\suitesrc\TFS\common\deploytools\httpcfg.exe" to "D:\a1\binaries.x86ret\SuiteBin\i386\TFS\Tests\httpcfg.exe".
             Copying file from "D:\a1\dd\suitesrc\TFS\common\deploytools\makecert.exe" to "D:\a1\binaries.x86ret\SuiteBin\i386\TFS\Tests\makecert.exe".
             Copying file from "D:\a1\dd\suitesrc\TFS\common\deploytools\winhttpcertcfg.exe" to "D:\a1\binaries.x86ret\SuiteBin\i386\TFS\Tests\winhttpcertcfg.exe".
           CopyFilesToOutputDirectory:
             Copying file from "D:\int\641\194\suitesrc\tfshttpsconfig.csproj_80399372\objr\x86\TfsHttpsConfig.exe" to "D:\a1\binaries.x86ret\SuiteBin\i386\TFS\Tests\TfsHttpsConfig.exe".
    
     * Nodes are reused by msbuild.  The result is that a given may process many projects, so it's not
     * possible to scan and pair up all of the nodes and project files at once.  In the code below, 
     * you will see that it always tracks the most recent node for that reason.
     * 
     */
    
    namespace FindBadCopies
    {
        class Program
        {
            static void Main(string[] args)
            {
                if (args.Length < 1)
                {
                    Console.WriteLine("Usage: findbadcopies <logfile>\r\n");
                    Console.WriteLine(
    @"This tool scans a build log, such as buildr.suitesrc.normal.log, and produces a
    list of file paths that are the targets of more than one copy and shows which
    project files initiated each copy.  These redundant file copies are prone to
    fail periodically in incremental builds, such as gated check ins and CI builds,
    because copies are a higher percentage of the operations in the build, making
    it more likely that two collide.");
    
                    return;
                }
    
                ProcessFile(args[0]);
            }
    
            private static void ProcessFile(String fileName)
            {
                Dictionary<int, String> nodeTable = new Dictionary<int, String>(1000);
                Dictionary<String, int> pathTable = new Dictionary<String, int>(1000, StringComparer.InvariantCultureIgnoreCase);
                String previousLine;
    
                string[] text = File.ReadAllLines(fileName);
    
                // Process all of the lines in the file, skipping the first line (we need the previous line,
                // and the first line in the file isn't important to this tool).
                int lastNode = 0;
                for (int i = 1; i < text.Length; i++)
                {
                    previousLine = text[i - 1];
    
                    // Record most recent node.  The text that appears with it can be different
                    // (see sample log data).
                    string prevLinePattern = @"([0-9]+)[>]";
                    Match match = Regex.Match(previousLine, prevLinePattern, RegexOptions.IgnoreCase);
                    if (match.Success)
                    {
                        lastNode = Int32.Parse(match.Groups[1].Value);
                    }
    
                    // If the line is recording the start of a project, add it to the table.
                    string pattern = @"[0-9]+[>]Project ""[^""]+"" \([0-9]+\) is building ""([^""]+)"" \(([0-9]+)\)";
                    match = Regex.Match(text[i], pattern, RegexOptions.IgnoreCase);
                    if (match.Success)
                    {
                        int node = Int32.Parse(match.Groups[2].Value);
                        String projectPath = Path.GetFullPath(match.Groups[1].Value);
    
                        // Because nodes are reused, we are only keeping the project path for the most recent use
                        // of a given node.
                        nodeTable[node] = projectPath;
    
                        // If we matched a project line, it can't be a copy line.
                        continue;
                    }
    
                    // If the line is one that records a copy, see if there was an earlier copy made to
                    // the same target path.  First, try the output of a copying task.
                    string copyingPattern = @"Copying file from ""[^""]+"" to ""([^""]+)""";
                    match = Regex.Match(text[i], copyingPattern, RegexOptions.IgnoreCase);
                    if (match.Success)
                    {
                        String targetPath = null;
                        try
                        {
                            targetPath = Path.GetFullPath(match.Groups[1].Value);
                        }
                        catch (Exception e)
                        {
                            // There is a file in the test tree that uses non-English chars that causes
                            // GetFullPath() to throw (TODO: understand why), so we keep the raw text.
                            // Console.WriteLine(match.Groups[1].Value);
                            targetPath = match.Groups[1].Value;
                        }
    
                        // If we have already seen the target path, then we have a duplicate copy path
                        // target to report.
                        int otherNode;
                        if (pathTable.TryGetValue(targetPath, out otherNode))
                        {
                            Console.ForegroundColor = ConsoleColor.Cyan;
                            Console.WriteLine("{0}", targetPath);
                            Console.ResetColor();
    
                            Console.WriteLine("      {0}", nodeTable[otherNode]);
                            Console.WriteLine("      {0}", nodeTable[lastNode]);
                            Console.WriteLine();
                        }
    
                        pathTable[targetPath] = lastNode;
                    }
                }
            }
        }
    }
Page 2 of 23 (563 items) 12345»