Curtis Pettit's Blog

Administrator Permissions are not enough for WIT migration

curtisp — Fri, 05 Dec 2008 18:15:43 GMT

I don't usually stick my nose into Work Item Tracking issues because its not my area. However, this issue is the cause of so much confusion that it reflects poorly on the whole TFS to TFS migration tool. When you are running a Work Item Tracking (WIT) migration with the TFS to TFS migration tool, Administrator level permissions are not enough. You need to be a member of the Service Accounts group.

Only members of the Service Accounts group are allowed to bypass the rules check for the WIT web services. The WIT migration session uses this ability to do things like attribute changes to people who are no longer valid users (left the company). Even a TFS server Admin cannot bypass those rules, and the Migration tool always makes the web service call with the bypass flag set (does not attempt to determine when its needed). If the user running the migration isn't a member of the Service Accounts group the WIT migration will crash and be unable to continue.

The Service Accounts group is such a special group that permissions cannot be set though the UI. You need to use the command line TFSecurity.exe from the install location.

TFSSecurity.exe /g+ "Service Accounts" "domain\userName" /server:http://serverName:8080

Does the TFS to TFS migration tool retain Source Control History?

curtisp — Wed, 03 Dec 2008 23:54:22 GMT

The short answer is yes. Every version of every file and folder that is mapped for migration will be downloaded to the client then checked into the target server.

However, it may not be preserved in the way you expect. The server does not allow the TFS to TFS migration tool (or any migration tool) to specify the original checkin time or changeset ID. These are instead applied as of the time of the migration checkin, as it would be for any other checkin. Please keep in mind that reports generated from version control history will show that all the migrated changes happened at the time of the migration. While changeset links for migrated Work Items will be updated with the linking session, external references to changeset numbers (such as in emails) will also be invalid.

Integration history is also not preserved. Integration history is basically a record of how merge conflicts were resolved, for the purpose of determining which changes to flag as conflicts in future merges. Unfortunately, there is no way to query the source server as to how the conflicts were originally resolved. The migration tool will still migrate the merge, but instead of resolving any conflicts in the way they were originally resolved, it uploads the file content from the source server and instructs TFS to resolve the conflict by accepting this version. The result is that after migration, TFS cannot tell which branch the resolved, migrated content came from. This could cause extra conflicts in future merges, so we recommend that your merges are up to date before attempting a migration.

VSSConverter Improvements for 2008 SP1

curtisp — Wed, 01 Oct 2008 17:11:34 GMT

We made a big investment in VSSConverter SP1. The most important change was to create a unique ID for each item in VSS. Without this it is impossible to track the history of a file or folder across renames and moves. Previous versions of VSSConverter would frequently lose track of files that had been renamed, making it impossible to migrate the rest of the history.

Related to this are the improvements made to client timestamp issues. The timestamp of all actions committed to VSS are recorded from the client, rather than the "server". If two clients are in different time zones (or one just has the clock set wrong) it is possible to create a sequence of actions that doesn't make sense, such as editing a file before it is added or after deleting it. Now that we can associate the entire history of the file to its parent we can override the VSS timestamp and put it into a logical order. By logical order I mean that the file (and its parent folder) exists and is not deleted before trying to commit a checkin.

SP1 VSSConverter is smarter about the way multiple VSS actions are batched into a single TFS changeset. On top of making sure there are no conflicts and keeping track of the checkin comments and owners, it will now limit the time span of the changeset to 10 minutes. The TFS checkin comment will also now reflect the time span over which the actions originally happened, instead of just the timestamp of the last action.

The "analyze" option catches many more potential problems then earlier versions. Previously it was doing a simple scan that could not detect issues within the history of a VSS item. Now, the analyze option is running the same algorithm that it does before migration. This does take longer, but it is the only way to report problems (like client timestamp issues) that could previously only be detected at migration time. In the long run I think this will save time.

By using the Team Foundation Client object model instead of making direct calls to the web services, Sp1 VSSConverter was able to cut the number of workspaces down from ten to one. This allowed us to get rid of the threading, which means improved performance and no more deadlocks. VSSConverter can do the harder history analysis (resulting in more migrate-able actions) on the same database in roughly the same amount of time.

VSSConverter will now attempt to remove old VSS source control bindings from the solution so that Team Explorer will immediately recognize that it needs to bind the solution.

These are just some of the highlights; there are too many bug fixes and other improvements to list them all here. Suffice to say, I strongly recommend that anyone who wants to move from SourceSafe to Team Foundation use the 2008 Sp1 or later. You can find it here

Planning for VSSConverter Performance

curtisp — Thu, 28 Aug 2008 19:23:00 GMT

A lot of people want to know how long it will take to migrate their VSS database into TFS using VSSConverter. There many factors that affect this, so it is impossible to give a truly accurate estimate. However, I believe I have done enough migrations to give a ballpark that will be useful for most customers. They are broken up into categories according to when you would be away from the keyboard. (so the migration doesn’t affect your productivity)

There is a closer correlation between the number of actions to migration time than there is the database file size, but I will give both because number of actions is hard to guess without actually running the converter. By “action” I mean a change recorded in VSS such as ”Added”, “Checked in”, or ” Renamed”.

“Over Lunch” 0-2 Hours

Database File Size (in Gigabytes): 0-2

Number of actions: 0- 20,000

“Over Night” 2-14 Hours

Database File Size (in Gigabytes): 2- 8

Number of actions: 20,000- 100,000

“Over the Weekend” 14-36 Hours

Database File Size (in Gigabytes): 5-10

Number of actions: 100,000- 700,000

Longer Migrations

The largest database I was able to get contained 1.6 million actions in 20 gigs and took roughly 20 days to migrate. If you are in a situation like this where you are not able to stop using the VSS database for long enough to complete the migration you have two options, smaller migrations or incremental migration.

Smaller Migrations

You can migrate smaller parts of the database at a time, by mapping a deeper path. (IE “$/project1”rather than “$/”) In this scenario you will run the VSSConverter once for each chunk of the database. You must make sure that the paths are completely unique.

The trade off is if items are moved from one part of the database to another they will be migrated as separate items in TFS. i.e. If “$/Project1/subproject” were moved to “$/Project2/subproject” and you were to map “$/Project1/” in the first migration and “$/Project2/” in the second migration. The item “subProject” would be two separate items in TFS. (Querying history on one would not show history from the other).

For more information on mapping paths please see http://msdn.microsoft.com/en-us/library/ms253161.aspx

Incremental Migrations

Or can you perform multiple Incremental migrations. This is where you create a backup copy of your VSS database and start a migration from that, let that run to completion, while using your production copy. Then start another migration from the production copy (while its idle) with the same mappings. The migration will start migrating from the last change it checked into TFS, thus migrating changes made since the backup was taken.

The trade off is that there is a cost each time a migration is started. The VSSConverter must query the VSS database and analyze all the changes, then use that information to determine the last change that it migrated before it can migrate the next change. If you have a large database and cannot dedicate long periods of time to the migration, there will be little net progress.

While we fixed all the bugs I was able to find in 2008 SP1, Incremental migration did have a higher incidence of problems than a straight forward migration.

For more information on Incremental Migration see http://msdn.microsoft.com/en-us/library/ms400695.aspx

Welcome to my Blog

curtisp — Fri, 15 Aug 2008 18:23:00 GMT

About the blog:

The goal of this blog is helping customers work their problems. There are often common issues/questions and it’s easier to blog it once than answer a bunch of different e-mails and message board posts. Blogging it also makes it a lot more discoverable for you.

Most of the topics I cover will have to do with moving source control history into Microsoft Team Foundation Server from some competing version control system. I will occasionally talk about other source control topics and if I have enough time, I will also share my thoughts about software testing in general.

About Me:

I graduated from Florida Tech just over 2 years ago with a BS in Software Engineering. There I studied software testing under Dr. Cem Kaner and worked for one semester at the Center for Software Testing Education and Research. Upon graduating I joined the Microsoft Team Foundation Server Version Control test team. I have been the main tester for the Version Control components of the migration toolkit, and the 2008 SP1 release of VSSConverter.

Thanks for reading

Curtis Pettit