Welcome to MSDN Blogs Sign in | Join | Help

Synchronizing two TFS servers

I've seen a lot of questions lately about synchronizing one TFS server with another.  I think the heightened interest in this scenario has been brought on by the announcement of the TFS Basic installation option.  People are starting to think of new and cool ways they can use TFS and they want to be able to synchronize them with their "central" TFS server.

We don't have a solution for this today.  I'm going to be watching the feedback over the next several months to judge how important this is to people to decide where to go with it.

Synchronizing TFS is potentially a VERY complicated problem depending on what all you want to synchronize and how much fidelity you want.  The issue is that there is a lot to synchronize.  Source code control, while posing some interesting scaling and branch/merge fidelity challenges is reasonably straight forward.  Work items also isn't too bad but there's a lot more stuff.  For example, process template changes, tests and test results, build definitions, permissions, and on and on.  We would need to think through the scenario very carefully and make sure we understand what all needs to be synchronized for the scenario.

We've got a start on this problem.  I've talked several times on my blog about a TFS to TFS synchronization tool we've been building for the past 6-9 months that we're using to drive new requirements for our migration and synchronization toolkit and to support our ability to dogfood very recent builds of TFS while not subjecting the broader division to the instability and increased downtime (for upgrades, patching, etc).

It's possible (and I say possible because I don't mean easy or straightforward) that we could move this tool towards supporting the scenario of a loosely connected star topology of TFS servers.  That would allow you to have TFS Basic on your laptop and have all that TFS capability while offline and later sync it up with the "mother" server.

I'm definitely interested in feedback on how valuable this would be to people.  I'm also interested in feedback on what the relative priority of synchronizaton of the various pieces of TFS data would be.

Brian

Published Thursday, October 08, 2009 8:40 AM by bharry

Comment Notification

If you would like to receive an email when updates are made to this post, please register here

Subscribe to this post's comments using RSS

Comments

# re: Synchronizing two TFS servers

I think one important point here is that for a lot of the data you describe, a one way sync would probably be perfectly fine. A solution where you can only edit and change a subset of the total data on TFS on local client installations would be fine. So, no need to be able to change build efinitions, permissions, process templates etc etc on my offline client.

Having said that, this obviously would still be an enormously complicated task... But way cool!

I still have hope that this once anounced grand Meshification will happen, i.e. that many products will use the Live Mesh infrastructure to do synch. If you look into this seriously, it would be great if this would somehow work togeter, if in fact Live Mesh ever gets into a proper product.

Thursday, October 08, 2009 9:17 AM by davidacoder

# re: Synchronizing two TFS servers

As an erstwhile educator and somebody who has mentored many junior programmers in industry, My first thought on reading this was that the personal TFS repository imagined on the corporate programmer's laptop would be just right for the amateur or student programmer.  They should be learning good process from the beginning so I don't have to tie them down and pump it down their throats until the discover that it actually makes their work easier.

Thursday, October 08, 2009 11:05 AM by Steve Nuchia

# re: Synchronizing two TFS servers

I think a lot of this has its roots in the git world.  I love TFS as an overall solution, but the distributed branching of Git is pretty powerful.  I'd love to see, at first, branching of source between TFS installations (enabled locally by basic).  Work item integration and branching policy would be enforced by the merge into the "mother" server as you say.

Thursday, October 08, 2009 5:34 PM by Ryan Cromwell

# re: Synchronizing two TFS servers

Offline support is essentially. I recently helped drive TFS adoption from FogBugz for Work Items at my employer. The two big sellers were the diversity of clients and the offline editing possible, especially in Excel. Local server would be 'offline on steroids' for me.

As for the fun of git... I've been in the software game for a long time. I've worked on complex issues like spherical harmonics and speech recognition. I could never wrap my head around git at the job where I used it, and I was explicitly told to not to try to understand it. So, if TFS source control goes the way of git, it had better have an intuitive UX.

Thursday, October 08, 2009 10:32 PM by Ted Howard

# re: Synchronizing two TFS servers

For me the abilty to have a local source code repository that I later could merge into the central source code repository would really be of great value. Have not worked with GIT but it sounds very tempting.

WorkItems, process templates and builds would be secondary.

Friday, October 09, 2009 2:46 AM by Håkan Forss

# re: Synchronizing two TFS servers

What about using TFS sync instead of using TFS Proxy? Wouldn't the TFS Proxy be obsolete?

TFS Sync is interesting in cases where the bandwidth between two or more locations is not stable?!

What about just syncing on SQL Server (active-active cluster?) is this a way to go?  

Friday, October 09, 2009 8:23 AM by Daniel K.

# re: Synchronizing two TFS servers

I'm not sure it would make the Proxy obsolete because any way you slice it the Proxy would still be much easier to manage - it's a single master system vs a multi-master system.  But I'm not sure there would be many people who would choose to use both a replica and a proxy.

While a TFS sync solution would help with limited bandwidth situations, so does a proxy.  Personally, I'd use a proxy to address bandwidth but replication to address intermittent connectivity.

It depends on what you mean about syncing a SQL Server Active-Active cluster.  If you mean using SQL replication, then that's not likely to be so simple.  There are application level ramifications of SQL replication but maybe you mean something else.

Brian

Friday, October 09, 2009 8:34 AM by bharry

# re: Synchronizing two TFS servers

Just to give you some Feedback, things which forced us to use git (not everybody was happy with that solution, especially the pms)

going to the customer => till network is approved, proxy configurations, content filtering, firewalls .... that need months, we still have to be able to continue working

working on the train => not everywhere is network coverage ...

i really like teambuild and the statistics. we really liked workitems and scrum templates.

work item tracking, bugtracking is now done by with a small custom webapplication which has google gears offline support.

I know this has history since vss and is hard to break. But try to stop thinking about file versions. Think about the content within files and try to evolve to a "content tracker"

Friday, October 09, 2009 10:50 AM by thomas

# re: Synchronizing two TFS servers

One of the most common scenarios is where your team is located in three different countries and yet you want to be able to have updated source at all three locations, even if it means syncing up once a day.  So something simple like source sync would be sufficient in such a case.  Can this be done using proxy or replication or is this something we need to wait for?

Saturday, October 10, 2009 2:23 AM by Danish

# re: Synchronizing two TFS servers

I work in a large Health Care company mainly as a development lead\Architect.

Our company works with numerous offsite developers and contractors and I would love to be able to use TFS Basic on their local laptops and when they come back to the office "sync" to the main TFS server.

We tend to bring SMEs onsite for a week or two and then let them go work remotely.

Obviously I don't expect them to be able to easily sync their changes back to TFS if they've made major changes (reachitectures, brances, builds).

However, trying to setup private branches with remote proxies with TFS 2008 has been a real pain...so much so that we have to find funding to keep the SMEs onsite.

We've explored GIT...and we have a TFS to GIT sync somewhat setup...but it's not as reliable as we need. Plus it doesn't handle workitems.

It would be a win-win-win if we could give these folks the ability to work on their local machines and from time to time sync back up to our corporate TFS server.

I will say that our number 1 request would be for source code sync. Closely followed by Work item "sync".

The more we've run into trying to get TSWA to work at scale the more "sync" concept seems to be the way to go.

With the new testing tools...it'd be fantastic to have the ability to have 1 remote tester in each geography for internationalization testing.

The more I think about "sync" the more excited I'm getting.

BTW....if TFS Synch is as easy as Ford Sync....and it "just works" you have a real winner on your hands.

Sunday, October 11, 2009 7:21 PM by Gary Herman

# re: Synchronizing two TFS servers

TFS will need to have an offline strategy eventually anyway (there are justs so many scenarios that would make use of it) and the post-2010 version should at least start some work on it. TFS with offline/sync capabilities would be an absolute killer - hope you guys can find a way to make it happen although it is a tough nut to crack for sure.

Monday, October 12, 2009 3:34 AM by Borek

# re: Synchronizing two TFS servers

I'm just beginning the process of learning about 2010 (we are looking for alternatives to vss and subversion).

The simplified installation is going to be quite helpful...2008 has been a big pain...and in demoing TFS i just use the free VPC you guys ship.

I can see different groups in our development shop wanting their own TFS environments to manage. Sometimes they just want to have their own servers.

However...the ability to sycronize them would be very helpful.

2010 seems to be releasing pretty soon...so all of this talk about a post 2010 version seems crazy.

If you won't have a solution for sycronization when 2010 launches...please, PLEASE don't wait till the next release...knowing microsoft that will be 3 long, LONG years.

atleast give us a point solution...that will work in 30% of the cases...even with BIG limitations...having something is better than waiting on the promise of the feature coming in the next version.

Monday, October 12, 2009 5:26 AM by Mike Mercel

# re: Synchronizing two TFS servers

We have 2 TFS 2008 servers and we need to share source code between both servers. We've tried using LiveMesh, Windows XP sync tool, database mirroring, and nightly batch scripts.

We've also tried using OnePulse and the TFS to TFS Migration tool.

All of our efforts have been fruitless and haven't had any successful results.

We attempted to "fix" the TFS to TFS Migration tool but quickly gave up. It appears Microsoft has abandoned this tool....which is a real shame.

The need for keeping 2 or more TFS servers synced up with each other is a pretty big need.

It's a gapping hole that needs to be filled.

It is telling that no patner to date has ventured into this space....either the problem is too hard or partner's don't see their solutions being able to profitable over multiple TFS releases.

So Brian I'd really like to see what we're supposed to do in the mean time between now and TFS.20XX

Wednesday, October 14, 2009 7:43 AM by DHL_Harsha

# re: Synchronizing two TFS servers

I was just reading Buck's blog:

http://blogs.msdn.com/buckh/archive/2009/10/21/what-features-are-in-tfs-basic-2010.aspx

And it's awesome to hear that there's no hard 5 limit user for TFS Basic.

Now that TFS Basic is free...we have an EA...I want to have each of my developers install TFS Basic on their machines and use their local machines as a repository and build machine. The reporting parts that aren't included in TFS basic aren't of interest to us. We just have too many small projects...and we don't have reporting today...so I doubt we'll need it in the next 2 years.

I'm thinking of writing a CI team build script that each developer would have on their machines that would copy and check in the files on every other dev TFS machine in our org. Basically I want all of the dev machines to have identical source code as every other developer in the organization.

We couldn't afford a TFS server before...so now my stakeholders will be very excited to see that TFS is free and can run on every dev workstation.

Buck mentions the 10 user limit on client workstations...that 10 user limit is just a tad limiting...but if it's just 10 concurrent users then it's highly unlikely we'll have 10 devs all working on 1 developer’s machine/TFS server.

What help can I get for writing the CI Build script to sync the TFS source code...I believe TFS Build 2010 uses WW 4.0 so i can't imagine it's too difficult?

Thanks for making TFS free! Now just to solve the sycronization challenge (shouldn't be too difficult...should it?).

Wednesday, October 21, 2009 8:52 PM by Jon

# re: Synchronizing two TFS servers

The 10 connection limit that Buck mentions is an OS limitation and is a "soft" limit.  If yoh hit it, subsequent connections will queue until there is roon in the 10 connection pool.  And yes, it refers to simultaneous connections.

Yes, TFS 2010 build uses WF 4.0.  You can use all of the standard WF development tools.  I don't expect what you are trying to do will be too difficult.

Brian

Thursday, October 22, 2009 6:43 AM by bharry

# re: Synchronizing two TFS servers

I seem to be not getting some of my blog comment notifications (maybe the spam filter is eating them) because I see some comments here that I didn't know about.

DHL, yes, server synchronization is a much harder problem than it at first appears.  The Migration and Synchronization tool kit hasn't been abandoned but we came to realize that it was not up to the task.  For the last 9 months we've been working on the next generation of it and it should be available within the next month or so.  I think you will find it much closer to what you need.

Brian

Thursday, October 22, 2009 6:45 AM by bharry

# re: Synchronizing two TFS servers

a team project contains source + workitems + reports + documents + security users.

for internal company project needed all components on 2 different domains.

need tool to keep n TFS servers together.

priority of synchronizaton=

1) Reporting

2) Documents

3) Security

4) Source

5) Workitems

will the MS tool keep together all team project data?

Sunday, October 25, 2009 2:37 PM by Tian

# re: Synchronizing two TFS servers

The tool we are working on right now is focued on Source and Workitems.  Generally that's what people most want synchronized across servers.  I'd be interested in learning more about your scenario.  In many scenarios, synchronizing permission isn't that interesting because it's different groups of people and therefore different permissions.  Reporting is sometimes unnecessary to sync if all the other data is synced because the independent reports each report over all the data.  Documents would be nice but not something we are currently focusing on.  Maybe in a second phase.

Brian

Monday, October 26, 2009 5:29 AM by bharry

# re: Synchronizing two TFS servers

I read on grant's blog about the internal TFS 2010 server (http://blogs.msdn.com/granth/archive/2009/10/19/vsts-pioneer-tfs2010-dogfood-server-upgrading.aspx)

We have a department TFS server with 30 users on it...and we have a test group ~6 users who want to start using TFS 2010 for testing. We also have BAs that want to start using the new use case modeling/requirement templates.

We can't do an in-place upgrade to TFS 2010 because we can't touch our production TFS server...we can't do a backup and restore because then the source code and work items will get out of sync very very quickly.

So we need a way to syncronize our production TFS server with the TFS 2010 Beta2 server we are standing up. We need to sync both ways.

We've read Willy's posts and followed his diagram:

http://blogs.msdn.com/willy-peter_schaub/archive/2009/04/06/vsts-rangers-projects-tfs2tfs-project-copy-migration-guidance-initial-thoughts.aspx

& we read most of his links:

http://blogs.msdn.com/willy-peter_schaub/archive/2009/10/07/tfs-integration-platform-summary-of-links.aspx

But when we see labels are not going to be "syncronized" with the tool you're going to be releasing...you might as well not even release the tool.

http://blogs.msdn.com/willy-peter_schaub/archive/2009/10/14/tfs-migration-platform-why-are-my-labels-not-moving-question-answers-5.aspx

While I appreciate the numerous blog posting and the work being done on the TFS Integration Platform...I have to be honest that this just feels cobbled together and very very flimsy.

For our department to be able to move forward with TFS we the developers NEED good tools from microsoft to handle syncronizing across 2 TFS servers. I'd like to have really sophsicated and reliable tools (with a great WPF UI).

We expect these to be delivered by Microsoft not on codeplex or sourceforge.

As mentioned before....I have to agr

ee that we need to have all the contents of a Team Project "synced"...not just work items and source...but we need documents, reports, areas, iterations, TFS security groups, etc, etc.

I can't believe we're the only company facing these pains.

Wednesday, October 28, 2009 5:59 AM by Allen

# re: Synchronizing two TFS servers

Thanks for the feedback Allen.  Requirements for the first version of the tool that we are using internally in production were focused primarily on content correctness, robustness and performance.  We had to have those things before moving to "nice to haves" like label support and a pretty front end.  Given our success using the TFS Integration Platform internally for the last few months in a production environment where the work of hundreds of people directly depends on the continuous and correct synchronization of results, I'd say we focused on the right things.

We do need label support and a WPF UI.  Those things and a binary release of the TFS to TFS Connector that we've built on the TFS Integration Platform are all in our plans.  We expect to publish the tools we use internally to synchronize two TFS systems to CodePlex soon.  That will be an alpha release.  The WPF UI we are working on will be in the beta and a binary release on a Microsoft downloads site will follow.

We have established a new forum for the TFS Integration Platform and related tools.  I would encourage you and anyone else with an interest in influencing our future direction to engage the team on that forum:

http://social.msdn.microsoft.com/Forums/en-US/tfsintegration.

Wednesday, October 28, 2009 8:02 AM by Bill Essary [MSFT]

# re: Synchronizing two TFS servers

I feel that MS is moving in the right direction to focus on VC and WIT first.  

Separate TFS servers imply a need for SOME of the information and context to be different, otherwise why not use a single server?

I do think that labels are a must and understand that this will be in the TiP shortly.

Our company needs OFFLINE synchronization because we have multiple environments that must be physically isolated.  I hope everyone and anyone else who needs this will let MS know.

Bob

Thursday, November 05, 2009 12:28 PM by Bob Hardister

# re: Synchronizing two TFS servers

I can't tell you how fantastic it would be to have TFS in a synchronize-able situation - I'm thinking syncing my work source-control with my home source-control (single coder at home, one of one/two at work). Also working for an higher-education institution, I can imagine the joy of students having TFS set up with AD for students to use at University, and the ability to sync a local copy of their account via their laptop - finally a solid source control setup we could implement and use to teach against.

Sunday, November 15, 2009 2:01 PM by Paul

Leave a Comment

(required) 
required 
(required) 

  
Enter Code Here: Required
 
Page view tracker