Welcome to MSDN Blogs Sign in | Join | Help

Orcas Version Control Performance Improvements

I'm heading to Disney World for the week in the morning so I may not be blogging too much next week (we'll see what my wife has to say about it :))

Today I was reviewing some results from some performance work we are doing for Version Control in the Orcas release.  We are also doing some work for Work Item Tracking but don't have full results yet.  These are not final numbers - there are still things we have left to investigate but they are exciting and I wanted to share them.

You'll need some notes to interpret them.

  • The chart includes the operation, an indication of whether it was a single file, a folder and its files or a tree.  In some cases it includes the number of items being operated on and in other cases it does not (I don't have all of the counts).
  • These measurements are times spent in SQLServer.  That tends to be the bottleneck in VC operations so improvements there tend to have the most dramatic end user effect.  Later in the cycle we'll have end-to-end client based numbers but this is what we have for now.
  • The chart includes the before changes and after changes times in seconds and a ratio of before/after and is sorted by that ratio.
  • The operations we measured are not exhaustive but were mostly chosen on how often they are used or the degree to which we knew from our internal dogfooding that we have problems with them.
  • You'll notice that 2 operations are slower than before "QueryLabel - file" - in this case the difference is so small we don't really care;  "Branch - 1533650 items" - this one we are looking at closely and expect to make additional changes to improve the operation.

Drum roll please...

 

 

Note the improvements are in multiples.  So the largest improvement is 219 times faster.  That's a big difference, eh?

The one that I'm happiest about (although it isn't the biggest) is "Get tree".  That's the operation I do all the time.  That gets any updates on the server down to my local machine.  It's 2.3 times faster.  That's gonna be nice.  Now you'll notice that it went from 103 seconds to 46 seconds and say well that's still not very fast.  This is a workspace will hundreds of thousands of files in it.  That's pretty darned fast.

As I say, these numbers aren't final.  I'm expecting them to get even better over the coming months but it's pretty cool to see the improvements we've made so far.

In the next couple of weeks I'm going to write a blog post about all of the great new stuff coming in Orcas.  I'm eager to hear your feedback.

 

Brian

Published Friday, October 06, 2006 7:28 PM by bharry

Comment Notification

If you would like to receive an email when updates are made to this post, please register here

Subscribe to this post's comments using RSS

Comments

# re: Orcas Version Control Performance Improvements

Tuesday, October 10, 2006 12:57 AM by Andrzej

It's really great - but the only thing I can't stop thinking about is: "why it was so slow in current version???".

In applications we develop in my company we often achive such big improvements and it's rather reason to shame ("why it was so slow...") than to be proud.

But internally we always are very, very, very happy :-))))))))))) Customer rather has ambivalent thoughts about it ;-)

# re: Orcas Version Control Performance Improvements

Tuesday, October 10, 2006 7:04 AM by bharry

That's the second time I've heard someone have that reaction (the other was sent to me privately) so it's probably going to be a fairly common reaction and worth me clarifying.

It’s all a matter of how you look at it.  The performance of TFS V1 is exceptionally good.  I’ve seen side by side comparisons with ClearCase, MKS, SourceSafe, Vault, SoftCM and others.  TFS blows them away on many operations and has rough parity on virtually all.  With the improvements we’ve made in Orcas I think we’ll be blowing them away on virtually everything.

You’ll notice on the benchmarks that the rows where I list # of items – the counts are very large (hundreds of thousands to over a million).  Most of the improvements we’ve made in Orcas have their most profound affect when working with such large numbers of files.  They’ll help everyone some but that’s where the differences get big.  We learn a lot from dogfooding at the scale we do and when we are done, we will have the most scalable team development server on the planet.  I expect we’ll have one more round of performance tuning in the release after Orcas but at that point we’ll be pretty much done.

Don’t be confused by the fact that we’ve made a lot of improvements.  The results are already really impressive.  Every customer who has done head to head perf comparisons between what they use and TFS has come away impressed.  On the other hand, I'd be lieing if I told you there we're any issues when you get to millions of files.  SP1 fixes the worst of them.  Orcas will fix almost all of the remaining ones and then we'll continue to tune it after that.

Brian

# re: Orcas Version Control Performance Improvements

Tuesday, October 10, 2006 7:59 AM by bharry

I just got this feedback from a customer and it's related to this conversation so I thought I'd pass it on:

"Universally, TFS source control is fast. It is so fast in fact, that the first time I downloaded a project, I tried to redownload because I thought it couldn't possibly have downloaded that fast."

This is not uncommon.  We get this kind of feedback all the time.

Brian

# re: Orcas Version Control Performance Improvements

Thursday, October 12, 2006 4:18 AM by Andrzej

Belive me, I understand you totally!!!

It's a shame, but sometimes we conceal reasons of such improvements from a client because we want to avoid such reactions as mine.

--------------

But my previuos post was not "teoretical" voice from some purist.

It comes from real expierience with TFS V1.

I belive your test plan of TFS is quite thorough but probably it doesn't consider scenario we dropped in (I don't know why. We can continue discussion privately - ask anything you need to narrow the problem).

--------------

We had about 100 users of SourceSafe. Part of that SourceSafe we have migrated to TFS-Source Control - about 100000 files with few years history, and than, after migration it was tragedy. One user taking whole project by TFS-MSSCCI Provider from Visual Studio 2003 takes our server to 100% CPU. 5 concurrent users takes our server to 400% CPU (2xXEON 2,4GHz) and very long HDD read queues: 100-600!!!

IMHO, one and only reason of such situation was stored procedure prc_Get that takes very, very long, many locking, deadlocking.

Instalation of SP1 Beta (it was only hope to change situation or go back to old, "good" SourceSafe) has broken our server because collation conflict - system tables had "Polish" collation and TFS databases - "Latin" collation and stored procedures have bug in using such scenario.

After long fight we stand up the server and now it is really faster!!! Only after SP1 performance is really good.

--------------

I think you should consider to give people mini-hotfix (micro-hotfix?) with only prc_Get improvement instead of unstable SP1. It would be much safer to install such hotfix and relief really big pain.

# re: Orcas Version Control Performance Improvements

Thursday, October 12, 2006 8:26 AM by bharry

I'm sorry you hit the perf problems and the problem with SP1 on a Polish collation sequence.  It sounds like you have SP1 installed and working properly now and the performance is really good for you.

We certainly did make quite a few perf improvements in SP1.  You are right that prc_Get is one of the most intensive operations in the system - both because it's run fairly often and because it has a very big job to do if you have a lot of files.

I'm afraid we can't separate it out as separate fix.  We'll fix any issues there are with SP and get that ready for people to install and hopefully it will help others who have hit the issues you have hit.

I''ll pass on the feedback about the SP1 install problems you hit and also check if there is anything about the MSSCCI provider access pattern that may have put extra load on prc_Get.

If you could send me your email at bharry@microsoft.com, I'd be happy to follow up with you in more detail on the issues you've experienced.

Brian

# re: Orcas Version Control Performance Improvements

Thursday, October 12, 2006 9:35 AM by Andrzej

Thank you very much for instant answer.

I must admit that current performance on our TFS SP1 server is very good.

I'm ready to help to find real reasons of our performance issues with TFS V1.

It may help other clients.

# re: Orcas Version Control Performance Improvements

Thursday, October 19, 2006 6:49 PM by Carl Daniel

Awesome speed improvements!  I've been using V1 since before Beta 1 and I'd have to agree with your assertion:  V1 is speedy!  Having V2 be even speedier is not a bad thing though!

# At the horizon: What are Microsoft's plans for the next version of Visual Studio - code-named "Orcas"?

Saturday, October 28, 2006 6:18 AM by Visual Studio Team System (VSTS) Blog

"Orcas" is the code name for the next version of Microsoft Visual Studio - "the next generation

# Orcas Work Item Tracking Performance Improvements

Sunday, January 14, 2007 10:22 AM by bharry's WebLog

Back in October I published some results from the performance work we've done on version control for

# Managing Quality (part 3) - Performance Testing

Tuesday, February 06, 2007 10:36 AM by bharry's WebLog

If you read my blog much, then you know performance and scale are near and dear to my heart. If you read

# TFS "Orcas" and Teamprise

Monday, June 04, 2007 2:22 PM by simply complex

As the upcoming release of the next major version of Team Foundation Server (code name "Orcas") gets closer, I want to write a bit about some of the new features that will be available. I'll also detail how Orcas will affect the Teamprise products and

# ???

Tuesday, July 24, 2007 7:01 PM by CyberJP

IS VSS dead or it will be included in Visual Studio 2008 ?

# re: Orcas Version Control Performance Improvements

Wednesday, July 25, 2007 12:09 PM by bharry

It is most certainly not dead.  We continue to sell it, support it, address issues customers find, etc.  It is true that we aren't implementing any big new features in Orcas for VSS – we are doing a service pack for VSS for VS 2008.  Priorities change release to release so we reevaluate our resource allocations periodically.  SourceSafe remains a solid solution for smaller teams that just need version control.

Brian

# VS2008 shipped

Sunday, November 25, 2007 5:53 PM by michal.Log

As you perfectly know (for example from Brian's blog , and probably hundred others), Orcas shipped. I'm

# VS2008 shipped

Sunday, November 25, 2007 6:03 PM by Noticias externas

As you perfectly know (for example from Brian's blog , and probably hundred others), Orcas shipped

# Visual Studio Team System 2008

Tuesday, December 04, 2007 4:04 PM by Yo sólo pasaba por aquí pero ya que estoy....

Como muchos ya sabréis hace poco Microsoft lanzó al mercado Visual Studio 2008. Si estáis

# Novedades en Visual Studio Team System 2008

Tuesday, December 04, 2007 4:06 PM by Yo sólo pasaba por aquí pero ya que estoy....

Como muchos ya sabréis hace poco Microsoft lanzó al mercado Visual Studio 2008. Si estáis

Leave a Comment

(required) 
required 
(required) 
 
Page view tracker