Welcome to MSDN Blogs Sign in | Join | Help

TFS Scalability

For a while now I've been alluding to the fact that we'd be updating our guidance on the size of teams that Team Foundation Server will support.  While we still have some work left to do to pin down the spectrum of team sizes that different classes of servers will support I'm happy to announce some exciting news.  We're officially changing the guidance from "Teams of up to 500 users" to "Teams of up to 2000 users"!  I'm really thrilled to be able to announce this and proud of the work we have done that has made it possible.  This is going to significantly increase the breadth of customers who can adopt Team Foundation Server as an enterprise wide development platform.

Now ,no scale number is meaningful without the details of the hardware configuration.  The test configuration that we used included:

2 server install (separate application and data tiers)

  • Application tier:  2P 2.8Ghz 4GB
  • Data tier:             4P 2.7Ghz 16GB
  • Disk system:       Direct attach storage, 14 – 15Krpm RAID 0 spindles

In truth, we could support  even larger teams with bigger hardware.  We are currently testing with a Unisys 8-way box and a XIOTech SAN to see how far we can push it but as that is getting to be a pretty expensive hardware configuration ($200K or so) we're not going to claim our scalability numbers based on that.  I may report them as a matter of interest but when we "publish" our team size stats I believe they will be based on a more moderately priced 4-way configuration like the one above.  We may eventually produce 4P/SAN numbers but as the 4P server is mostly CPU limited I don't think that will make a significant difference (although I'd recommend any large scale production installation go that way for the reliability that it can bring).

Some additional scaling could be obtained by deploying proxy servers to offload file downloads from the main server (even when the proxies are on the same LAN).  We are also going to experiment with moving WSS and the data warehouse to separate databases/servers to see what impact that will have - although those are not supported V1 configurations.

The other thing that matters for any reasonable database benchmark is the size of the dataset used in the simulation, so here's what we used (just giving the stats for the two primary resources).

Data set size

  • Work items:      750,000
  • Files:                1,350,000

These resuts were created by following the benchmarking methodology documented in my earlier blog posts.

At this point we are about done with server performance tuning for this version so I'm not expecting these numbers to change significantly - although we'll probably see a few percent improvement yet.  For those that notice the result I listed is a suspiciously nice round number - the actual result achieved was 2230 users but I want stick with a nice round number with some margin for error.

Based on these results, we are now moving forward with developing plans to roll out TFS as the standard for our entire division.  What do they say?  Be careful what you wish for - you might get it? :)  It is with great pride (and some breath holding) that that we embark on deploying a single server supporting thousands of people in a day to day mission critical role.  I'm confident it's going to work out well but I'm sure all my collegues will have great feedback on how we can make the next version even better.  I'll keep you guys posted as we make progress on this.

Brian

Published Friday, December 09, 2005 4:22 PM by bharry

Comment Notification

If you would like to receive an email when updates are made to this post, please register here

Subscribe to this post's comments using RSS

Comments

# re: TFS Scalability

Friday, December 09, 2005 9:33 PM by Dave McKinstry
Certainly good news Brian. Thanks for sharing. Can you share what portion of the system became the bottleneck first? Was the App Tier or the Data Tier? Was it RAM or CPU or I/O? That will help those of use doing capacity planning better guess where there is some wiggle room in the hardware configuration. Thanks!

# re: TFS Scalability

Saturday, December 10, 2005 4:15 PM by Etienne Tremblay
WOW! great job on the perf guys. This will go a long way toward eliminating a few skeptics that this was only for small shops and not for the enterprise.

Keep it up guys, looking forward to the release of TFS.

Cheers,

ET

# re: TFS Scalability

Saturday, December 10, 2005 8:20 PM by bharry
Thanks for the compliments - the team appreciates it.

The limit was the data tier (the app tier was only around 50% utilization). The data tier was a little over 80% CPU utilization. I don't have the stats on the I/O handy but based on what I do have with me I'm certain disk I/O was not a problem. I'm also certain memory wasn't either. The memory usage never went above 12GB.

I'll look more at the data tomorrow morning and let you know if I find anything different.

# re: TFS Scalability

Tuesday, December 13, 2005 12:05 AM by Sanjay Narang
That is really wonderful bharry. Compliments to you and your team! I have two follow up questions:

1. What build these results are tested on. Is this any of publicaly released build (like Beta 3 Refresh or Dec CTP), OR your internal current build?

2. As you mentioned that limit was in Data Tier. Does that mean there has not been much work done on AT front? I'm asking this because for some applications that are writing web interface over OM, the AT becomes bigger bottelnect (in terms of CPU) in stead of DT.

# re: TFS Scalability

Friday, December 16, 2005 7:55 PM by bharry
This was done on an internal build and there are quite a few perf improvements since the Beta 3 refresh. It's pretty close to the Dec CTP (although remember that's not a go-live build). We'll have a release candidate early next year that will have all the tuning in it.

The DT is the bottle neck because we do so much of the work in TSQL. We don't cache a ton in AT. Probably the most expensive thing that we do on the AT is generate tickets for download (this is an important part of enabling the proxy). I think in future versions we'll be looking at how we can better balance the load between them.

Brian

# re: TFS Scalability

Monday, December 19, 2005 11:55 AM by Lee
I think it's very good news that the data server is CPU bound.

In fact Intel is now releasing *dual-core* processors at 3.46Ghz, which would surely double your CPU power.

If the processing power on the data server could be doubled, wouldn't this have a big impact on your scalability limit?



# re: TFS Scalability

Tuesday, December 20, 2005 7:25 AM by bharry
Yes, the bottleneck (today) is the datatier CPU. More CPU will mean more supported users. We are in the process of testing on an 8 proc and we'll see what results that yields.

# DEV - Scalabilitate TFS

Friday, February 09, 2007 8:45 AM by Weblogul lui Zoli

Vesti bune pentru Visual Studio 2005 Team Foundation Server: vom actualiza recomandarea pentru numarul

# re: TFS Scalability

Wednesday, August 08, 2007 2:53 PM by Pollis

Hi, nice very nice page..!

<a href="www.volny.cz/porsche356">356 porsche sale </a>[URL=www.volny.cz/porsche356]356 porsche sale [/URL] 356 porsche sale - www.volny.cz/porsche356

<a href="http://volny.cz/pearlbridaljewelry">pearl">http://volny.cz/pearlbridaljewelry">pearl bridal jewelry</a>[URL=http://volny.cz/pearlbridaljewelry]pearl bridal jewelry[/URL] pearl bridal jewelry - http://volny.cz/pearlbridaljewelry

<a href="http://volny.cz/buypearlnecklace">buy">http://volny.cz/buypearlnecklace">buy a pearl necklace</a>[URL=http://volny.cz/buypearlnecklace]buy a pearl necklace[/URL] buy a pearl necklace - http://volny.cz/buypearlnecklace

<a href="www.volny.cz/gmclosangeles">gmc los angeles</a>[URL=www.volny.cz/gmclosangeles]gmc los angeles[/URL] gmc los angeles - www.volny.cz/gmclosangeles

<a href="www.volny.cz/porsche944">944 porsche</a> [URL=www.volny.cz/porsche944]944 porsche[/URL] 944 porsche - www.volny.cz/porsche944

Good luck !

PS: do you listen Linkin Park ?

# County Animal Shelters &raquo; Eric Jarvi : December 2005 - Posts

Leave a Comment

(required) 
required 
(required) 
 
Page view tracker