Orcas Work Item Tracking Performance Improvements
Back in October I published some results from the performance work we've done on version control for Orcas. We've also been working on work item tracking performance improvements - I mentioned them in our last dogfood server statistics. I'm very pleased with the results and thought I'd share them with you.
First you have to understand the context. The numbers are taken on a system under load and don't reflect changes you would see with a few people using a server. In fact this work was done as part of preparing to roll TFS out to the Microsoft Office organization and is from a 2000 user, constant load, statistical simulation based on their observed historical work load. A high load like this yields substantially different results than light loads because performance curves, as load increases, are rarely linear - we work hard to make them so but it's generally just a best approximation.
In addition to performance improvements under load - we have observed over the past year or so that in our dogfood environment work item tracking would have periodic "pauses". They seemed random (a couple of times a day per user) and would last 20 seconds to a minute. We have tracked down the cause of the problem and fixed it in Orcas. It had to do with rebuilding dynamic views in SQL and the times were exaggerated due to the number of columns we have in our views on our dogfood server. We had approximately 500 columns and what we learned is that rebuilding views on a busy server when the views have hundreds of columns is a pretty slow operation. You'll find in Orcas we gave greatly simplified our views and reduced our dependence on them.
So, on to the results... Here's the timing of the response time of the server for 4 of the most common operations in work item tracking and the corresponding data tier CPU utilization at a load of 2000 users. User perceived times will be longer as there is client/UI time that is not captured here (this is just measuring the server response time).
|
|
Save |
Open |
Query |
Page results |
CPU utilization |
|
Before (in sec) |
4.06 |
0.78 |
1.18 |
0.85 |
62.1% |
|
After (in sec) |
0.29 |
0.06 |
0.16 |
0.08 |
38.6% |
|
Times faster |
13.9 |
12.4 |
7.6 |
10.1 |
1.6 |
As you can see, not only is Orcas about 10 times faster under this load than TFS 2005 is, it is accomplished with just over half the CPU utilization. Both are results we're very excited about!
Lastly, we've also been working on client performance improvements. We've primarily been concentrating on work item form rendering speed. The improvements we've made have reduced the time to view a work item form to 20%-50% of the TFS 2005 times. That's 2-5 times faster. These times were taken under single user load scenarios so as the number of users increases, the performance improvements I described above will kick in and the difference will be even greater.
I'm looking forward to getting this stuff in your hands in the next CTP.
Brian
