Everything you want to know about Visual Studio ALM and Farming
Brian Harry is a Microsoft Technical Fellow working as the Product Unit Manager for Team Foundation Server. Learn more about Brian.
More videos »
The biggest area of growth in the last month is in work item tracking activity. The reason is that we hooked up a mirror between our "old" work item tracking system and TFS. This means that changes in each system is being replicated into the other. This has added some additional load to the server (although the increment is fairly minor in terms of available server capacity).
In addition to the additional WIT load, we continue to add source branches for more teams as we bring them on - that accounts for most of the file growth (about 6,000,000 files).
The big learning (for me) in the past month or two has been around SAN configuration and performance. A month or so ago we started getting event log errors on our SQL Server during peak load times saying that some I/O operations were taking more than 15 seconds to complete. This has been coincident with us seeing some performance degradation for end users during these peak times.
We initially suspected the SAN and ultimately concluded that the SAN is significantly underpowered and that there are some software improvements we can make.
What I learned about our SAN is that it is RAID5 with 16 spindles. There are 2 problems here:
We also learned some stuff looking at the source of the I/Os. We tracked back I/Os to the top sprocs generating them and some were obvious - prc_Get and prc_Checkin. However, some were surprising (prc_iFindPendingChange - was an example, I think). After investigation we found 3 indexes that could be tuned that would reduce the numbers of I/Os we generate for the same operations. We're continuing to investigate and I expect we'll find further improvements that will reduce the I/O we do per operation. As always, we'll roll these changes into our next service pack so everyone can benefit from them.
And here are the latest statistics...
UsersRecent users: 680Users with assigned work items: 1,662Version control users: 1,312
Work itemsWork items: 105,2770Areas & Iterations: 6,059Work item versions: 770,438Attached files: 28,582Queries: 9,954
Version controlFiles/Folders: 19,580,651/2,570,303LocalVersion: 110.9MTotal compressed file sizes: 193.3GWorkspaces: 2,702Shelvesets: 4,098Checkins: 81,292Pending changes: 397,954
Requests (last 7 days)Work Item queries: 250,592Work Item updates: 36,793Work Item opens: 157,748Gets: 14,033Downloads: 4.3MCheckins: 3,719Uploads: 10,852Shelves: 450
As always comments and questions are welcome (...even encouraged) You can even tell me if I'm boring you to death :)