Capacity testing for TFS
Well, I'm behind a couple of blog posts already. I'll do this and get to the one on Team Project organization tonight or tomorrow.
We've been finalizing the limits that we are going to test TFS to for its release. I wanted to share them so you know how far we are pushing the system and can use that information as one of many factors to decide if TFS will work for your organization. First a few notes on methodology and scope.
This testing is primarily focused around the "capacity" of the system. Generally for each thing that TFS can store or manipulate we must ask "how many" and "how big". While the list you'll find below does not include every single datum that can be stored, it includes the majority and particularly the ones where we want to ensure the capacity is very large. Many of our other "limits" (like length of a file path, for example) are covered elsewhere in our functional testing and are not called out here.
For each item, you'll see a "requirement" and a "stretch goal". The requirement is the capacity we must achieve in order to ship. The stretch goal is the capacity to which we will actually measure. Anything over the requirement is considered good enough. Anything at the stretch goal can probably go higher but we aren't going to measure any higher.
You'll also see a "count" and a "size". It's not always obvious how to interpret these. Generally counts are "how many" - like how many files can the system have or how many work items. Size can get a bit wierd because sometimes it is also measured as a count. For example: Workspaces - count means how many workspaces can be stored on the server; size - means how big can a workspace be (measured as the number of files/folders mapped in the workspace). I don't think we've been exactly consistent in how we use them but I've tried to clarify with text. For some items we haven't included both count and size if we felt having both didn't make much sense.
Generally this only includes the counts and sizes of things and doesn't talk about operations. We will be testing operations on the counts and sizes to make sure that the system is really working with that much data in it. We have called out a few specific operations that we were particularly interested in.
Lastly all of this refers to the data that can be stored in one server. Many (especially larger) organizations will setup multiple servers. And for the disclaimer - all information is subject to change without notification :)
As always, we're interested in any feedback or comments that you have...
|
General |
Target |
Notes |
|
Team projects |
|
|
|
Count |
|
Each project has 200 files. |
|
Requirement |
500 CMMI |
|
|
Stretch Goal |
1250 CMMI |
|
|
VS Projects |
|
|
|
Count |
|
Projects in a solution |
|
Requirement |
50 |
|
|
Stretch Goal |
150 |
|
|
Size |
|
Files in a VS project |
|
Requirement |
TBD |
|
|
Stretch Goal |
TBD |
|
|
Process Templates |
|
|
|
Count |
|
|
|
Requirement |
50 |
|
|
Stretch Goal |
200 |
|
|
Size |
|
|
|
Requirement |
200MB |
|
|
Stretch Goal |
2GB |
|
|
Users |
|
|
|
Count |
|
# of users authorized to use the system |
|
Requirement |
5,000 |
|
|
Stretch Goal |
30,000 |
|
|
Groups |
|
|
|
Count |
|
|
|
Requirement |
4000 |
|
|
Stretch Goal |
10000 |
|
|
Areas & Iterations |
|
|
|
Count |
|
|
|
Requirement |
10,000 |
|
|
Stretch Goal |
100,000 |
|
|
Event Subscriptions |
|
|
|
Count |
|
|
|
Requirement |
7,500 |
|
|
Stretch Goal |
20,000 |
|
|
Size |
|
|
|
Requirement |
10 clauses |
|
|
Stretch Goal |
100 clauses |
|
|
Work Item Tracking |
Target |
Notes |
|
Work Item Types |
|
|
|
Count |
|
|
|
Requirement |
3,500 |
|
|
Stretch Goal |
8,750 |
|
|
Size |
|
Size of the work item type XML file |
|
Requirement |
800K |
|
|
Stretch Goal |
2M |
|
|
Queries |
|
|
|
Count |
|
|
|
Requirement |
50,000 |
|
|
Stretch Goal |
100,000 |
|
|
Size |
|
|
|
Requirement |
50 clauses |
|
|
Stretch Goal |
500 clauses |
|
|
Work Items |
|
|
|
Count |
|
|
|
Requirement |
10,000,000 WI versions |
1M with 10 versions each |
|
Stretch Goal |
25,000,000 WI versions |
2.5M with 10 versions each |
|
Size |
|
|
|
Requirement |
200 versions, 100K long text, 25 attachments, 25 links |
|
|
Stretch Goal |
1,000 versions, 1MB long text, 200 attachments, 200 links |
|
|
Links |
|
|
|
Count |
|
This count is total number in the database rather than per work item. |
|
Requirement |
500,000 |
|
|
Stretch Goal |
2,500,000 |
|
|
Attachments |
|
|
|
Count |
|
This count is total number in the database rather than per work item |
|
Requirement |
500,000 |
|
|
Stretch Goal |
2,500,000 |
|
|
Size |
|
|
|
Requirement |
500MB |
|
|
Stretch Goal |
2 GB |
|
|
Version Control |
Target |
Notes |
|
Versioned Items |
|
Files and folders |
|
Count |
|
# of files and folders |
|
Requirement |
2,000,000 |
|
|
Stretch Goal |
20,000,000 |
|
|
Size |
|
Size of a single file. |
|
Requirement |
10GB binary, 200,000 lines text |
|
|
Stretch Goal |
250GB binary, 1,000,000 lines text |
|
|
Workspaces |
|
|
|
Count |
|
|
|
Requirement |
1500 with an average of 10,000 files/folders each |
|
|
Stretch Goal |
5,000 with an average of 50,000 files/folders each |
|
|
Size |
|
# of files/folders mapped in the workspace. |
|
Requirement |
200,000 |
|
|
Stretch Goal |
1,000,000 |
|
|
Working Folders |
|
|
|
Count |
|
Total in database |
|
Requirement |
7,500 |
|
|
Stretch Goal |
25,000 |
|
|
Size |
|
Number for working folders in one workspace. |
|
Requirement |
40 |
|
|
Stretch Goal |
250 |
|
|
Pending Changes |
|
|
|
Count |
|
# of pending changes in a single workspace or whole database |
|
Requirement |
200,000 |
|
|
Stretch Goal |
1,000,000 |
|
|
Labels |
|
|
|
Count |
|
Total # in database |
|
Requirement |
10000 will 5000 files in each |
|
|
Stretch Goal |
100,000 |
|
|
Size |
|
# of files in the label |
|
Requirement |
200,000 |
|
|
Stretch Goal |
1,000,000 |
|
|
Shelvesets |
|
|
|
Count |
|
|
|
Requirement |
5,000 |
20 pending changes, 5 work items each and 3 checkin notes each |
|
Stretch Goal |
100,000 |
20 pending changes, 5 work items each and 3 checkin notes each |
|
Size |
|
|
|
Requirement |
10,000 |
25 work items, 20 checkin notes |
|
Stretch Goal |
100,000 |
100 work items, 50 checkin notes |
|
Branches |
|
|
|
Count |
|
# of branches in the database |
|
Requirement |
200 branches with 2,000 files/folders each |
|
|
Stretch Goal |
5000 branches with 10,000 files/folders each |
|
|
Size |
|
Files and folders in the branch |
|
Requirement |
200,000 files/folders |
|
|
Stretch Goal |
1,000,000 files/folders |
|
|
Merging |
|
|
|
Count |
|
Counting number of files to merge |
|
Requirement |
100,000 with an average of 1 rev per file |
|
|
Stretch Goal |
1,000,000 with an average of .5 revs per file |
|
|
Size |
|
Size of files being merged |
|
Requirement |
200,000 lines of text |
|
|
Stretch Goal |
1,000,000 lines of text |
|
|
Changesets |
|
|
|
Count |
|
|
|
Requirement |
200,000 |
20 pending changes, 5 work items each and 3 checkin notes each |
|
Stretch Goal |
2,000,000 |
20 pending changes, 5 work items each and 3 checkin notes each |
|
Size |
|
|
|
Requirement |
200,000 pending changes |
25 work items, 20 checkin notes |
|
Stretch Goal |
1,000,000 |
100 work items, 50 checkin notes |
|
Project Management |
Target |
Notes |
|
Office Document |
|
Excel and MS Project |
|
Count |
|
|
|
Requirement |
1,000 WIs |
|
|
Stretch Goal |
10,000 WIs |
|
|
Build |
Target |
Notes |
|
Build Types |
|
|
|
Count |
|
Total in the database |
|
Requirement |
1,000 |
|
|
Stretch Goal |
3,000 |
|
|
Size |
|
|
|
Requirement |
n/a |
|
|
Stretch Goal |
n/a |
|
|
Builds |
|
|
|
Count |
|
Total in the database |
|
Requirement |
50,000 |
|
|
Stretch Goal |
250,000 |
|
|
Size |
|
|
|
Requirement |
250 solutions, 100,000 files, 1,000 change sets, 1,000 work items, 50000 test results |
|
|
Stretch Goal |
2000 solutions, 500,000 files, 10,000 changesets, 10,000 work items, 200000 test results |
|
|
EDT |
Target |
Notes |
|
Test Results (Total in database) |
|
|
|
Count |
|
|
|
Requirement |
1,000,000 |
yogitam to revisit based on the email thread |
|
Stretch Goal |
5,000,000 |
|
|
Size |
|
|
|
Requirement |
TBD |
|
|
Stretch Goal |
TBD |
|
|
Tests (Total in databases) |
|
|
|
Count |
|
|
|
Requirement |
200,000 |
|
|
Stretch Goal |
1,000,000 |
|
|
Test Runs |
|
|
|
Count |
|
|
|
Requirement |
50,000 |
|
|
Stretch Goal |
100,000 |
|
|
|
|
|
|
|
|
|
|
|
|
|
