Many of our TFS 2010 customers have reported the problem of test attachments taking up a lot of space on TFS database. In order to reduce the rate of increase in database size because of test attachments, following 4 features have been implemented in Visual Studio 11 Beta:

1. Automatic upload of Deployment Items has been stopped: For automated test runs, Test deployment items and Dlls\Pdbs no longer get published to the TFS automatically. However if required, user can publish these attachments by making some changes in the test settings used during the test run.

This feature has also been released as a hotfix for Visual Studio 2010 SP1 TFS and more details about this hotfix, are available here.

As mentioned in this link, in case of Visual Studio 2010, if code coverage data collector is enabled in the test settings then Dlls\Pdbs are always uploaded to the TFS because these files are required for code coverage analysis. However in Visual Studio 11 Beta, code coverage data collector has been improved to generate code coverage data dynamically during test run. Hence it no longer requires Dlls\Pdbs to be uploaded to the TFS. Thus, in Visual Studio 11 Beta, the only way to upload deployment items is by modifying the test settings used during the test run.

2. Only one System Information file will be uploaded per automated test run: In Visual Studio 2010 (or Microsoft Test Manager 2010), during an automated test run, each test case used to upload its own System Information file. Hence, if there are 100 test cases in the run, there used to be 100 system information files. Though the system information files are extremely small (<1KB), over the period, they still use to add up. With Visual Studio 11 Beta, there is only one System Information file uploaded per test run.

3. Test Attachment Compression: Now all the test attachments generated during manual or automated test runs will get gzip compressed before getting published to the TFS thus saving a lot of database space. During download, these attachments first get downloaded in compressed form and then automatically decompressed on the client machine. This also reduces the time and bandwidth required for upload and download of test attachments.

4. Tr_ to Trx: In Visual Studio 2010, a tr_ file gets published to TFS for each test case result contained in a test run. Each tr_ file is generally of size < 2 Kb but their total size become considerable for test runs having thousands of test case results. When a test run is exported, all the individual tr_ files corresponding to the test case results contained in the test run, are downloaded and then merged to generate a trx file. This resulted in performance issues for large test runs.

In Visual Studio 11 Beta, a single trx file having information about all the test case results for a test run will be published to the TFS.

It has been seen that for a test run the size of a single Trx file is less as compared to sum of the sizes of each individual tr_ file. Moreover downloading a single trx file also gives performance improvement over downloading individual tr_ files and then merging them to obtain the trx file.

Benefits from feature 3 and 4 are explained with help of the result of a test run that we executed against a Visual Studio 11 Beta TFS.

Details of the test run:

Number of test cases executed as part of test run = 11

Each test case had some deployment items. There were two test Dlls and Pdbs.

Test settings used by the tests had following data collectors enabled:

  • Event Log
  • System Information
  • Test Impact
  • Video Recorder

Upload of deployment items was deliberately enabled in the test settings, so as to compare the results with TFS 2010.

Result of the test run:

Feature

Size of attachments uploaded in TFS 2010
(without compression)

Size of attachments uploaded in Visual Studio 11 Beta TFS
(with compression)

Percentage size reduction

Test attachment compression

12.17 MB

3.87 MB

68.17

Tr_ to Trx

39.36 KB

3.57 KB

90.93

 

Hence, test attachment compression and Tr_ to Trx features are really helpful in saving valuable database space without compromising on any functionality in the product. On the basis of our testing on large real customer databases, most customers will see 80%-90% dip in the rate of increase of database space. The actual number obviously will vary depending on the usage.

However with time as database starts filling up, there may arise a need to purge unwanted test attachments. This can be accomplished using Test Attachment Cleaner power tool. Visual Studio 2010 compatible version of this tool is available as part of TFS 2010 Power Tools release which is available for download here. This version of Test Attachment Cleaner power tool is not compatible with Visual Studio 11 Beta. A new release of this tool, which will be compatible with Visual Studio 11 Beta, is expected soon.