Over the past few weeks, we have heard a spate of complaints about TFS database being bloated with test data and growing in size rapidly with time. Running tests regularly generates different kinds of data in your TFS database, primary categories being:

  1. Deployment binaries - the pdbs and dlls that you use to deploy during your test run are all stored in TFS each time you kick off a test run. Typically, this data is needed only if user downloads the test result's trx file , associated test binaries and attempts to "rerun" the test in the same test result
  2. Test attachment data from diagnostic data adapters - Diagnostic data adapters like intellitrace, video recording, event logs etc. collect data during your test execution and store this data in the test run if you file bugs during the test run. This data is usually heavy in manual test runs since it is uncommon for automated test runs to capture all of this data each time a set of tests is run
  3. Code coverage files - The code coverage data diagnostic adapter when used in automated runs, generates a fairly heavy coverage file and also uploads all the pdb files of your app under test since the coverage file needs those. This can tend to be fairly heavy

After analyzing several customer TFS databases, we realized that 1 was contributing to a significant chunk of the data being accumulated in the TFS database - anywhere from 30% to 80% of all test data. Most users did not however want these binaries in TFS since the rerun operation was very rarely done. Given this, we released a QFE last week, that would stop uploading these deployment binaries into TFS every time a test run is published into TFS: Team Foundation Server 2010 Service Pack 1: Reduce the size of the test data saved to the TFS database

 More details on the specifics of the fix are in the KB article from where you can also download the hotfix by simply typing out your email ID where you will receive the hotfix link. Sweet! I tried this with my mail id and here is the link I was sent.

To deal with the remaining test data though, we recommend running the Test Attachment Cleaner to routinely clean up the TFS database of unwanted or stale data. The tool is recommended to be run on the application tier of your TFS and has several modes that it can be executed in - see the description here for more details on the tool. The tool generates a whole lot of database records when you clean up your database and we recently learnt of a couple of SQL bugs that would cause these deleted records to persist as ghosted records instead of getting cleaned up automatically. To address this, SQL shipped 2 fixes in their Cumulative update package 10 for SQL Server 2008 R2 - it is recommended you install this update on your TFS before running the test attachment cleaner. In addition, we are also working on reducing (either in space or the number of db records) other kind of test data published to TFS.

In the meantime, if you are experiencing growth in your test data size in TFS, please download and install this hotfix and send your comments my way :-)