I recently attended and presented at both TestExpo and the Software & Systems Quality Conference and at both events people seemed surprised to see a Microsoft stand.  In my experience testers don’t seem to be as aware of our tools as developers so hopefully I introduced a few more people to them at these events.  I promised to put together a blog posting laying out what Visual Studio Team System is all about and what should testers really should know about it?

Visual Studio Team System is really broken into two parts: Team Foundation Server and Visual Studio Role Editions.

 

image

Team Foundation Server

Team Foundation Server (TFS) is an integrated collaboration server to help you drive your development process from requirements through to deployment.  It allows you to store all project artefacts in one location and access that information and data from a tool of your choice (eg. Visual Studio, Outlook, Microsoft Project, Excel, Eclipse, Web Access, SharePoint etc)

  • Source Code Repository
    • TFS is an enterprise quality Version Control for all source code (not just .Net).  TFS is NOT based on Visual SourceSafe
  • Project Portal
    • TFS provides a Windows SharePoint Services Portal for every project to give one central location for project documentation
  • Work Item Tracking
    • Work Items give visibility of all work outstanding in your project (eg. use cases, development tasks, test cases and bugs).  Work Items are fully customisable
  • Project Reporting
    • SQL Server Reporting Services make it simple to produce high quality reports to provide better project transparency.
  • Build Server
    • TFS offers a tightly integrated and highly configurable build component based on MSBuild
  • Supports your Development Process
    • TFS is process agnostic, and can be configured to support any process with a Process Template.

Visual Studio Team System Role Editions

The 4 (or really 3 now) Visual Studio Team System Role Editions are:

  • Development
    • This Edition is focused on code quality.
  • Database
    • This edition has now been rolled into the Development Edition so will no longer be a stand alone product.  It provides a way to improve quality and change control at the data layer.
  • Test
    • Providing web testing, load testing and manual testing tools.
  • Architecture
    • Improves the design and validation of distributed systems giving architects and developers the ability to visually construct service-oriented solutions and validate them against their operational environments prior to deployment.
  • Suite
    • Includes all functionality provided by the role editions

All VSTS Role Editions are built on top of Visual Studio Professional Edition, so all features that are part of VSPro are included in the VSTS roles alongside their extra functionality

 

So what do Testers really need to know?

So, as this post is aimed at testers, what should testers be aware of in the current version of VSTS and also what is coming down the line in VSTS 2010 (codename Rosario)?

 

Visual Studio Team System 2008

Quality Assurance

Testing and maintenance are expensive so teams need to ensure that the builds that are actually being tested are of good quality.  The only way to do this is to encourage your developers to think about quality from day one!

Unit Testing

The VSTS Unit Testing Framework has been pushed down to VSPro so the tools are there for all developers to write unit tests.  Code coverage of these tests can also be measured to give more confidence on code quality

Working with Unit Tests

Database Unit Testing

Unit Test your stored procedure to ensure quality at the data layer as well as the code layer

Verifying Databases with Unit Tests

Static Code Analysis

You can think of Static Code Analysis as a first pass automated code review.  Code Analysis will flag up many common issues which can be difficult to spot or tedious to find. 

Guidelines for Using Code Analysis Tools

Code Metrics

How Complex is your code?  How difficult will it be to test properly?  How maintainable is it?  These are the kind of questions that can be easily answered through the code metrics.  They will give the developers a heads up on where they should be concentrating their code reviews to ensure quality, maintainable applications.

Measuring Complexity and Maintainability of Managed Code

Profiling

How does the application actually perform?  VSTS Profiling tools will enable developers to locate and eliminate bottlenecks in their code to ensure that the applications perform as expected

Analyzing Application Performance using Profiling Tools

Check In Policies

Essentially Check-In Policies are a quality gate on version control, tasks that must be completed before the code can be checked in and built.  Examples are the testing policy to ensure regression bugs are not introduced by a code change, the code analysis policy to ensure checked in code meets certain agreed standards and the Work Item policy to reduce scope creep and help testers to pinpoint what has been included in any given build of the application.

Working with Check-In Policies and Notes

Continuous Integration & Automated Builds with Build Verification

TFS makes it very simple to set up automated builds.  It is easy to include build verification tests and code analysis as part of these automated builds but they can be further extended to create installation packages, deploy test data etc.

Team Foundation Build Overview

Development Process & Defect Tracking

TFS can assist teams in adopting a development process or simply encourage them to take a step back and understand how they can improve their in-house (possibly undocumented) process.  By storing requirements, development tasks, test cases, builds and test results in one central location and allowing links between them, teams can gain much better visibility of project health and improve communication between all members of the team. 

Choosing a Process Template

Team Foundation Server Reporting

Take advantage of the power of TFS reporting to ensure your project is on track!  As everything related to your project is stored in TFS, all of that data is collected into the data warehouse.  Use SQL Reporting Services or Microsoft Excel to access the data you are interested in.

Using Reporting and Metrics

Testing

As I mentioned earlier in the post, many people are simply unaware of the testing tools included in VSTS.  These tests can be authored by non-technical testers but can be enhanced and extended through code if required.

Web Testing

Web tests are used to automate testing the functionality of Web applications and to see how they perform under load.  Record your interaction with a website and play it back.

Working with Web Tests

Load Testing

Bringing load testing to the masses!  The primary goal of a load test is to simulate many users accessing a server at the same time.

Working with Load Tests

Manual Testing

Manual Tests can be executed within Visual Studio like any other test.  Once the tester has followed the instructions in the manual test script they can add their feedback and apply a test pass or fail.  This allows you to report against your manual test results and keep track of build quality.

Working with Manual Tests

Data Generation

Testing with live data can be dangerous so if you are using SQL Server as your data store then Data Generation plans allow you to generate meaningful data the same size and shape as your live data.

Generating Data with Data Generators

Test Case Management

Test cases can be managed through the use of TFS Work Items and Test Lists,  test results can be published so they can be reported against and test failures can be attached to bugs which are then assigned to the dev team

Work Item Only View

Team System Web Access provides testers or users a “Work Item Only” view that restricts functionality so that you can create and view only your own work items. This means that you can submit defects and change requests directly to the dev team without the need for a Team Foundation Server Client Access License.  For more information, see Visual Studio Team System 2008 Licensing White Paper.

 

Visual Studio Team System 2010

Some of the manual testers I spoke to in the last few weeks were a little daunted by Visual Studio.  They don’t write any code and Visual Studio Test Edition contains a lot of functionality that they don’t need.  Well Visual Studio Team System 2010 is going to address these concerns and add a bucket load of cool features around testing

More details to follow when I get a chance to digest all the new stuff myself but keep an eye out for:

  • Test planning tools to breakdown requirements, plan tests and test configurations and assign testing responsibilities.
  • A TFS based test case management system that enables you to track all of your test cases, relate them to requirements and plans, track progress and perform impact analysis.
  • Test prioritization that highlights the tests that need to be run based on the changes that have actually been made to an application.
  • A tool for manual testers to help them manage their test runs, automate them to reduce manual labour and report actionable bugs that the developer is sure to be able to reproduce and fix.
  • Tools for developers to see exactly what testers have done when finding a bug, including the ability to "debug" the code as it was running at the time they hit the bug.
  • A new "gated checkin" feature which provides a natural evolution of continuous integration to catch build breaks before they happen rather than after.
  • And much more...

(Thanks to Brian for the list)

If you need anymore information on testing with VSTS then check out the links below or Contact me

Resources

Videos: Getting Started with Team System

Microsoft Tester Centre

Getting Started with Team System Testing Tools

"How Do I?" Videos for Team Foundation Server

Visual Studio 2005 Team Test Edition: Testing Demos

These demos show the unit and load testing features in Visual Studio 2005 Team Test Edition which are similar to the 2008 features

Brian Harry (Probably the best Team System blog around)

VSTS Case Studies

Changing a Web Vanguard – Windows Live Hotmail

This case study documents how the Windows Live Hotmail development team successfully utilized Visual Studio Team Edition for Software testers to build one of the most ambitious new releases of their iconic Web-based email service.

March Madness and Visual Studio Team Edition for Software Testers

This case study illustrates how the Windows Live Development team was able to design, build and deliver a high profile consumer facing Web application under extremely tight deadlines by making use of Visual Studio Team Edition for Software Testers.

 

Happy Testing!

Richard Erwin