Microsoft UK Developer Tools Team

This week I presented a session at the British Computer Society Configuration Management Specialist Group's Agile SCM Event. 

It was an excellent day with a number of speakers from various backgrounds talking about the challenges and best practice of Agile development but what really struck me while listening to the speakers was that Visual Studio Team System is going in the right direction to address those issues.

My presentation was on "SCM and Agile Practices in the Microsoft Developer Division". The slides should be available soon on the BCS site but if anyone is interested then I will post them here.  If you attended the event and have any feedback or if you need any further information then please get in touch.

Regards,

Richard

After two years in the Dev Tools Team, I have recently changed role and moved into the Microsoft Technology Centre as an Architect. In this role get to play with all of Microsoft's latest and greatest technologies while developing Proof of Concepts for our customers. It also means that, after specialising solely in VSTS for the last two years, I get to use it in real development projects! It didn't seem appropriate to continue posting to this blog, so I have started my own.

  http://blogs.msdn.com/NeilKidd/

If there are any updates to the tools on CodePlex (IEeee, Web test Plugin and the Work Item Controls) then I will cross post them to this blog - assuming Rich doesn't revoke my permissions to  :-)

Neil Kidd

I recently attended and presented at both TestExpo and the Software & Systems Quality Conference and at both events people seemed surprised to see a Microsoft stand.  In my experience testers don’t seem to be as aware of our tools as developers so hopefully I introduced a few more people to them at these events.  I promised to put together a blog posting laying out what Visual Studio Team System is all about and what should testers really should know about it?

Visual Studio Team System is really broken into two parts: Team Foundation Server and Visual Studio Role Editions.

Team Foundation Server

Team Foundation Server (TFS) is an integrated collaboration server to help you drive your development process from requirements through to deployment.  It allows you to store all project artefacts in one location and access that information and data from a tool of your choice (eg. Visual Studio, Outlook, Microsoft Project, Excel, Eclipse, Web Access, SharePoint etc)

• Source Code Repository
  • TFS is an enterprise quality Version Control for all source code (not just .Net).  TFS is NOT based on Visual SourceSafe
• Project Portal
  • TFS provides a Windows SharePoint Services Portal for every project to give one central location for project documentation
• Work Item Tracking
  • Work Items give visibility of all work outstanding in your project (eg. use cases, development tasks, test cases and bugs).  Work Items are fully customisable
• Project Reporting
  • SQL Server Reporting Services make it simple to produce high quality reports to provide better project transparency.
• Build Server
  • TFS offers a tightly integrated and highly configurable build component based on MSBuild
• Supports your Development Process
  • TFS is process agnostic, and can be configured to support any process with a Process Template.

Visual Studio Team System Role Editions

The 4 (or really 3 now) Visual Studio Team System Role Editions are:

• Development
  • This Edition is focused on code quality.
• Database
  • This edition has now been rolled into the Development Edition so will no longer be a stand alone product.  It provides a way to improve quality and change control at the data layer.
• Test
  • Providing web testing, load testing and manual testing tools.
• Architecture
  • Improves the design and validation of distributed systems giving architects and developers the ability to visually construct service-oriented solutions and validate them against their operational environments prior to deployment.
• Suite
  • Includes all functionality provided by the role editions

All VSTS Role Editions are built on top of Visual Studio Professional Edition, so all features that are part of VSPro are included in the VSTS roles alongside their extra functionality

So what do Testers really need to know?

So, as this post is aimed at testers, what should testers be aware of in the current version of VSTS and also what is coming down the line in VSTS 2010 (codename Rosario)?

Visual Studio Team System 2008

Quality Assurance

Testing and maintenance are expensive so teams need to ensure that the builds that are actually being tested are of good quality.  The only way to do this is to encourage your developers to think about quality from day one!

Unit Testing

The VSTS Unit Testing Framework has been pushed down to VSPro so the tools are there for all developers to write unit tests.  Code coverage of these tests can also be measured to give more confidence on code quality

Working with Unit Tests

Database Unit Testing

Unit Test your stored procedure to ensure quality at the data layer as well as the code layer

Verifying Databases with Unit Tests

Static Code Analysis

You can think of Static Code Analysis as a first pass automated code review.  Code Analysis will flag up many common issues which can be difficult to spot or tedious to find. 

Guidelines for Using Code Analysis Tools

Code Metrics

How Complex is your code?  How difficult will it be to test properly?  How maintainable is it?  These are the kind of questions that can be easily answered through the code metrics.  They will give the developers a heads up on where they should be concentrating their code reviews to ensure quality, maintainable applications.

Measuring Complexity and Maintainability of Managed Code

Profiling

How does the application actually perform?  VSTS Profiling tools will enable developers to locate and eliminate bottlenecks in their code to ensure that the applications perform as expected

Analyzing Application Performance using Profiling Tools

Check In Policies

Essentially Check-In Policies are a quality gate on version control, tasks that must be completed before the code can be checked in and built.  Examples are the testing policy to ensure regression bugs are not introduced by a code change, the code analysis policy to ensure checked in code meets certain agreed standards and the Work Item policy to reduce scope creep and help testers to pinpoint what has been included in any given build of the application.

Working with Check-In Policies and Notes

Continuous Integration & Automated Builds with Build Verification

TFS makes it very simple to set up automated builds.  It is easy to include build verification tests and code analysis as part of these automated builds but they can be further extended to create installation packages, deploy test data etc.

Team Foundation Build Overview

Development Process & Defect Tracking

TFS can assist teams in adopting a development process or simply encourage them to take a step back and understand how they can improve their in-house (possibly undocumented) process.  By storing requirements, development tasks, test cases, builds and test results in one central location and allowing links between them, teams can gain much better visibility of project health and improve communication between all members of the team. 

Choosing a Process Template

Team Foundation Server Reporting

Take advantage of the power of TFS reporting to ensure your project is on track!  As everything related to your project is stored in TFS, all of that data is collected into the data warehouse.  Use SQL Reporting Services or Microsoft Excel to access the data you are interested in.

Using Reporting and Metrics

Testing

As I mentioned earlier in the post, many people are simply unaware of the testing tools included in VSTS.  These tests can be authored by non-technical testers but can be enhanced and extended through code if required.

Web Testing

Web tests are used to automate testing the functionality of Web applications and to see how they perform under load.  Record your interaction with a website and play it back.

Working with Web Tests

Load Testing

Bringing load testing to the masses!  The primary goal of a load test is to simulate many users accessing a server at the same time.

Working with Load Tests

Manual Testing

Manual Tests can be executed within Visual Studio like any other test.  Once the tester has followed the instructions in the manual test script they can add their feedback and apply a test pass or fail.  This allows you to report against your manual test results and keep track of build quality.

Working with Manual Tests

Data Generation

Testing with live data can be dangerous so if you are using SQL Server as your data store then Data Generation plans allow you to generate meaningful data the same size and shape as your live data.

Generating Data with Data Generators

Test Case Management

Test cases can be managed through the use of TFS Work Items and Test Lists,  test results can be published so they can be reported against and test failures can be attached to bugs which are then assigned to the dev team

Work Item Only View

Team System Web Access provides testers or users a “Work Item Only” view that restricts functionality so that you can create and view only your own work items. This means that you can submit defects and change requests directly to the dev team without the need for a Team Foundation Server Client Access License.  For more information, see Visual Studio Team System 2008 Licensing White Paper.

Visual Studio Team System 2010

Some of the manual testers I spoke to in the last few weeks were a little daunted by Visual Studio.  They don’t write any code and Visual Studio Test Edition contains a lot of functionality that they don’t need.  Well Visual Studio Team System 2010 is going to address these concerns and add a bucket load of cool features around testing

More details to follow when I get a chance to digest all the new stuff myself but keep an eye out for:

• Test planning tools to breakdown requirements, plan tests and test configurations and assign testing responsibilities.
• A TFS based test case management system that enables you to track all of your test cases, relate them to requirements and plans, track progress and perform impact analysis.
• Test prioritization that highlights the tests that need to be run based on the changes that have actually been made to an application.
• A tool for manual testers to help them manage their test runs, automate them to reduce manual labour and report actionable bugs that the developer is sure to be able to reproduce and fix.
• Tools for developers to see exactly what testers have done when finding a bug, including the ability to "debug" the code as it was running at the time they hit the bug.
• A new "gated checkin" feature which provides a natural evolution of continuous integration to catch build breaks before they happen rather than after.
• And much more...

(Thanks to Brian for the list)

If you need anymore information on testing with VSTS then check out the links below or Contact me

Resources

Videos: Getting Started with Team System

Microsoft Tester Centre

Getting Started with Team System Testing Tools

"How Do I?" Videos for Team Foundation Server

Visual Studio 2005 Team Test Edition: Testing Demos

These demos show the unit and load testing features in Visual Studio 2005 Team Test Edition which are similar to the 2008 features

Brian Harry (Probably the best Team System blog around)

VSTS Case Studies

Changing a Web Vanguard – Windows Live Hotmail

This case study documents how the Windows Live Hotmail development team successfully utilized Visual Studio Team Edition for Software testers to build one of the most ambitious new releases of their iconic Web-based email service.

March Madness and Visual Studio Team Edition for Software Testers

This case study illustrates how the Windows Live Development team was able to design, build and deliver a high profile consumer facing Web application under extremely tight deadlines by making use of Visual Studio Team Edition for Software Testers.

Happy Testing!

Richard Erwin

Woohoo! It's all going off today!  I can finally start talking about what's coming next in VSTS Rosario!

Check out these links for more information:

Visual Studio 2010 and .NET Framework 4.0 Overview 

Visual Studio Team System 2010 Week on Channel 9!

And finally we have just announced the merger of the Database and Development Visual Studio Team System Editions.  So if you have the Development or Database Edition with an active MSDN subscription you will be able to download the other edition from October! 

Cheers,

Rich

Hello to anyone who attended one of the Visual Studio Events in Reading over the last couple of weeks.  I promised to upload some links to useful resources, training and the presentation so I will try and get that done by the end of the week.

In the meantime I've added some links to the trial versions and service packs that you might need if you want to have a play with the features I demonstrated.

I'm off to Manchester this evening as I'm presenting at TestExpo 2008 tomorrow, so if you find yourself at that event then feel free to pop over to the Testhouse stand and say hello. 

Download Virtual PC 2007

Download Visual Studio® Team System 2008 Team Foundation Server and Team Suite VPC Image (Trial)

If required you can also install the Visual Studio Team System 2008 Team Suite (90-day Trial) locally.

For the SQL Server 2008 Support and new features  you’ll also need the GDR & SP1 and a few other things. Make sure to read the pre-requisites before you install. 

Microsoft® Visual Studio Team System 2008 Database Edition GDR August CTP

Microsoft® Visual Studio® 2005 Team Suite Service Pack 1

Visual Studio 2008 Team Foundation Server Service Pack 1

Microsoft .NET Framework 3.5 Service Pack 1

Microsoft® SQL Server Express 2008

Also take a look at Gert's blog for lots more information.

Thanks,

Richard Erwin

Have you installed the Microsoft® Visual Studio Team System 2008 Database Edition GDR August CTP so that you can target SQL Server 2008 with your Database Projects and found this error message when you create a new project?

---------------------------
Microsoft Visual Studio
---------------------------
Could not load file or assembly 'System.Data.SqlServerCe, Version=3.5.1.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91' or one of its dependencies. The system cannot find the file specified.
---------------------------
OK  
---------------------------

Yes? Well, like me you should read the installation instructions (am I allowed to say RTFM here?)

As per Gert's blog you need to install the Microsoft SQL Server Compact Edition 3.5 SP1 as well as VS2008 SP1.

Get that badboy installed and it'll fix it right up for you.

Cheers

Rich

I don't like to talk much about licensingy things - it's not my area and I'm no expert, but this is important. Brian Harry (TFS overlord and all round good egg) has posted a blog entry around the future of the TFS power tools. I would strongly recommend having a read of it and feeding back any opinions you have. Basically it boils down to deciding if the power tools should only be available to customers who purchase SA (a.k.a. software maintenance). Personally I think that people are missing a big trick if they don't buy SA (SA means that you get upgrades to any new versions of the product - a.k.a. Rosario - that are release during the term of your SA).

Neil Kidd

I've updated my web test scriptable parameters plugin to fix a bug with scripted context parameters and extracted tests and add the ability to to script the file name property of any file upload params.

For more information see:

  http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=3006040&SiteID=1

Many thanks to Lewis for his hard work in developing a repro and to Duat Le for spotting the casting crime I was committing.

Hope this is useful.

Neil Kidd.

I only found this feature of DB Pro the other day - must remember to spend more time exploring menus - so thought I should share.

When you are creating a Data generation Plan, one of the options on how to control the shape of the data is to use a Regular Expression. To help you develop these, the Reg Ex builder dialog is a great tool to use. This is on the Data Generator menu on the Data menu.

When you enter your Reg Ex, it quickly generates a preview of the data that will be created - a great way to play with your Reg Ex. Once you have created your master piece, you can save it for later use. There are also some pre-built Reg Exs that you might find useful (URL, email address, Zip Code, ...).

I've been showing this tool off as part of my demo at the MSDN Roadshows in the UK and have a couple of useful Reg Exs, that are probably worth sharing:

For a persons title: Mr|(Mrs|Miss|Mz)

50% of the rows will be Mr, then the other 50% will be an even split of Mrs, Miss or Mz - ok so not 100% accurate, but probably good enough for a decent test.

UK Street Address: ([1-9]|[1-9][0-9]) [A-W][aeiou][b-y][aeiou][b-y]{2} (Street|Road|Avenue|Close)

Will give strange sounding street names, but again close enough to real data for most purposes

UK Post code: [A-Y]{2}([1-9]|[1-9]{2}) ([1-9]|[1-9]{2})[A-Z]{2}

Doesn't really account for London Post Codes, but if you aren't actually validating addresses against post codes could save you lots of typing.

Hope this helps,

Neil Kidd

As promised I have listed a number of useful links to get you started or help expand your existing knowledge of Testing using Visual Studio Team System

General Resources

Visual Studio Team System 2008 Test Developer Center

http://msdn2.microsoft.com/en-gb/vsts2008/test/default.aspx

Visual Studio Team System 2008 Test Edition provides a comprehensive suite of testing tools for Web applications and services that are integrated into the Visual Studio environment. These testing tools enable testers to author, execute, and manage tests and related work items—all from within Visual Studio

Microsoft Tester Centre

http://msdn2.microsoft.com/en-gb/testing

The Microsoft Tester Centre showcases the test discipline as an integral part of the application lifecycle, describes test roles and responsibilities, and promotes the test investments required to deliver high-quality software.

Testhouse – Our VSTS Testing Partners

http://www.testhouse.org

Testhouse is a global company specialising in Quality Assurance Consulting, Testing Solutions and Training.  They are a member of the Visual Studio Inner Circle programme and the main UK Partner focusing on testing using Visual Studio Team System.

Getting Started

If you’re completely new to Testing with Visual Studio Team System then I suggest you start with these short tutorials to give you an idea what is possible:

Short Video tutorials (<10 Mins)

http://www.asp.net/learn/vsts-videos/

Including:

Introduction to Manual Testing with Team System                                           (8 minutes, 56 seconds)

Introduction to Managing and Running Tests with Team System                (8 minutes, 37 seconds)

Introduction to Testing Web Applications with Team System                       (9 minutes, 43 seconds)

Introduction to Load Testing Web Applications with Team System            (9 minutes, 8 seconds)

Introduction to Unit Testing with Team System                                                  (10 minutes, 54 seconds)

When you’ve installed your trial copy of Visual Studio Team Suite 2008 that you picked up at the conference (see downloads below if you missed out) and you’re ready for some more detail and walkthroughs then have a look at these links:

Getting Started with Team System Testing Tools

http://msdn2.microsoft.com/en-us/library/ms243146.aspx

Working with Manual tests

http://msdn2.microsoft.com/en-us/library/ms182615.aspx

Working with Web tests

http://msdn2.microsoft.com/en-us/library/ms182536.aspx

Working with Load Tests

http://msdn2.microsoft.com/en-us/library/ms182561.aspx

Database Unit Testing with Team Edition for Database Professionals

http://msdn2.microsoft.com/en-us/library/bb381703(VS.80).aspx

Some Additional Resources

Patterns & Practices Performance Testing Guidance for Web Applications

http://www.codeplex.com/PerfTestingGuide

Testing WebCasts on Demand

http://www.microsoft.com/events/series/Msdnvsts.aspx?tab=webcasts&id=42505

Ed Glass’ Content Index for Web Tests and Load Tests

http://blogs.msdn.com/edglas/pages/content-index-for-web-tests-and-load-tests.aspx

Windows Communication Foundation Load Testing

http://www.codeplex.com/WCFLoadTest

Blogs

Ed Glass on Load Testing

http://blogs.msdn.com/edglas/

IM Testy Blog

http://blogs.msdn.com/imtesty   

Dustin Andrews Blog

http://blogs.msdn.com/dustin_andrews/

Sean Lumley

http://blogs.msdn.com/slumley

Bill Barnett

http://blogs.msdn.com/billbar

Dennis Stone

http://blogs.msdn.com/densto   

Mike Taute

http://blogs.msdn.com/mtaute

Trials

Try Visual Studio Team System 2008 - Trial Software

http://msdn2.microsoft.com/en-gb/vsts2008/products/cc268310.aspx

Visual Studio® Team System Code Name "Rosario" April 2008 CTP (VPC Image)

http://www.microsoft.com/downloads/details.aspx?familyid=65d0e3bd-9df3-421a-804f-8f01bd90f0b4&displaylang=en&tm

Richard Erwin

Last Thursday I delivered a short presentation on Testing @ Microsoft as part of the Test Expo conference in London.  I have uploaded the presentation here

I will post some useful test related links at the beginning of next week so check back later!

The presentation is in *.pptx format so if you aren't using Office 2007 then you might need the Compatibility Pack or the PowerPoint 2007 Viewer

Richard Erwin

Just in case you haven't seen it, Sara Ford has been doing a heroic job of detailing all the tips and tricks for working in Visual Studio. Some how she manages to send out a new one every day and they are worth knowing about - so far she is up to number 186!

For the complete list, see:

  http://blogs.msdn.com/saraford/archive/tags/Visual+Studio+2008+Tip+of+the+Day/default.aspx

Well worth a read, and don't forget to subscribe to the RSS feed to keep up-to-date with what ever she find out next.

Nice one Sara.

Neil Kidd.

Last week I presented a session on Team System at this years QCon. Now this is traditionally more of a Java conference than a Microsoft one; however I the feedback I got and the discussion I had during the week were very interesting and encouraging. From an interoperability point of view, TFS has a great story with the TFS MSSCCI provider and products from our partners (like Teamprise's Eclipse plugins and non windows command line tools).

Unfortunately the demos I gave aren't really in a format that I can easily share, but I've uploaded the slides I used to our SkyDrive ...

I'm hoping to record some screencasts of the demos, so watch this space :-)

Neil Kidd

Registration has opened for the latest set of MSDN Roadshows. This year we will be in:

Cardiff - April 21st

London - April 25th

Manchester - May 1st

Glasgow - May 13th

Newcastle - May 16th

The agenda isn't just about Team System, in fact we only get one quarter of the agenda. The other subjects are ADO.Net Entity Framework and Data Services, ASP 3.5 and Silverlight. In my mind this is a great content line up showing the major new technologies that will be affecting software development for the next few years.

For more information and the links to sign up, see:

  MSDN Roadshow

If you can make it, please come and say hi :-)

Neil Kidd