Sachin Rekhi's Ramblings : Database Unit Testing

Database Unit Testing Whitepaper Published

SachinRe — Sat, 17 Mar 2007 01:48:00 GMT

I wanted to let you all know that I just published my detailed whitepaper on database unit testing.

Summary: Microsoft Visual Studio 2005 Team Edition for Database Professionals is the first toolset to deliver a comprehensive framework for performing database unit tests. This product brings to the data community the same first-class capabilities for performing unit tests that many application developers enjoy today. This paper explains exactly what database unit testing is, shows you how to develop database unit tests by using the framework, and walks you through how to leverage its rich capabilities to develop an effective strategy for performing database unit tests inside your organization. (21 printed pages)

You can check it out here!

http://msdn2.microsoft.com/en-us/library/bb381703(VS.80).aspx

Sachin Rekhi

Programmatic deployment sample

SachinRe — Tue, 02 Jan 2007 22:30:00 GMT

Jamie Laflen, Tech Lead for database unit testing, has put together another great sample showing you how to programmatically deploy your database project using the database unit testing framework.

The scenario here is that you can control the deployment of the database project to happen whenever you want it to during the database unit testing lifecycle. This gives you much more flexibility compared to the automatic deployment supported through the database unit testing UI. This can also be used for deploying your database project outside of the context of database unit testing in a programmatic way.

The sample is in the form of a console application that supports deployment of your database project through a command-line argument specifying the configuration file.

//-----------------------------------------------------------------------

// This file is part of:

// Visual Studio Team Edition for Database Professionals

// This source code is intended only as a supplement to Microsoft

// Development Tools and/or on-line documentation. See these other

// materials for detailed information regarding Microsoft code samples.

// THIS CODE AND INFORMATION ARE PROVIDED AS IS WITHOUT WARRANTY OF ANY

// KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE

// IMPLIED WARRANTIES OF MERCHANTABILITY AND/OR FITNESS FOR A

// PARTICULAR PURPOSE.

//-----------------------------------------------------------------------

using System;

using System.Collections.Generic;

using System.Text;

using System.Diagnostics;

using System.IO;

using System.Configuration;

using Microsoft.VisualStudio.TeamSystem.Data.UnitTesting.Configuration;

using Microsoft.VisualStudio.TeamSystem.Data.UnitTesting;

namespace DeployDBSample

{

class Program

{

FileInfo _configFile;

[STAThread()]

static void Main(string[] args)

{

try

{

ConsoleTraceListener l = new ConsoleTraceListener();

l.TraceOutputOptions = TraceOptions.Timestamp;

Trace.Listeners.Add(l);

Trace.AutoFlush = true;

Debug.AutoFlush = true;

Program p = new Program();

p.Run(args);

}

catch (Exception ex)

{

Trace.TraceError("Unhandled exception {0}{1}", Environment.NewLine, ex);

}

public Program()

{

}

void Run(string[] args)

{

if (ValidArgs(args) == false)

{

PrintUsage();

return;

}

// Load the config section

string sectionName = "DatabaseUnitTesting";

ExeConfigurationFileMap map = new ExeConfigurationFileMap();

map.ExeConfigFilename = _configFile.FullName;

Configuration config = ConfigurationManager.OpenMappedExeConfiguration(map, ConfigurationUserLevel.None);

DatabaseUnitTestingSection section = config.GetSection(sectionName) as DatabaseUnitTestingSection;

if (section == null)

{

Console.WriteLine("Could not load section {0}", sectionName);

return;

}

// If the Database deployment section is defined, attempt to deploy the DBProject

if ( section.DatabaseDeployment != null

&& string.IsNullOrEmpty(section.DatabaseDeployment.DatabaseProjectFileName) == false)

{

// Deploy the database project

HelperService s = new HelperService();

s.Deploy(

section.DatabaseDeployment.DatabaseProjectFileName,

section.DatabaseDeployment.Configuration,

section.PrivilegedContext.Provider,

section.PrivilegedContext.ConnectionString);

}

#region Console app helpers

void PrintUsage()

{

string exeName = this.GetType().Assembly.Location;

string usage = string.Format("Usage: {0} configFile", exeName);

Console.WriteLine(usage);

}

bool ValidArgs(string[] args)

{

bool isValid = true;

if (args == null || args.Length != 1)

{

return false;

}

// The first argument is the configuration file

_configFile = new FileInfo(args[0]);

if (_configFile.Exists == false)

{

isValid = false;

}

return isValid;

}

#endregion

class HelperService : DatabaseTestService

{

public void Deploy(

string dbProjectFileName, string projectConfig,

string providerName, string connectionString)

{

DeployDatabaseProject(dbProjectFileName, projectConfig, providerName, connectionString);

}

Take it for a test drive!

Jamie Laflen

Sachin Rekhi

Disabling triggers to support data generation

SachinRe — Tue, 02 Jan 2007 21:47:00 GMT

Jamie Laflen, Tech Lead for database unit testing, has developed a code sample to show you how to automatically disable triggers to support data generation.

The scenario is that you have delete or insert triggers defined on your database that inhibit proper test data generation. So this sample code will allow you to modify your database unit test setup such that before generating test data, the triggers will be disabled and then re-enabled after the data has been appropriately generated. This is an important scenario that customers have been running into when attempting to generate data for, say, the AdventureWorks database that has delete triggers defined.

To use the sample code, replace your existing DatabaseSetup.cs in your test project with the code found below. You will have to update the namespace, etc. to be appropriate for your test project. The sample code uses SMO to determine what insert and delete triggers exist in your database and then appropriately disables them. Make sure you thus add the appropriate reference to SMO to your test project.

//-----------------------------------------------------------------------

// This file is part of:

// Visual Studio Team Edition for Database Professionals

// This source code is intended only as a supplement to Microsoft

// Development Tools and/or on-line documentation. See these other

// materials for detailed information regarding Microsoft code samples.

// THIS CODE AND INFORMATION ARE PROVIDED AS IS WITHOUT WARRANTY OF ANY

// KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE

// IMPLIED WARRANTIES OF MERCHANTABILITY AND/OR FITNESS FOR A

// PARTICULAR PURPOSE.

//-----------------------------------------------------------------------

using System;

using System.Collections.Generic;

using System.Data;

using System.Data.Common;

using System.Configuration;

using System.Data.SqlClient;

using System.Diagnostics;

using Microsoft.VisualStudio.TestTools.UnitTesting;

using Microsoft.VisualStudio.TeamSystem.Data.UnitTesting;

using Microsoft.SqlServer.Management.Smo;

using Microsoft.SqlServer.Management.Common;

namespace TestProject1

{

[TestClass()]

public class DatabaseSetup

{

[AssemblyInitialize()]

public static void IntializeAssembly(TestContext ctx)

{

// Setup the test database based on setting in the

// configuration file

DatabaseTestClass.TestService.DeployDatabaseProject();

ConnectionContext triggerCtx = DatabaseTestClass.TestService.OpenPrivilegedContext();

List<Trigger> disabledTriggers =

DisableTriggersForPopulate(triggerCtx.Connection as SqlConnection);

try

{

DatabaseTestClass.TestService.GenerateData();

}

finally

{

EnableTriggers(disabledTriggers);

triggerCtx.Dispose();

}

/// <summary>

/// Disables all Insert and Delete triggers defined and enabled on tables

/// in the database connected to by the supplied connection string.

/// <remarks>

/// This disables all triggers by enumerating all tables (and their triggers)

/// in the entire database. This can have a negative performance, if performance

/// becomes an issue, a script that directly disables/enables triggers should be

/// used instead.

/// </remarks>

/// </summary>

/// <param name="targetConn"></param>

/// <returns></returns>

private static List<Trigger> DisableTriggersForPopulate(SqlConnection targetConn)

{

if (targetConn == null)

{

throw new ArgumentNullException("targetConn");

}

ServerConnection serverConn = new ServerConnection(targetConn);

Server targetServer = new Server(serverConn);

Database db = targetServer.Databases[targetConn.Database];

// Loop through all tables and disable all Insert/Delete triggers

List<Trigger> disabledTriggers = new List<Trigger>();

foreach (Table currentTable in db.Tables)

{

foreach (Trigger currentTrigger in currentTable.Triggers)

{

if (currentTrigger.IsEnabled)

{

if (currentTrigger.Insert || currentTrigger.Delete)

{

currentTrigger.IsEnabled = false;

currentTrigger.Alter();

Debug.Print("Disabled {0}", currentTrigger.Name);

disabledTriggers.Add(currentTrigger);

}

return disabledTriggers;

}

/// <summary>

/// Enables all supplied triggers

/// </summary>

/// <param name="triggers"></param>

private static void EnableTriggers(List<Trigger> triggers)

{

if (triggers == null)

{

return;

}

foreach (Trigger t in triggers)

{

t.IsEnabled = true;

t.Alter();

Debug.Print("Enabled {0}", t.Name);

}

Hope that helps!

Jamie Laflen

Sachin Rekhi

Database Unit Testing Whitepaper Draft

SachinRe — Wed, 20 Dec 2006 23:02:00 GMT

So I’ve spent the last week authoring a detailed whitepaper on Team Edition for Database Professional’s database unit testing framework. I thought I would post the draft online here prior to publishing it to MSDN.

Summary: Team Edition for Database Professionals is the first toolset to deliver a comprehensive database unit testing framework, bringing the same first-class unit testing capabilities many application developers enjoy today to the data community. This paper explains exactly what database unit testing is, shows you how to go about developing database unit tests using the framework, as well as walks you through how to leverage its rich capabilities to develop an effective database unit testing strategy inside of your organization. (23 pages)

Please let me know any feedback that you have!

Sachin Rekhi

Change in Database Unit Testing RAISERROR Behavior in CTP 7

SachinRe — Sat, 11 Nov 2006 02:46:00 GMT

We have changed the behavior of the usage of the RAISERROR command in database unit tests. Prior to CTP7, a RAISERROR statement of any severity greater than 0 would cause the database unit test to fail. We have changed this behavior based on feedback such that the database unit test will only fail if the RAISERROR severity is 11 or greater.

This did not conform with how the industry uses RAISERROR - i.e. a RAISERROR of less that 11 does not fail the transaction and processing continues on the server.

So, to update your database unit tests, simply change the RAISERROR statement to have a higher severity as shown below:

Original Code

RAISERROR('Fail test',1,1)

Updated Code

RAISERROR('Fail test',11,1)

Sachin Rekhi

Change in Test Conditions \ Generators Extension File in CTP7

SachinRe — Sat, 11 Nov 2006 02:32:00 GMT

I wanted to make sure everyone was aware of a change that we have made in the extensions file for both custom Test Conditions & Generators in CTP7.

We now perform xsd validation over the xml file and thus require the extensions xml to appropriately map to the required xsd. This thus requires all extension files to be updated to incorporate this. You can see an example extension file below. I have highlighted the new attributes that must be added.

<?xml version="1.0" encoding="us-ascii"?>

<extensions assembly="Generators, Version=1.0.0.0, Culture=neutral, PublicKeyToken=<Public Key>" version="1" xmlns="urn:Microsoft.VisualStudio.TeamSystem.Data.Extensions" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="urn:Microsoft.VisualStudio.TeamSystem.Data.Extensions

Microsoft.VisualStudio.TeamSystem.Data.Extensions.xsd">

</extensions>

Sachin Rekhi

DevConnections Presentation: Database Unit Testing

SachinRe — Sat, 11 Nov 2006 02:19:00 GMT

I wanted to make available the other presentation that I gave at Dev Connections in Las Vegas, Nevada this week as well. You can find the title, synopsis, and powerpoint deck below.

SDB310: Database Unit Testing with Team Edition for Database Professionals

Come learn how to best leverage Team Edition for Database Professionals for developing a database unit testing strategy for your organization. Learn how you can develop a suite a database unit tests to ensure the quality of your database. Dive into generating associated test data, managing the database state with transactions, creating data-driven unit tests, and adding your own custom client-side test conditions.

Sachin Rekhi

DevConnections Presentation: The Database Development Process

SachinRe — Sat, 11 Nov 2006 02:02:00 GMT

This week I attended the Dev Connections series of conferences in Las Vegas, Nevada. I had a great time giving several presentations on Team Edition for Database Professionals, as well as interacting with many excited customers about their experiences thus far with the product.

I thought I would make available to everyone the presentations that I gave at the conference. You can find below the title and synopsis as well as the attached powerpoint presentations.

VMS308: The Database Development Process

Visual Studio Team Edition for Database Professionals provides a new set of tools for working with databases throughout the development process and beyond. But just having tools is not enogh. In order to reap the benefits of this new way of working with databases, you also need to embrace a new process for the way you do the work. In this session, we will discuss the new MSF Agile and CMMI processes and show how with a combination of the new Visal Studio Team Edition for Database Professionals tools and the adoption of slightly new processes, you can see significant benefits both in productivity and quality.

Sachin Rekhi

Test Case Management

SachinRe — Tue, 24 Oct 2006 04:07:00 GMT

If you have both Team Data and Test Test (or simply Team Suite), you’ll be able to take advantage of the Test Manager Tool Window and test lists. This provides more sophisticated test case management capabilities.

The most useful aspect of this is the ability to create test lists. You can easily group all tests associated with whatever database objects you are working on into a test list and then repeatedly execute that test list. You can organize test lists in a variety of ways. It could be by database object, by object owner, etc. This makes it very easy to run the relevant set of tests.

This is also very useful for creating a check-in test suite. Simply add the tests that you wish users to execute prior to any code check-ins to a test list. Then you can setup a Testing Check-in Policy to enforce that the test list is run prior to all checkins. For more info on this, check out this previous post.

Check out this MSDN article to show you just how to create a test list.

Sachin Rekhi

The Many Ways to Execute Database Unit Tests

SachinRe — Tue, 24 Oct 2006 01:48:00 GMT

I wanted to make sure you were all aware of the various methods of executing database unit tests. I’ve talked with some users who were pleasantly surprised to learn of several of the other options, so I thought I’d throw it up on my blog for everyone’s delight. These should help for those trying to quickly and efficiently implement the dev\test cycle that you should be doing with every change you make to your schema objects.

Top Level Test Menu Items

You can use the following two top-level menus to execute tests:

§ Test -> Start Selected Test Project with Debugger (Shift+Alt+X)

§ Test -> Start Selected Test Project without Debugger (Ctrl+Shift+X)

To use these, the test project containing the tests which you wish to run must be the active project in Solution Explorer. These commands will execute all the tests inside the currently selected test project.

Test View Window

You can use the Test View to quickly run specific tests. To bring up the Test View Window, go to Test -> Windows -> Test View. This brings up a nice little window that you can dock side-by-side with, say, Solution Explorer. Then you simply select the tests which you wish to run, right-click, and say “Run Selection”. The Test View also has simple filtering capabilities, so you can easily find the specific test you are looking for.

Test Manager Window

The Test Manager can also be used for executing database unit tests. The Test Manager provides a more sophisticated view of your tests and some test case management capabilities. To open the Test Manager, select Test -> Windows -> Test Manager. The Test Manager offers the same capabilities of the Test View and more. You can check a set of tests that you wish to run and execute them. More importantly, you can define tests lists and execute the tests list as a single unit. This is very useful for setting up check-in test suites that you ask your developers to execute prior to every check-in.

Test Results Window

The Test Results window appears whenever you execute tests to report to you the status of those tests. If a test has failed, it’s very easy to re-execute the test from the Test Results Window. Simply select the test and press the run toolbar button.

You can make the process of making database changes and the cycle of build, deploy, generate test data, and re-run tests very simple through the use of F5.

To set this up, make the test project the startup project. Right-click on the test project and select “Set as Startup Project”. Then go to Tools -> Options -> Projects and Solutions -> Build and Run and check “Only build startup projects and dependencies on Run”. Then ensure that you have configured in Test -> Database Test Configuration to automatically deploy and generate test data.

Now whenever you make changes to your database schema, simply select the test project in Solution Explorer and press F5! The database project will be built and deployed, test data will be generated, and your tests will be re-run. This makes iterative development and testing very simple.

Command-line

You can checkout a previous blog post that describes just how to go about using the mstest.exe command-line tool to execute database unit tests.

And here is a link to the MSDN Documentation drilling in deeper on running Tests.

Hope that helps!

Sachin Rekhi

Video on adding transaction support to db unit tests

SachinRe — Fri, 13 Oct 2006 00:19:00 GMT

Learn how to add transaction support to your database unit tests in Team Edition for Database Professionals in order to better manage the database state during your test run.

http://channel9.msdn.com/ShowPost.aspx?PostID=244523

Sachin Rekhi

Adding transaction support to database unit tests

SachinRe — Thu, 12 Oct 2006 22:54:00 GMT

I wanted to describe how to add transaction support to your database unit tests. In this previous post I motivate why you would want to do this. It’s all about managing the database state prior to and during your test run. For managing the database state at the beginning of a database unit test run, Team Edition for Database Professionals’ data generation capabilities ensure that a repeatable set of test data is appropriately generated in the database prior to executing tests.

The next important place where state must be appropriately maintained is between database unit tests. As I mention in the previous post, there are 2 ways to achieve this. One is to author the test in such a way that the state is appropriately cleaned up as part of the test. You can imagine a test for a CreateAuction stored procedure ensuring that it deletes the auction that was created as part of the test script.

This process can be simplified if you utilize transactions in your database unit tests. The idea here is that you open a database transaction at the beginning of your test, perform your test, and then rollback the transaction at the end of the test to return your database to the state prior to running this test. This will ensure that all database unit tests execute with the state that was originally created for the test run by your data generation plan.

There are several ways to go about adding transaction support to a database unit test. I’ll walk you through those approaches here.

Let me first set up the scenario. Let’s say we start out with the Northwind database, which supports an online e-tailer’s storefront. We have recently added online auctions support to the database and thus have tables like Auctions, AuctionTypes, Bids, and their associated relationships with the existing tables. In addition, we have a couple of new stored procedures: GetAllAuctions and CreateAuction. Let’s say we want to create a database unit test for the CreateAuction stored procedure.

CreateAuction

CREATE PROCEDURE CreateAuction

@AuctionTitle nvarchar(50),

@AuctionDescription nvarchar(2000),

@AuctionStartDate datetime,

@AuctionLength int,

@ProductID int,

@ItemQuantity int,

@EmployeeID int,

@AuctionTypeID int,

@MininumBidAmount money

INSERT INTO Auctions

(

AuctionTitle,

AuctionDescription,

AuctionStartDate,

AuctionLength,

ProductID,

ItemQuantity,

EmployeeID,

AuctionTypeID,

MinimumBidAmount

)

VALUES

(

@AuctionTitle,

@AuctionDescription,

@AuctionStartDate,

@AuctionLength,

@ProductID,

@ItemQuantity,

@EmployeeID,

@AuctionTypeID,

@MininumBidAmount

)

RETURN SCOPE_IDENTITY()

So a typical database unit test for this stored procedure might look like the following:

dbo_CreateAuctionTest

-- database unit test for dbo.CreateAuction

DECLARE @AuctionID Int,

@AuctionTitle NVarChar( 50 ),

@AuctionDescription NVarChar( 2000 ),

@AuctionStartDate DateTime,

@AuctionLength Int,

@ProductID Int,

@ItemQuantity Int,

@EmployeeID Int,

@AuctionTypeID Int,

@MininumBidAmount Money

SELECT @AuctionID = 0,

@AuctionTitle = 'Chai',

@AuctionDescription = '2 25-pack boxes of India Chai',

@AuctionStartDate = '10/12/2006',

@AuctionLength = 10,

@ProductID = 1,

@ItemQuantity = 2,

@EmployeeID = 1,

@AuctionTypeID = 1,

@MininumBidAmount = 10.00

EXEC @AuctionID = [dbo].[CreateAuction] @AuctionTitle , @AuctionDescription , @AuctionStartDate , @AuctionLength , @ProductID , @ItemQuantity , @EmployeeID , @AuctionTypeID , @MininumBidAmount

--query AuctionTitle, AuctionStartDate of created auction

--to verify appropriate results

SELECT AuctionTitle, AuctionStartDate

FROM Auctions WHERE AuctionID = @AuctionID

--delete created auction to return db to previous state

DELETE FROM Auctions WHERE AuctionID = @AuctionID

In this database unit test, we are managing the database state by cleaning up the addition of the auction by executing a DELETE SQL command. Now let’s look at how we can enhance this database unit test to use transactions instead to manage the database state during the test run.

Approach 1 – Transactions in T-SQL

The simplest approach to adding transaction support is to merely add the appropriate begin and rollback transaction statements to the beginning and end of your test script.

So in the above test script the first step I did was comment out the DELETE FROM SQL statement:

--delete created auction to return db to previous state

--DELETE FROM Auctions WHERE AuctionID = @AuctionID

Then I added at the top of the test script the following statement to begin the transaction:

--begin database transaction

BEGIN TRANSACTION

And finally I added the following statement at the end of the test script to rollback the transaction:

--rollback database transaction

ROLLBACK TRANSACTION

That’s all you need to do to add transaction support to a specific database unit test!

Approach 2 – System.Transactions in C#\VB

The second approach is to modify the generated C#\VB code to use System.Transactions. This approach allows you to easily add transaction support to all database unit tests by simply adding a few lines of code to your test class.

The first thing we are going to want to do is jump to the code view for our database unit test class. Simply right-click on the test class in Solution Explorer, and select View Code.

Now we want to declare a member variable inside the test class for our transaction. You can add this near the top of the test class:

//member variable for the transaction scope

TransactionScope _ts;

Then we’ll want to modify the Test Initialize to instantiate the transaction scope. This is all we need to do to start the ambient transaction. Make sure that you add the instantiation line above the call to base.InitializeTest():

[TestInitialize()]

public void TestInitialize()

{

//instantiate the transaction scope

_ts = new TransactionScope();

base.InitializeTest();

}

We want to similarly modify the Test Cleanup to dispose of the transaction:

[TestCleanup()]

public void TestCleanup()

{

base.CleanupTest();

//dispose of transaction scope

_ts.Dispose();

}

Since the Test Initialize and Test Cleanup run prior to every database unit test in the test class, all of our database unit tests are now appropriately transacted!

Note: You may need to manually enable the Distributed Transaction Coordinator in order for you to take advantage of TransactionScope. To do this, you can simply start the Distributed Transaction Coordinator service by going to Start -> Settings -> Control Panel -> Administrative Tools -> Services.

I hope that shows you how easy it is to add transaction support to your database unit tests to manage the database state during a database unit test run.

Sachin Rekhi

Video on Creating a Custom Test Condition

SachinRe — Thu, 12 Oct 2006 19:37:00 GMT

I just put together a 15 minute video on just how you go about creating a custom test condition. So check it out! You’ll see how quickly you can code, deploy, and use a custom test condition to extend the database unit testing functionality in Team Edition for Database Professionals.

You can find the video on Channel 9 here:

http://channel9.msdn.com/Showpost.aspx?postid=244377

Look at this post for additional details on creating a custom test condition:

http://blogs.msdn.com/sachinre/archive/2006/08/22/Creating-a-custom-test-condition_3A00_-ResultSet-Column-Count.aspx

As always, let me know if you have any questions!

Sachin Rekhi

Extensibility in Team Edition for Database Professionals

SachinRe — Wed, 30 Aug 2006 22:48:00 GMT

I just gave a presentation at the annual Visual Studio Partner Summit on extensibility points in Team Edition for Database Professionals. So I thought it would be a great opportunity to share this on my blog as well.

When I think of the extensibility points, I typically group them in three categories. The first are the core extensibility points within our product. Then there are the Team System integration extensibility points, as well as the Process Guidance extensibility points.

Team Data Extensibility

§ Data Generators \ Distributions. The data generation feature allows you to bind specific data generators to each table column. We ship in the box with a motley of generators. We have simple generators for every SQL data column. We also have some more powerful generators including the Regular Expression Generator and the Data Bound Generator. You have the opportunity to build your own custom generators to improve the meaningfulness of the data generated for your environment. In addition, you can specify a distribution that you want your generated values to adhere to. We ship with a set of distributions, but those are completely customizable as well.

Ideas

w Person Generator. Most databases have person information, like names, addresses, birthdates, etc. You can imagine creating a custom generator to provide these sorts of values easily for these tables.

w XML Generator. You can imagine creating a custom generator that would take in an xml schema and generate valid xml adhering to that schema.

w Credit Card Generator. You can create a generator that will generate credit card numbers according to an appropriate format (AMEX, Visa, etc)

w Logarithmic Distribution. You can create a distribution that adheres to the logarithmic curve.

§ Test Conditions. The database unit testing feature allows you verify tests using either SQL assertions or easily configurable UI client-side test conditions. We ship a set of test conditions in the box, including row count, scalar value, empty resultset, etc. But these test conditions are completely extensible so you can imagine creating your own to do more powerful test verification.

Ideas

w Data Compare Test Condition. People will want the ability to verify that the resultset data returned is as expected. You can imagine creating a test condition to verify this.

w Schema Compare Test Condition. In addition to just comparing the data, the schema of the returned resultsets is also important. For example, verifying the right number of columns and of the appropriate data types are returned.

§ Build & Deploy. We have 2 MS Build tasks for build and deploy inside the project system. These are standard MS Build tasks and thus can be extended using MS Build.

Ideas

w Deploy to server farm MS Build Task. People will want to deploy to an entire farm of servers and you can imagine making that process simpler.

w Data Motion MS Build Task. While we ensure that we never lose data, we do not appropriately migrate data for you automatically. You could add an MS Build task to do this work for you.

Team System Extensibility

§ Work Items \ Queries \ Reports. Team System allows you to completely customize the work items types and associated fields. You could add, for example, database specific information into these work items. You could then create custom queries and reports that were based off of these fields.

§ Check-in Policies. Team System also allows you to create custom check-in policies that require certain actions to be performed prior to check-in. For example, a testing policy that ships with TFS enforces that a specific set of tests is run prior to checking in your code. You can implement other such db specific policies if you desired.

Process Guidance Extensibility

§ Process Guidance Templates. Team System ships with a set of process guidance templates, including MSF Agile and MSF for CMMI. Many partners have already creating their own process guidance templates, including Scrum. You can go about creating your own custom process guidance templates. This includes not only prescriptive guidance for roles, workstreams, and activities, but associated work items, reports, queries, check-in policies, etc.

I hope that has given you a slew of ideas on how to extend Team Edition for Database Professionals. Of course there are other ways to extend the product through standard Visual Studio Extensibility.

Got other ideas? Wondering if something is possible? Let me know!

Sachin Rekhi

Creating a custom test condition: ResultSet Column Count

SachinRe — Tue, 22 Aug 2006 19:59:00 GMT

I wanted to walk you through just how you can go about creating your own custom test condition. This test condition will be added to the existing suite of in the box conditions and can be leveraged just like all of the other conditions inside the database unit testing designer.

Let me first motivate the sample test condition we are about to build. In the box we offer several “smoke test” types of test condition validation. For example, checking whether the returned resultset is nonempty allows you to quickly verify that the test script at least returns some rows. Similarly, the Row Count condition allows you to quickly check whether the appropriate number of rows were returned. However, almost as important as the data that is returned is the schema of the resultsets returned. You may want to create simple verification to ensure that the schema of the returned resultset is what we expect. In this example we’ll create a simple test condition that verifies that the number of columns returned by the resultset is what we expect. This is a quick smoke test to ensure that the contract for say a stored procedure is in fact intact.

Creating a custom test condition has three key steps

1. Code the test condition class

2. Deploy the test condition

3. Use the test condition

Code the test condition class

1. Create a new C# Class Library project in Visual Studio 2005 named SachTestConditions

(Of course you can use any .NET language, but here we walk you through a C# example)

2. Add the following references to your project:

Microsoft.VisualStudio.QualityTools.UnitTestFramework.dll

Microsoft.VisualStudio.TeamSystem.Data.UnitTesting.dll

(C:\Program Files\Microsoft Visual Studio 8\DBPro\Microsoft.VisualStudio.TeamSystem.Data.UnitTesting.dll)

3. Rename the Class1.cs file to ResultSetColumnCountCondition.cs

4. Copy the following code into ResultSetColumnCountCondition.cs

//ResultSetColumnCountCondition

//Sample custom test condition

//Team Edition for Database Professionals

//Author: Sachin Rekhi

//Date: 8.22.06

using System;

using System.Collections.Generic;

using System.Text;

using System.Data;

using System.Data.Common;

using System.ComponentModel;

using System.ComponentModel.Design;

using TestTools = Microsoft.VisualStudio.TestTools.UnitTesting;

using Microsoft.VisualStudio.TeamSystem.Data.UnitTesting;

using Microsoft.VisualStudio.TeamSystem.Data.UnitTesting.Conditions;

namespace SachTestConditions

{

[DisplayName("ResultSet Column Count")]

public class ResultSetColumnCountCondition : TestCondition

{

private int _resultSet;

private int _count;

private int _batch;

public ResultSetColumnCountCondition()

{

_resultSet = 1;

_count = 0;

_batch = 1;

}

//method you need to override

//to perform the condition verification

public override void Assert(DbConnection validationConnection, ExecutionResult[] results)

{

//call base for parameter validation

base.Assert(validationConnection, results);

//verify batch exists

if (results.Length < _batch)

throw new TestTools.AssertFailedException(String.Format("Batch {0} does not exist", _batch));

ExecutionResult result = results[_batch - 1];

//verify resultset exists

if (result.DataSet.Tables.Count < ResultSet)

throw new TestTools.AssertFailedException(String.Format("ResultSet {0} does not exist", ResultSet));

DataTable table = result.DataSet.Tables[ResultSet - 1];

//actual condition verification

//verify resultset column count matches expected

if (table.Columns.Count != Count)

throw new TestTools.AssertFailedException(String.Format(

"ResultSet {0}: {1} columns did not match the {2} columns expected",

ResultSet, table.Columns.Count, Count));

}

//this method is called to provide the string shown in the

//test conditions panel grid describing what the condition tests

public override string ToString()

{

return String.Format(

"Condition fails if ResultSet {0} does not contain {1} columns",

ResultSet, Count);

}

//below are the test condition properties

//that are exposed to the user in the property browser

#region Properties

//property specifying the resultset for which

//you want to check the column count

[Category("Test Condition")]

[DisplayName("ResultSet")]

[Description("ResultSet Number")]

public int ResultSet

{

get { return _resultSet; }

set

{

//basic validation

if (value < 1)

throw new ArgumentException("ResultSet cannot be less than 1");

_resultSet = value;

}

//property specifying

//expected column count

[Category("Test Condition")]

[DisplayName("Count")]

[Description("Column Count")]

public int Count

{

get { return _count; }

set

{

//basic validation

if (value < 0)

throw new ArgumentException("Count cannot be less than 0");

_count = value;

}

#endregion

}

Let’s analyze the code that we’ve just copied into our class. We started by adding the appropriate using statements. We then had our class inherit from the base TestCondition class.

We then added properties to our test condition. These are the properties that the user will be able to configure from the property browser inside of Visual Studio once they add your custom condition. For this example, we added 2 properties. The first is the ResultSet property, which allows the user to specify the number of the ResultSet they wish for this condition to check the column count on. The second property is the Count property, allowing the user to specify the expected column count. You can use the same attributes available to you through the component model. These are understood by our designer as well as by the Visual Studio Property Browser. As you can see here, we added 3 attributes – the category name to organize the properties under, the display name of the property, as well as a description. We also do some basic input validation which will be appropriately displayed to the user on an input error.

The most important next step is to override the Assert method. This method does the real work in validating that the expected condition is actually met. As you can see, the method provides you with two parameters. The key parameter is the results array. This returns a single array element for each batch executed. In V1, only a single batch is supported per test script, so your test condition will always interrogate the first array element. The most important property of the ExecutionResult object is the contained DataSet. This contains the returned resultsets for the test script. So for this particular condition we need to interrogate the appropriate data table in the dataset and verify that it contains the appropriate number of columns. Since you are returned a standard ADO.NET dataset, you can use this familiar construct to perform the validation. The code shown above shows you exactly how to do this for our scenario.

5. You should now set your class library to be signed. You can do this by going to project properties, going to the Signing tab, and choosing Sign the assembly. Then specify a key or create a new one.

6. You can now go ahead and build the test condition

Deploy the test condition

1. The first thing you are going to want to do is add the test condition assembly to the GAC (Global Assembly Cache). You can do this using the gacutil.exe command. If both gacutil and your test condition assembly are in your path, you can use a command like the following. This will install (or force a re-install if it already exists) of the assembly.

gacutil /if SachTestConditions.dll

2. The next step is to register the test condition. You do this by adding a new extensions.xml file in the appropriate location. The default location is the following:

C:\Program Files\Microsoft Visual Studio 8\DBPro

Add the file SachTestConditions.extensions.xml to the directory with the following contents:

<?xml version="1.0" encoding="us-ascii"?>

<extensions assembly="SachTestConditions, Version=1.0.0.0, Culture=neutral, PublicKeyToken=<Public Key>" version="1" xmlns="urn:Microsoft.VisualStudio.TeamSystem.Data.Extensions" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="urn:Microsoft.VisualStudio.TeamSystem.Data.Extensions

Microsoft.VisualStudio.TeamSystem.Data.Extensions.xsd">

<extension type="SachTestConditions.ResultSetColumnCountCondition" enabled="true" />

</extensions>

Here we are registering the strongly named assembly and each of its contained conditions. In this example we have only our one condition. You must put the public key token for the signed assembly here as well. The easiest way to obtain the public key is to browse to the assembly directory in your Windows dir, find the binary that you just added to the GAC, and copy the public key token from there.

You can clearly automate this process and create a nice little setup tool if you wished to GAC the assembly and register it on the client machine. This way your entire organization can easily take advantage of your custom library of conditions. I’m not going to get into the details here, but you can use regular deployment technologies to accomplish this.

Use the test condition

§ Now we are ready to put the test condition to use. Make sure you close out of Visual Studio and re-open it. This is necessary because the test conditions are only loaded on initial load of Visual Studio and are not refreshed.

§ Add a database unit test to a new test project. When the database unit test designer appears, it should contain your custom test condition in the available test conditions drop-down.

§ Note: If this is the first time you are adding a database unit test to a test project, the appropriate assembly references to your custom conditions are automatically added. If the test project already contains database unit tests, you will need to manually add the reference to the custom test condition assembly.

§ Add the test condition to your test method and configure its properties.

§ And now you are all set to go!

I hope that gave you a comprehensive look at exactly what it takes to build your own custom test condition and add it to the library of built-in conditions.

We are very excited about this extensibility and hope it creates a rich community around database test verification.

Have questions? Let us know! We are hear to help J The forums are the best way to get your questions answered from the product group.

Sachin Rekhi