VS Team System for Database Professionals QA Blog

Visual Studio Team System will be holding a Web Chat on December 5th.

2007-12-05T00:49:00Z

If you're doing .NET or database testing, Team System is holding a web chat tomorrow that may be of interest to you.

And now, a word from my sponsor: ;)

Come chat with the Visual Studio Team System group tomorrow

Join members of the Visual Studio Team System product group to discuss features available in Team Foundation Server, Team Suite, Architecture Edition, Development Edition, Database Edition, and Test Edition. In addition, discuss what's new for these editions for Visual Studio 2008.

We will be holding two sessions:

Join the chat on Wednesday, December 5th, 2007 from 10:00am - 11:00am Pacific Time. Add to Calendar | Additional Time Zones

-and-

Join the chat on Wednesday, December 5th, 2007 from 4:00pm - 5:00pm Pacific Time. Add to Calendar | Additional Time Zones

Achieving breadth of testing while remaining agile. The Data-Driven P1 test: Our new bread & butter

2007-11-17T00:42:00Z

Wow, I just looked at the date of my last post and had no idea it had been that long. This have been rather crazy with VS 2008 signoff, hiring to do, and lots of feature team leadership matters to cover.

In my last post I talked about P0 tests being the mechanism by which we stay tightly coupled with dev work and get tests in place very quickly after feature implementation. This model continues to work well for us, though the 24-hour turnaround that we shoot for is not something we always achieve.

Anyway, once QA is actively engaged with Dev, has identified and implemented initial P0 tests, what's next on the priority list? Of course, you're going to need to expand both the breadth and depth of scenarios that you hit in the P0 tests. There are a number of ways that QA can go about this. Here are a couple of the approaches we've taken before:

The classic approach is to write up a detailed test plan, including all of the test cases you plan to automate. Once you have a few tests in place, you continually clone them and alter them, somewhat, for each new scenario. By the time you're done you have hundreds, if not thousands, of nearly-identical tests that have minor differences to cover various scenarios. The drawbacks of this approach are many, and in many ways are all due to having insufficient abstraction in your test automation:

If the feature design changes in such a way as to make the tests invalid, you must go in and repair hundres, if not thousands, of tests. "Clone and alter" is very fragile
The logical 'input data' for the test is hard-coded into the test itself. This not only means the test isn't flexible, but also that it is extremely difficult to review what is and is not being covered by your tests. It's bad engineering to merge your data and logic, but QA does it all the time.
Nobody outside of QA is really able to improve the test suite since devs are unlikely to go about cloning and altering tests that a QA owner has.

Utilize model-based testing (MBT) to maximize abstraction and code reuse as you generate a broad set of product tests. We've tried 3 different MBT tools over the last few product cycles I've been in. We've had some success, and there's no doubt that the approach forces greater test abstraction; nevertheless, we've moved away from using MBT for a few key reasons:

Individual tests become almost indecipherable to those who did not work on them
In order to avoid 'model explosion' (your model spits out tens of thousands of tests), you have to choke the model to the point that you lose key benefits of MBT
Test data is somewhat abstracted from individual tests, but the data sets are usually not easily accessible and often not very granular.
Using MBT you REALLY can't clearly explain to Dev what the purpose of each test is.
Getting a model going is very time-consuming. The approach is simply not at all agile.

OK, with all of this said (or, perhaps, resaid as I mentioned several of these points in earlier posts), what approach are we taking to meet our key goals of agility, data separation, code reuse, and transparency to other disciplines? Well, most all of our P1 tests are data-driven both in terms of input data as well as verification.

Some of you may have used data-driven testing through nUnit or VSTT in which you associate a test with a data-source for input data. That data source may be a table in SQL Server, an Excel spreadsheet, or, perhaps, an .mdb. We're taking this aproach a step or two further and, so far, we're seeing improved results across QA than we've seen in previous cycles and much better feedback from Dev as that discipline feels like it understands what it is that we're testing.

Our QA team has created and continues to expand and maintain a single SQL Server 2005 database dedicated to our team's data-driven testing. Within this database we have created a variety of schemas, one per feature area and one schema for shared data. Tables within the database are of two basic types, input data and verification data.

Input data tables contain all data and associated metadata (I know this sounds cryptic, but in subsequent posts I'll give specific details on some of the tables that we use in our team). Our QA feature owners are expected not only to cover all the scenarios they can with data in their input tables, but also to ensure that Dev and PM not only have the chance to review those data but also to encourage them to input additional data for the tests as they identify it. This focus on input data, rather than on imperitive test logic, means that Dev, PM, and other QA team members who don't own a given feature can contribute to the test coverage of that feature simply by knowing the breadth of input data that a feature should support.

Verification tables all have a foreign key relationship to an input data table. The data contained in a verification table may be pure baselines (a bad idea), or an abstraction such as RegEx, an XML representation of expected object state, or whatever. This means that a single input data table may have multiple associated Verification tables (in other words, there is a 1...n relationship between input tables and verification tables), each of which may verify some of the same data in different ways for different test cases. For most test cases, the input test data used in the test is limited to those data having associated verification.

My next post will go into specifics of some of our input and verification tables

"What is QA doing?!?": Moving QA into an iterative, Agile environment with 'P0' testing

2007-09-24T04:43:00Z

Our single biggest misstep in our test approach last version was that we failed to sufficiently change our testing methodologies to fit the Agile environment under which we developed our V1 product. As we had done in our previous years here at MS, we placed our emphasis on system-level, UI-based test automation, which had served us well in the past when working under more of a waterfall model.

So, what did our test approach look like heading into V1? Well, let me try to encapsulate a book-sized chunk of info into a succinct, semi-directed list:

Receive the initial feature specs
Review and provide feedback on the spec
Create a comprehensive test plan based on the spec. The focus of the test plan is at the system level, meaning that it is abstracted more or less at an end user action level
Determine the supporting test libraries, both UI and non-UI, to be authored
Author supporting libraries as you author automation. This is a labor-intensive process, particularly for the UI libraries as they are not only complex, but also fragile, being highly succeptible to product changes
Author test automation
Excecute recurring test runs in a test management environment upon which our automation was entirely dependent
Repair automation as product changes occur
Expand and further abstract the supporting libraries to enable model-based testing tools to generate broad suites of tests to be used as static regression tests.

We found that following this traditional, system-level testing approach our automation was consistently about a sprint behind feature implementation. This gap led to many unintended consequences:

For about a sprint, there was little to no consistent QA automation backing up some features, meaning that serious breaks could go uncaught for far too long
QA and Dev were at different points in the feature development timeline, severely dampening critical QA/Dev person-to-person conversations. Simply, Devs were focusing on implementing new features while QA was still looking at the 'old stuff' in an effort to get automation in place
Changes made to the product meant that QA would, at times, spend time working on supporting libraries that were now out of date
QA constantly felt behind, putting a damper on the amount of time that many QA feature owners devoted to critical hands-on testing. Getting the automation in place and keeping it running was soaking up too much time.

As we moved on to V2 and looked back at the lessons learned, it was clear that we needed to make a number of bold steps:

QA and Dev owners of the same feature must work in a far more collaborative fashion and develop strong interpersonal relationships and a sense of joint ownership of their feature. This was accomplished by some owners in V1, but we needed it to be more broadly true in the organization.
QA cannot afford to focus all, or even the majority, of its testing efforts at a system level as can be done in a waterfall environment. Instead, much of QA's efforts must be at the subsystem level.
When a new feature comes online or is significantly altered, QA needs to get some high-pri tests automated immediately before moving on to broader testing.
The overhead of test automation must be made far lighter to free up time for additional QA hands-on testing early in a feature's life and QA management must ensure that such testing is a highly valued priority. To lighten the overhead, use of UI-driven automation is avoided, instead leveraging API testing and DTE testing.
To further strengthen Dev/QA synergy, all Dev unit tests and QA tests are run together on a nightly basis.

The P0 test:

The initial focus of test automation is what we call the 'P0' test. This is the quick-turnaround automation that ensures that QA and Dev are in synch, gets regression automation in place quickly, and seeks to push QA/Dev to stay in synch through the cycle. Here are the basic requirements for P0 tests:

They should be automated and in place within 1 day after a feature or feature change appears in the build. In order to make this happen, P0s must be few in number (this is by design) and Dev must give QA a heads-up on the changes being made before they are checked in.
P0 tests must be associated with the dev workitem(s) that they cover. This is recorded in the tests' metadata
P0 tests must have been agreed to by both Dev and QA before they are implemented. Both should agree that failure of the test represents a critical failure. If not, the scenario envisioned is likely a P1 or P2 test (I'll cover those in another posting)
P0 tests must not duplicate Dev unit tests but, instead, be at a bit more abstract level, what we call 'API integration testing'. What does this mean? Well, if you have methods A, B, and C, Dev should have unit tests for each; however, QA P0 tests will cover how A, B, and C interact. B may appear to work correctly and C may appear to work correctly, but if C is dependent upon upstream action from B and that integration point is faulty, QA should catch it.
All P0 tests are run on a nightly basis. Many P0 tests are also labeled as being 'check-in' tests and must be run by Dev before any code check-in takes place.

This is a 100k foot view of something that we've spent a lot of time on and, as I read the post, can see that it may be confusing in many ways on its own. As I follow-up with more info on other testing that we are doing, though, I hope that the place of P0 testing is put into proper context

Rededication and retargeting of this blog to a pure QA focus

2007-09-20T02:33:00Z

The original incarnation of my blog went fallow for a simple reason. At the same time that I launched it, numerous other blogs focusing on Visual Studio Team System for Database Professionals were created by management, PM, marketing, etc. Most topics that I would have covered were being covered repeatedly elsewhere. This fact gave me the excuse to leave the blog behind and focus on driving testing of our product as our V1 RTM data rushed toward us.

Why resurrect the blog now? Well, we learned a lot of lessons about doing QA in a Scrum development environment in V1. While we achieved good product coverage, as measured by code coverage numbers for our test automation, in our V1 cycle, it was not always pleasant. The problem was that we were using 'old' approaches to QA in a new, highly iterative development environment. Here were some lessons we learned:

Scrum is well-suited to 'greasing the wheels' of dev, but it is a methodology that is not naturally QA-focused. Yes, I know that the methodology stresses quality upstream. I've read all of the books and been through a full product cycle under the methodology. Does this mean that you cannot deliver a high-quality product under Scrum? Of course not. What it does mean, though, is that QA must interact with Dev in new ways and re-prioritize how it executes. The Scrum methodology suggests this, of course, but now how it is to be accomplished in the real world. We've made a lot of progress here.
QA must be much more agressive in its communication than under non-Agile methodologies. Not snarky, mind you, just more productively agressive :)
Teams used to puting a primary emphasis on system-level testing must shift their priorities in Scrum or other Agile environments. You will still write system-level tests, but your automation efforts will need to 'grow' to these tests rather than starting there.
Inefficient internal tooling that is merely annoying in a non-Agile environment can present a major productivity hurdle under Scrum
In highly iterative development there is a greater need for metadata associating specific test automation with specific Dev work items so that QA can quickly identify specific scenarios that are and are not covered and quickly create automation runs that are targeted to specific changes.
If you think that communication between QA feature owners and their Dev counterparts is 'OK', it's not good enough.
If you want your QA team to engage in customer-scenario-centric, extended, 'deep dive' testing, you must plan it out sprints in advance, defend it vociferously to Dev and to management, and be ready to show off the results that justify the efforts.
A lot of teams in Microsoft are placing bets on model-based testing (MBT). We learned a lot about how MBT helps and where it's full of hype. Many aspects of MBT do not fit the Scrum environment in my view; nevertheless, there are some key uses of MBT tooling and methodologies that can pay off for QA in an iterative development environment

As a result of lessons learned in our V1 product cycle we are radically changing how we do our work in QA for our V2 cycle. We were successful in V1, but we are confident that we now know how to significantly improve our effectiveness in V2. The approaches I'll discuss in this blog are already in place and already showing productivity gains.

Most of what I discuss in this blog will be specific to the work our team is doing, not an academic discussion of what testing should be like. Further, I'll be providing details on customer scenarios that we're testing, database schema constructs we're testing, input data we're using for our tests, and verification approaches we're using. It is my sincere hope that customers of our product, real or potential, will see what we're doing and provide feedback that validates where we're doing the right thing and correcting us where we're making incorrect assumptions about how customers build their databases. Such feedback will help us to get more and more into the customers' shoes.

We really do work very hard in the Developer Division to provide quality tools to our customers. I've spent a decade of my life, so far, trying to maximize customer satisfaction and I take that responsibility very seriously, as do the excellent members of my team.

Report of activities at Tech Ed 2006

2006-07-21T00:59:00Z

Well, after Tech Ed, catching up on e-mail, a couple of weeks of vacation, then catching up on e-mail again (gotta love the 'conveniences' of the 21st century), I'm finally getting back to the blog.

Rather than repeating the analytical exercise that resulted in my 12-page Tech Ed report to our team, I figured I'd just toss into the blog some of the sections of that report (understandably omitting sensitive, internal information).

Oh, one quick note before the report. I gave a very impromptu podcast interview to Tech Net Radio while I was on booth duty (if you listen, it will quickly become apparent just how impromptu it was :)). I'll toss in the links to the WMA and the MP3 versions. My interview is the second one done:

WMA: http://download.microsoft.com/download/B/1/5/B15D95AC-F710-43A3-962D-CB50F71483B5/TechNetRadio-060616.wma

MP3: http://download.microsoft.com/download/B/1/5/B15D95AC-F710-43A3-962D-CB50F71483B5/TechNetRadio-060616_Hi.mp3

Selections from my report on Tech Ed 2006

Summary

Every customer I spoke to, DBAs, DB devs, and managers, was excited about the direction that we are taking VSTS. They are split on whether or not we have enough functionality in V1 for them to immediately commit to our product (which is standard for a V1). Those who are not ready to commit to V1 gave specifics about the features that they need to have for them to jump in.

There were few surprises in the feedback and feature requests that we received and the feedback was remarkably narrow and consistent, in my opinion. This speaks well of our understanding of our customers and of our opportunity to increase ‘customer delight’ about our product in the next version with specific, targeted additions.

We, like most other V1 teams, have overly simplified the profile of our target customers. In fact, we have different groups of customers who are passionate about different directions that we are going, and the passions of one group sometimes directly conflict with those of another group.

The Booth and Other Customer Feedback

We spoke to hundreds of customers at the booth during the week, engaging in conversations that lasted anywhere from 30 seconds to 30 minutes or longer. As often as we could, we jotted down notes from our conversations (Robert and Jon will send out contents from their notes, too, I’m sure). Additionally, we created a feedback form for those who did our hands-on labs to gather their impressions, positive and negative, about our product. In this section I’ll go through the highlights of what we learned as well as quotes (usually paraphrased) from customers I met.

What customers like best about our product:: There is not a single ‘dud’ feature in the product. Not every customer wants to use every feature of the product, of course, but all features received a good amount of love:

SCC/versioning as a first-class citizen in a DB tool is a huge hit. Many already use SCC, but not in the central fashion that we do; they want to use our vision of DB versioning

“What problems will this solve? Trying to figure out who changed what and when”
“This solves the problem of managing changes between my dev and production databases”
“This ensures a set location for the ‘latest and greatest’ DB version and a structured method of moving DB changes from the developer to the master copy of the DB”
“Now we can instigate a code review process for our DB changes”

Developing in a sandbox, not against a live DB resonated well with everyone. They quickly see how this improves their development process.
Involving DBA in the development cycle

DBAs who talked to us were nervous about the idea of using VS until we showed them that the barrier to entry was low and that they would be working with TSQL, not .NET code.
DBAs, and many devs, are excited that having DBA in the development cycle will increase their projects’ chances for success at deployment time

Many described the wall that currently exists between DBA and Dev, with Devs often seeing DBAs as some kind of killjoy troll while DBAs saw Devs as wanting to keep them out of the loop on work being done that targeted their databases.

“As a DBA, I feel, sometimes, that devs want to keep DBAs ‘in the basement’. They want to make changes on their own when they can”. He doesn’t like current VS DB tools because they’re not integrated with the rest of the development process (he says that they’re ‘shoved in a corner’ of VS). Needless to say, he very much likes the prospect of DBA being made part of the solution process.
Heck, if our sales are as strong as we hope, we might get a Nobel Prize to go along with our Ship It! awards ;)

“This product will give us an increase of shared knowledge between DBA and developers”

Data Generation

One of the women completing DAT006 in the HOL gushed about DGen for a full 20 minutes

She’s thrilled that DGen will allow her to populate test tables with 1,000,000 rows for testing (Deep Dive Testing Target)
She currently uses 4 tools to do what our tool does in DAT006
They currently have a team dedicated to creating and populating their test DBs.

There is overwhelming feedback that we need to have Select/Deselect All checkbox for DGen.

Unit Testing

Several of those we talked to didn’t know what to make of this feature right off the bat because they weren’t doing anything like it. Many more, though, immediately saw it as a big improvement over their current testing methodologies.
For those already using unit testing of code, it was important that our DB unit testing story integrate with their code unit testing processes in VSTS.

Schema Compare

“One of the main reasons I don’t currently have a test database is because I’d have to remember the differences between my test DB and my deployment DB”

Data Compare

A developer from a small shop works in an environment in which each dev keeps an identical copy of a test DB. He likes our lifecycle story for maintaining a ‘true’ copy of the schema for deployment; however, he does not want to use DGen, as they have specific customer data that they use for all testing. I showed him how he could leverage Data Compare for his task. He was thrilled and felt that the product met all of their needs. This was an interesting conversation, to me, because it pointed out how heterogeneous our customers’ work environments are. This was not what we would call an ‘enterprise shop’, but we still benefit as a product by meeting their needs and those of the other small shops.

Refactoring

“This tool makes a really tough job for me much easier”

T-SQL Editor

“I love the validation of scripts on save” (Adolfo, trainer, consultant, and author)

VSTS/TFS integration

“Now we can have communication and tracking of bugs around our DB”

”I came to Tech Ed to check out this product for our group. Before I did the hand-on lab I expected to hate it. Your product changed my mind, though!” She’s very excited about it. She didn’t expect MS to give her anything new or innovative (ouch!). She found SQL 2k5 to have some added features, but that it didn’t change much for her. SSMS is a bit easier to use, but not much better.

What customers feel is missing from our product: Few surprises here, although it was disappointing to hear how a number of customers considered some of our missing features in V1 to be adoption blockers. On the bright side, though, each of those, without exception, said that they love the direction that we’re going and will be watching what we do in VNext.

Here are key missing features that we heard about:

Support for non-SQL Server databases
Visual database designers/modeling. This is an adoption blocker for a number of customers.

“A dream would be to have an ERD, as well”
“Graphical Query/Table editors”
“I will not write TSQL in a text editor. I’ll keep trying to use the visual DB tools in VS”. Interestingly, though, a number of DBAs believe that working in a visual designer makes one ‘not really a DBA’. Go figure.

Deeper refactoring support. Rename is exciting and they want more.

In particular, there is a strong expectation by many we spoke to that in the future we have refactoring cascade changes into middle-tier code that calls the refactored objects.

Intellisense

People have come to expect this in VS and are very disappointed to not yet see it in our product. This is not an adoption blocker, but it’s a hole we want to fill as soon as we can.

Prescriptive guidance/best practices

“We want integration with existing standards and perhaps an automated rules engine”

Performance tools

SQL2k5 SSMS provides performance tools. One of our most enthusiastic HOL visitors wants to see us have integration with that.

Team support for encryption

Store encrypted files in SCC and have anyone on his team work on them.

Custom TSQL templates

Several customers want to customize the TSQL templates that we provide.

Prescriptive guidance around security and architecture. People want help in making some tough decisions.
After a long conversation with one gentleman at the booth, I asked him what might be missing from our product that would help him. He responded that he had been trying to come up with something during our discussion, but couldn’t. “Everything you’ve shown me is stuff that I do and need”.

In what ways can we make VSTEDP easier to use?:

“Not many. Well done on the VS2005 framework integration”
“Some kind of Intellisense”
“Clicking on an error did not take me to the error line, even though the line number is displayed. I want error navigation”
“Give us integration between the Server Explorer and Import Schema”
“I want to use the Query Builder to make my query for the Data Bound Generator. Oh, and if I have only one result field from my query, please autobind that to the column to be populated”
Gary, who we met at the booth, heavily relies on stored procs. In VS he wants to be able to step through his .NET code and right into the stored proc in our T-SQL editor.

How customers currently manage DB change:

Some customers don’t really have a defined process

“Good question. We only have CA Erwin”
“We individually apply changes to the database”

Others have home-grown processes, no two of which seem to be same.

“We run manual tests against a testing instance” (yeesh!)
“We manually track changes to the dev database and rolling changes into a master upgrade script”
“Individuals creating the change test it, then it is tested as part of front-end application testing”
“We use Erwin and have DBA reviews of changes”
“We diff schema scripts manually and in VSS”
“We have test and staging servers and use SQL Delta to compare them after changes”
“It is a very manual process. Unit tests change and we run test harness code against stored procs for regress testing”
“We get sql scripts, periodically, from the DB and put them into SCC”. So, their SCC version is reactive to the deployed DB, not the other way around.

The nature of the databases that our customers work with: Desiring to learn more about how we should be testing our product, we asked many customers to describe the databases on which they work. Not surprisingly, we received a broad range of answers:

When we asked customers about the number of tables in their databases, we received answers in the range of ~15-200+, skewed toward the lower end of this range.
Asked about the number of rows they have in tables, answers ranged from tens of thousands to tens of millions, with many reporting 1,000,000+. This highlights the need that we have to do extensive performance testing and tuning late in the cycle.
Many customers are tasked with maintaining and testing legacy databases with poor architecture. I spoke to multiple customers who work with databases that simply are not normalized, but which cannot be changed architecturally. These customers want to be able to use DGen (Deep Dive Testing Target) successfully against these databases.

“Our previous developer created Access DBs, then used a wizard to migrate to SQL Server. The design was terrible, but it produces 50 reports per day and cannot be taken offline to rearchitect. Your tool will let me work in a sandbox to make and test changes to the architecture and version while I fix this bad DB”
“Our legacy DB has no PK-FK relationships. Despite this, there are inter-table and inter-column dependencies in the DB, all handled through business logic. I want to be able to declare these relationships in DGen and have population work as if a PK-FK relationship existed. Updating the schema is not an option for us”

Lookup tables are quite common and it’s very important to those customers that our product allow them to RevEng, check into SCC, and otherwise manage such tables.

Performance expectations for DGen: Since perf has been such a concern for DGen, we took the opportunity to ask many attendees how many rows of data they might want to populate in a test server and what their performance expectations are:

One corporate customer wants to populate 1,000,000 rows in a ‘short time’ (a few hours or less) or they won’t use it.
Another company wants to populate 50,000 rows with test data. He expects this to complete in less than one minute, or he’d begin to wonder if something is wrong.
Yet another (Blog note: I’m eliminating company names for the blog posting) tables with 120,000,000 rows. Our current pre-beta doesn’t work for them because it takes too long. Statistical analysis tells them that they need to populate ~1,000,000 rows with test data. He wants that population to complete within a day
A company in The Netherlands expect to populate 100,000 rows with test data within one hour.
A final customer, involved in our TAP program (Andrew and I spoke to them after Tech Ed) wants to populate 1,000,000 rows in a highly normalize DB at a rate of 250,000/hour

For Visual Studio Team Edition for Database Professionals, 3 days at Tech Ed down, 2 more to go

2006-06-15T07:02:00Z

So far, I’ve spent three extremely busy days here at Tech Ed Boston, talking to scores of attendees at our booth, meeting with DBAs, trainers, and dev team leads for extended discussions about the benefits of our product (as well as its current gaps and our future plans to fill them). I’ve had the opportunity to do a brief podcast segment (I’ll post a link as soon as it’s up at sigs.sqlpass.org) about our product. Finally, I’ve taken advantage of every opportunity possible to meet with folks who are completing our hands-on labs to get their take on how our V1 looks, what problems it might solve for them, and, most importantly, where we’re still missing the mark for their needs.

I absolutely love this kind of constant customer engagement. It’s always amazing how many assumptions that we take for granted as being true are shattered when you get face to face with the folks who pay your check (sorry, Bill, but it’s the customers who ultimately do that). Customers always bring up scenarios that, despite our best efforts, we’ve never thought of and would never think of from our point of view. Learning about these different needs and points of view through face-to-face interaction is irreplaceable, especially for someone, like me, in QA. It is impossible to do a thorough job of integration testing without a deep understanding of the problems that customers will try to solve with your product and how they currently go about solving those problems. With the information that I and my teammates are gathering through these customer engagements, we’ll do a much better job of getting this product ready to ship before the end of the year.

So, what’s in it for the customers? Aside from the quasi-selfish view that they’ll get a higher-quality version of our product, customers come to us with many, many questions about how to use current products, best practices and how they can leverage extensibility of our products to improve their own sales. My favorite so far is the gentleman who wrote a tool that uses Team Foundation Server an Outlook to automatically schedule a meeting when certain bug criteria are met. Great. Just what the world needs, an automatic meeting-creation factory. Maybe next he can work on a tool to automatically cut employee benefits and raise executive pay when a company’s stock drops to a certain level. But, I digress.

Finally, today we had a 3 hour focus group with a dozen prospective customers. This was another invaluable experience. The attendees were generous with their time and did not hold back. They let us know exactly where they felt that we fall short in V1, though they all love the direction that we are going and a number of them are going to adopt V1 even if we add nothing else to it. Despite the constructive criticism, I see this as all good. We want this product to grow over a number of cycles and even if we don’t get everyone with V1, we’ll get their eyes on us, watching for V2.

So, what, exactly, were the main customer requests? I’ll address that with my next posting. As for now, it’s almost midnight, I have another long day ahead, so I’m going to bed.