Yesterday I wrote about the need to reduce the number of things a project attempted to do in order to deliver a great product.  Too many seemingly good ideas can make a product late or fragmented or both.  The same is true of testing a product.  Great testing is more about deciding what...

It seems obvious that test pass rates are important.  The higher the pass rate, the better quality the product.  The lower the pass rate, the more known issues there are and the worse the quality of the product.  It then follows that teams should drive their pass rates to be high. ...

This came out of a conversation I had today with a few other test leads.  the question was, “What are the top 5 books you should read if you want my job?”  My job in this case being that of a test development lead.  At Microsoft that means I lead a team (or teams) of people whose job it...

My team recently finished what we call a “ bug bash .”  That is, a period of time where we tell all of the test developers to put down their compilers and simply play with the product.  Usually a bug bash lasts a few days.  This particular one was 2 days long.  We often make a competition...

James Whittaker is the author of books like How To Break Software.  He ran one of the few university-level testing programs at Florida Tech.  He's now as Microsoft and helping Visual Studio become better at testing.  The guys at .Net Rocks caught up with him for an interview .  James...

Toyota was able to eclipse the makers of American cars in part due to its production and development systems.  The system has been popularized under the rubric of "Lean" techniques.  Among the tenets of the Lean advocates is asking the "Five Why's."  These are not the...

A friend turned me on to this post by James Whittaker.  I didn't know he had a blog so now I'm excited to read it.  He has a lot of really interesting things to say on testing so I encourage you to read his blog (now linked on the left) if you are intrigued by testing. Microsoft prides itself...

I just read a very interesting research paper entitled, "The Impact of Test Suite Granularity on the Cost-Effectiveness of Regression Testing" by Gregg Rothermel et al.  In it the authors examine the impact of test suite granularity on several metrics.  The two most interesting are...

All good development projects follow certain basic practices to ensure code quality.  They use source control, get code reviewed, build daily, etc.  Unfortunately, sometimes even when the shipping product follows these practices, the test team doesn't.  This is true even here at Microsoft...

We recently went through a round of test spec reviews on my team. Having read a good number of test specs in a short period of time, I came to a realization. It is imperative to know the failure condition in order to write a good test case. This is at least as important if not more important than understanding...

Testing started simply. Developers would run their code after they wrote it to make sure it worked. When teams became larger and code more complex, it became apparent that developers could spend more time coding if they left much of the testing to someone else. People could specialize on developing or...

Someone recently related to me his experience using the new Microsoft Robotics Studio . He loaded it up and proceeded through one of the tutorials. To make sure he understood, he typed everything in instead of cutting and pasting the sample code. After doing so, he compiled and ran the results. It worked...

BVTs or Build Verification Tests are standard Microsoft parlance for the tests we run every day to ensure that we didn't break anything important with our checkins the day before. I've previously written about the importance of keeping them clean . Within the range of tests that consistently pass, which...

There's a balancing act in testing between automation and manual testing. Over my time at Microsoft I've seen the pendulum swing back and forth between extensive manual testing and almost complete automation. As I've written before, the best answer lies somewhere in the middle. The question then becomes...

I talk about it a lot, but I don't know that I've ever defined it. A reader recently wrote in and asked what exactly this was. I suppose that means I should give a better explanation of it. Long ago in a galaxy far, far away, testers were computer-savvy non-programmers. Their job was to use the product...

It is becoming accepted in the industry that teams should produce a build on a daily basis. Every project at Microsoft does this as do most projects elsewhere. If you happen to be on a project that does not, I suggest you work to get one implemented soon. The benefits are great. After a daily build is...

MSDN now has a home for test information. Check out the new MSDN Tester Center . It has articles, videos, and a collection of blog posts all revolving around the idea of testing. If you are a tester or test developer, bookmark this site. It looks like it will be useful.

This post over on TestingReflections brings up an interesting point. Michael answers the question, "What are the useful metrics for software quality" with another question. He asks, in a roundabout fashion, what is it that we value about the software? He rightly points out that some of the things we...

In my post, Test Developers Shouldn't Execute Tests , Antony Marcano asked if we actually need test developers or if developers would do. If the more traditional testing tasks are being done by one group and the automation by another, does it even make sense to have a test development role any more?...

This view puts me outside the mainstream of the testing community but I feel strongly that test teams need to be divided into those that write the tests (test developers) and those that execute them (testers). Let me be clear up front. I don't mean that test developers should *never* execute their tests...

A promising young executive at IBM was involved in a risky venture that lost $10 million for the company. When Tom Watson Sr., the founder and CEO of IBM, called the executive to his office, the executive tendered his resignation. Watson is reported to have said, "You can’t be serious. We’ve just spent...

An interesting report over at Gamasutra. Jami Johns from Microsoft Game Studio gave a talk at Gamefest 2007 about how they tested Crackdown. The thing that strikes me most from the article is the increasing need for automated testing and thus test developers who are true programmers. A few excerpts...

A year and a half ago I talked about how I was running scrum meetings with my team. Since then, we've refined the process but have consistently held scrums on a regular basis. Note that I'm not running a full Scrum system with sprints and product backlogs and such but rather just adopting the scrum meetings...

Two recent Dilbert comic strips are a picture of how things sometimes work in our business. In the first , Dilbert complains that funding for test automation has been eliminated. The PHB responds that Dilbert should write some himself. Because Dilbert is already paid, the work is "free." In the second...

Most software has a user interface. That means that most test teams spend time testing that interface. The easiest way to do this is to just click on all the buttons and make sure the right thing happens. This works, but it doesn't scale. Eventually you look at the hordes of testers clicking buttons...