Test Guide

It seems to me that lots of people are experiencing lots of confusion regarding what lots of the testing terms we throw around signify. In an effort to remedy this circumstance I have applied my investigatory powers to observe what people really mean...

I think my team - much of Microsoft, in fact - is going about testing all wrong. My team has a mandate from on high to automate one hundred percent of our testing. We aren't the only ones either, I don't think. This is a great goal (even if it isn...

Recently I read through the notes from Brett Pettichord's workshop on Homebrew Test Automation . It's a great overview of available test automation techniques, although it is old enough that he doesn't mention UIAutomation (the Longhorn replacement for...

Verifying that a test case’s actions had the expected result is perhaps the most important part of testing. Every test case does something at least a little differently than every other test case, so the expected results are often a little different....

I found this bug while working as one of the first QA engineers at the MathWorks (makers of MATLAB and Simulink technical computing software) in the early 1990s. ANSI standard 754 mandates rounding and such behavior for real finite-precision floating...

Last week I took Rapid Software Testing from Michael Bolton . The three days of stuffing my brain in the beautiful downtown campus of the University of Toronto was loads of fun. Points I especially remember: I had already experienced one of the exercises...

Mark asks: So how *do* you find tester-developers? Certainly a job description that says "QA" seems to be an invitation for lots of resumes from script-followers. I want people who think repetetive tasks are what computers are for, not people, and that...

The execution section of a test case is itself composed of three subsections: setup, execution, and teardown. Execution is often thought to be the most important part of a test case (after all, a test case that doesn’t verify anything is rather pointless...

Signs such as this one are a common sight in Seattle's residential neighborhoods. I imagine the transportation department believes it to be clear and unambiguous. I beg to differ. Your assignment for this week is to devise as many different interpretations...

With the LFM defined and hooked up to Execution Behaviors it makes sense to move on to defining the Physical Object Model . The POM is often the simplest part because it's just a straightforward API around the user interface. Given Surveyor's UI: ...

Yesterday and today have been jam-packed with interesting sessions. The best of the bunch by far, though, was the roundtable discussion on agility and architecture Bob Martin led this afternoon. I'm still trying to decide whether SD was incredibly...

The sum of these factors makes testers little more than factories churning out test case after test case, each of which is different only in some small detail (e.g., how a particular operation is invoked) than the others. Another way to say this is that...

"Testable" is one of those words with a bajillion meanings. Dave Catlett, a Test architect here at Microsoft, defines testability as "[t]he degree to which components and systems are designed and implemented to make it easier for tests to achieve complete...

Futura writes that s/he has been a developer and then a tester for many years. In that time, s/he has used many languages, including C++,VB, Python, and MatLab. Futura is planning to apply for an SDE/T position and is looking for advice on answering the...

Details regarding how data is pulled out of our application’s internal data structures and other parts of the system environment are factored into an Application Internals model. This isolates knowledge about where data is found and how it is accessed...

Factoring these details out of the test case into intuitively organized libraries allows the test case to focus on the essence of what it is testing rather than incidental details. This helps us write more comprehensive tests faster but spend less time...

Once you create your Logical Functional Model the other pieces can follow in any order immediately as chunks of your feature and LFM are defined. I'll follow the most typical pattern here, which brings us to Execution Behaviors. When you first define...

My previous post hinted that the Physical Object Model takes advantage of some underlying magic that makes different types of controls look similar. This "smudging" of controls into similarity comes courtesy of the controls abstraction layer. While...

Of all the bugs I've came across I'll never forget this one because it was so severe and easy to miss. Later, when I was a Test Lead it was the first thing I added to my sign-off checklist… A few years ago, our product was practically out the door when...

These problems all combine to make test cases a maintenance nightmare. Anytime the application under test changes – regardless of whether the UI changed a tiny bit or an entire feature was dramatically revamped – every affected test case must be updated...

Several people have emailed me recently asking for tips on getting through their upcoming interviews at Microsoft. My advice: Be Yourself . I believe this is the most important tip I can give you. I find that how someone fits (or does not fit) with...

Diligent Reader Ayaz asks: Everywhere there is talk about *the tester mentality* and how the testers should refine their approach towards a problem. My question is what would you advise a *developer* so that he can test his code and catch the bugs...

When I was a fresh-faced programmer I was the fastest codeslinger around. I could spew code out like nobody's business. I didn't need to bother with design or spec'ing. You want a program to do what? Bam! Here it is. Something not quite right? Bam! Here...

Many tests don’t care exactly what data they are executed against. Our Data Manager uses techniques similar to those used with Execution Behaviors to manage Data Providers that generate test data. This solves with a single stroke the dual problems of...

What is quality to you? How do you measure quality? When is quality accomplished? If you don't know how your customers would answer these questions, your product probably doesn't meet their needs as well as it could. You can fix this problem. But first...