It seems to me that lots of people are experiencing lots of confusion regarding what lots of the testing terms we throw around signify. In an effort to remedy this circumstance I have applied my investigatory powers to observe what people really mean when they say these words. Forthwith, the answers which I have compiled:

Categories of Tests

  • Build Verification Tests: "Did they even bother to test this?" The oft-used synonym "smoke test" comes from the similar question "What were they smoking when they checked this in?"
  • Exit Scenario Tests: A test of the executives' exit strategies.
  • Basic Functionality Tests: The set of all tests which are executed.
  • Comprehensive Tests: The set of tests which will never ever be executed.

Build Ratings

  • Self Toast: The build had a nasty bug and now the lab machines are all being rebuilt.
  • Self Test: The build had a nasty bug and now the product team's machines are all being rebuilt.
  • Self Host: The build had a nasty bug and now the executives' machines are all being rebuilt.

Types of Tests/Testing

  • Positive Test: A test which is so confident it will never fail that it does not bother to verify and/or log anything.
  • Negative Test: A test which has never ever passed. And no one cares.
  • Black Box Testing: Blindly flailing about hoping against hope to find a bug.
  • White Box Testing: Coding.
  • User Scenario Testing: Pretending we understand how our customers are going to use our product.
  • Acceptance Testing: Pretending we know how to tell whether a feature is done.
  • Regression Testing: Verifying that the developers didn't really fix those bugs.
  • Performance Testing: Annual exercise to determine how big of a bonus each executive receives.
  • Load Testing: Determining how much overtime the team can take before they fall ill.
  • Exploratory Testing: "Hey tester, go do whatever you want and tell me when you're done."
  • Scripted Testing: "Hey tester, exactly execute these test steps I wrote ages ago, back before anybody knew anything about how anything would actually work."

Bug Resolutions

  • By Design: That "bug" is really a feature.
  • Duplicate: A quick way to raise your score in bug bashes.
  • External: Not our problem.
  • Fixed: It works on my machine!
  • No Repro: I don't want to deal with this.
  • Postponed: Let's all pretend we will fix this some day.
  • Won't Fix: As if! Why did you even bother logging this?

(Yes, my tongue is firmly in my cheek! To learn what I really think these terms mean, see my DDJ posts Test Taxonomy Decoder Ring and More Decoder Rings.)


*** Do these definitions seem all too familiar? I need a tester! Interested? Let's talk: Michael dot J dot Hunter at microsoft dot com. Great testing and coding skills required.