Our BVTs have grown rather encumbered and encrusted of late, so we recently embarked on a project to trim them down to a more reasonable size. This turned out to be an interesting process, mostly because it quickly became clear that we didn't have a single agreed-upon definition of what exactly these tests' purpose was.
That old saw about asking opinions of three different people and getting four different answers certainly held true in this case. It still amazes me somewhat that a group of people can work together for months or years and still have such disparate understandings of concepts fundamental to the work we do every day. (Aside: This is why feature teams are so important: it's pretty much impossible to write a spec sufficiently precise that all parties involved will have exactly the same understanding of it.) This state of affairs isn't unique to my current team, either. Every team I've been on has defined the various levels of testing differently than every other team.
Despite all this variation, however, the broad strokes are similar even though the details (most especially the names) can be wildly different. Here are the names and definitions my team is using (this week, anyway <g/>):
A failure at any point invalidates the results of tests further down the line. If BVTs fail, FVTs aren't even looked at. (On some teams they aren't even run.) If the exit criteria aren't all passing then the results of the basic functionality runs are ignored. And so on. The sum of all these results is used to rate each build; common terms at Microsoft are:
BVTs and FVTs are critical to daily work as they tell you when you hadn't better sync your enlistment to the latest source if you want to get anything done that day. The rest of the tests are one indicator of your app's quality and correctness. Taken together they help keep your project on the right track.
*** Comments, questions, feedback? Want a fun job on a great team? I need a tester! Send two coding samples and an explanation of why you chose them, and of course your resume, to me at michhu at microsoft dot com. Great coding skills required.