Model Based Testing – An Introduction
In this article I will attempt to give a brief overview of Model Based Testing (MBT) and provide links to help you dig in for more details. I have been increasingly getting more and more fascinated (obsessed?) with this paradigm, which I believe holds great promise for the future of the software testing discipline. Hope to rub some of that passion for MBT on to you.
MBT is an approach in which you define the behavior of a system in terms of actions that change the state of the system. Such a model of the system results in a well-defined Finite State Machine (FSM) which helps us to understand and predict the system’s behavior. For e.g. when working with a model of a car, the action of “press accelerator” has a different result depending on whether the state of the car is “neutral” or “drive”.
The biggest MBT take-away from a testing perspective is that any test case that you define is really just a traversal of the model that you have of the system – either an explicitly defined one or the one in your head. Generating test cases thus boils down to traversing this state machine. MBT is therefore especially useful for automatic generation of test sequences from the defined model. Various graph theory algorithms can be employed to walk this graph (e.g. shortest path, N-states, all-states, all-transitions etc.). The best part is that when you are using a good MBT tool, you do not need to know or dig into all this graph theory all the time. Simply define the states, actions, transitions etc. for your model and click a button to start generating tons of interesting test cases.
Model based testing offers a lot of benefits, some of which I list here. I shall expound on some of these in later posts.
1. Forces detailed understanding of the system behavior
2. Early bug detection (which is much cheaper)
3. Test suite grows with the product
4. Manage the model instead of the cases (useful when features changing constantly).
5. Can generate endless tests (since test cases are machine generated)
6. Resistant to pesticide paradox
7. Find crashing and non-crashing bugs
8. Automation is cheaper and more effective
9. One implementation per model, then all cases free (test code is better enabled for changes).
10. Gain automated exploratory testing
11. Testers can address bigger test issues
Any other benefits that you think it brings to the table? I would love to hear from anyone actually using model based testing outside of Microsoft, and what their experience has been using MBT.
If I had to recommend just one and only one short introduction to Model Based Testing in order to sell the idea and get you hooked to it, I would point to this paper, Intelligent Test Automation, by Harry Robinson, which appeared a few years ago in the STQE magazine (which I believe has morphed into the Better Software magazine). Interestingly on this page, Harry has put up the 2-file state model of the example in the paper and also a Chinese Postman traversal for the model.
Here is an extensive list of technical papers related to model based testing that can help you dig into the details and the theory behind this super powerful testing paradigm.
One great publicly available tool you can use for Model Based Testing is Abstract State Machine Language(ASML) produced by the Foundations of Software Engineering(FSE) group Microsoft Research(MSR). You can actually have executable specifications with ASML, so when your Program Manager generates the functional specs, you can simply execute that spec and generate test cases directly from the spec. Buckle up - The future is here (almost)!
