Efficiency was a recurring sub-theme at the Ørdev software conference in Malmö, appearing explicitly in at least five of the presentation titles.  However, most of the speakers were quick to point out that efficiency has little meaning without a focus on effectiveness.  Scott Hanselman (http://www.hanselman.com/blog/) made the point most clearly in his keynote “Information Overload and Managing the Flow:”

  • Efficiency is doing something right
  • Effectiveness is doing the right thing

This point is very relevant to software testing efforts.  We want our software testing efforts to be as efficient as possible, but because we always face more potential tests than can be created and executed in the time that is allotted, it is important that we focus on testing activities that are the most effective.  It doesn’t do the users much good if we are very efficient at doing the wrong types of testing.

Efficiency is often an important consideration because management would like to maximize the return on invest (ROI) for the testing efforts.  The premise is that being able to complete the planned testing faster or for less effort will have a positive impact on ROI.  This makes sense, but it is still a good idea to ask the question,

“Is this the best way to test?”

as improving the effectiveness of the testing is likely to have a larger impact on ROI than performing the existing testing as efficiently as possible.    

Qualitatively, effective testing activities are those that maximize the opportunity to discover problems with the software under test.  However, the effectiveness of testing is difficult to evaluate quantitatively.  In future posts, I will consider quantitative metrics for test effectiveness and and touch a little more on test efficiency.