How long will it take?

When getting close to shipping a product and the team discovers a critical defect that must be fixed one of the first questions asked by the management team is, "how long will it take?" From a testing perspective they generally want to know 2 things; what has to be done to reduce perceived risk (or increase their confidence the fix hasn't introduce new defects), and then they ask how long will it take. Unfortunately, many teams take a swag at the time element and proceed to beat on the product until the magically agreed upon time expires.

I have seen a lot of test case templates, and one field that I often see omitted from the template (or ignored by the test designer) for a test case is duration, or a reasonable approximation of how long it should take an experienced tester to complete a particular test (whether it is a discrete, functional test or a complex user or business scenario). Adding an estimated time for completion to a test then allows us to better approximate (within an hour or so) how long it will take to execute a particular test suite or a portion of that test suite.

For automated tests written in C#, getting the time for a test to complete is now simple with the Stopwatch class introduced in the .NET Framework 3.0. The Stopwatch class is exactly what the name implies; it provides methods and properties to accurately measure the elapsed time between starting the stopwatch instance and stopping the stopwatch instance. The following code snippet provides a simple example of how the Stopwatch instance might be used in a functional or behavioral automated test.

 using System;
using System.Diagnostics;

namespace TestingMentor.Examples
{
    class StopwatchExample
    {
        static void Main(string[] args)
        {
            // initialize a new stop watch instance
            Stopwatch myTotalTestTime = new Stopwatch();
            // start the stop watch
            myTotalTestTime.Start();
            
            // test code goes - simulated by sleep
            System.Threading.Thread.Sleep(3500);

            // stop the stop watch
            myTotalTestTime.Stop();

            // log the elapsed time to the log file 
            // (simulated by writing to the console window in this example)
            Console.WriteLine("Total test time: " + 
                GetElapsedTestTime(myTotalTestTime.Elapsed));
        }

        private static string GetElapsedTestTime(TimeSpan timeSpanObject)
        {
            string time = String.Format("{0:00}:{1:00}:{2:00}:{3:00}",
                timeSpanObject.Hours, timeSpanObject.Minutes,
                timeSpanObject.Seconds, timeSpanObject.Milliseconds / 10);
            return time;
        }  
    }
}

It is also possible to create several instances, or start and stop several stopwatches in a single test to measure not only the duration of the complete test, but how long it takes to complete a particular task within an automated test. This is extremely valuable for performance testing (a test used to determine the time required to perform and complete a particular task), or stress testing to measure mean time to failure (MTTF) and mean time between failures (MTBF).

Adding a stopwatch to the tests in your automated test suite will better enable you to accurately determine the duration or amount of time required not only to execute a complete test suite such as the build verification test suite or an automated regression test suite, but it also allows us to better calculate the time required to execute a particular subset of tests in a test suite (such as all priority 1 tests, or tests in a particular functional area).