Our Software Test Life Cycle

Syed Aslam Basha here from the Information Security Tools team.

When we used to start working on a software project, we would start by preparing a test strategy and identify set of test activities. This involved spending days, grinding out documents largely for the sake of it.

Recently, I have tried to enumerate and redefine the test activities to save time in planning and improve test estimation. I thought I would share this approach in this post, hoping it acts as a easy to use template for test community to quickly get started with actual test activities.

This article should apply to any kind of execution methodology you may follow, be it water fall or agile.

• First and foremost is the estimation of the test effort with respect to work that is identified in the sprint backlog (planned list of all features to be delivered by the end of that sprint).
• Preparing test plan/ test strategy of how we would be executing the sprint.
  • Features to test
  • Environments to be tested
  • And many more details mentioned in detail in the below table
• Test activities start as early as possible by reviewing the documents such as any business requirement document (BRD) / functional specifications (FS)/ technical specifications or from knowledge transfer (KT) sessions.
• Authoring functional test cases/ test scenarios
  • Authoring test cases covering all functionalities / features planned for the release
  • Authoring test cases for - installation and its supporting documentation (help files, privacy, end user license agreement (EULA) and read me etc)
  • Authoring system test cases
• Set up the test environment we have defined such as;
  • 32 bit - win7 operating system (OS)
  • 64 bit - win7 OS
  • 32 bit - windows 2008 OS
  • Web server set up
  • Database (DB) set up
  • And also test users creation and service account creation
• Review of functional test cases, it could be internal to test team or from DEV/PM team
• Rework on the feedback on the review of functional test cases
• Review of system test cases with the team
• Rework on the feedback on the review of system test cases

The activities listed above could be executed before your actual testing begins. The below listed activities would be taken up once we receive a build.

• Installing build
  • Could be stand alone or website, web service along with DB
• Build verification test (BVT)
  • Perform build verification test to take appropriate action as to continue with testing with the build or to raise show stopper and request for hotfix/full build
• Execution of functional test cases
  • Execute the test cases and log bugs if any against the build
• Bug fixes and regression
  • Verify the bug fixes in the release and take appropriate action and regress the functionality and impacted areas
• System testing (ST) – Install ST build
• ST BVT
  • System testing build verification test
• ST Regression testing - Bug fixes, if BVT is pass verify the bugs fixed in the release
• Execute system testing test cases 
  • Execute the authored system test cases
• Adhoc testing
• ST Bug fixes - If we get another drop in ST

If there aren’t any bugs in combination of sev1, sev2, pri1 and pri2 identified during ST or open bugs the test team signs off the build with release mail being sent to user acceptance test (UAT)/ PM team.

In a nutshell the following table shows the test activities from beginning until test team signs off along with estimations.

Estimations:

Now coming to the interesting part i.e. giving estimations( I would be concentrating on authoring, automating and execution of test cases which forms the core of testing), team needs to go through the documents (BRD/FS/Tech spec) available before going for estimations and quantify the amount of work that is expected in the sprint. For example we would identify the number of test cases to be authored and its complexity say 30 test cases are expected then if the complexity is (low, medium and high) low then 5 test cases to be authored per hour, likewise for medium and high. Similarly we can get estimations for the manual execution of the test cases. The same applies for automating test cases and while execution would differ. Decide on per day task hours per person you would be able to come to end date.

If any activity repeats, you can add it again and update the hours.

Bug life cycle:

The following is the bug life cycle going through different phases:

During testing of the application tester finds a bug and logs the bugs in detail say in visual studio team foundation (VSTF), it will be in proposed state and assigned to Triage team

During bug triage meeting, the team goes through the bugs one by one and if it’s a valid bug its state is changed to accepted and assigned to a “developer” if its invalid/by design then state will be changed to closed with resolution set to appropriate as by design/ not repro etc

The DEV’s work on the accepted bugs and fix the them. After fixing them change the state to resolved, assign to a tester and set resolution to fixed, along with fix details in fix column. If the developer is not able to repro the bug, state will be set to closed and resolution as not reproducible.

The testers picks up the resolved bugs in released build and verifies them. If the feature is working as expected then it will be closed else state will be made active and reassigned to developer.

PS: The numbers given are hypothetical.

-Syed Aslam Basha ( syedab@microsoft.com )

Microsoft Information Security Tools (IST) Team

Test Lead