Different type of automation
Clearly, there is no standard rules on how you should categorize your automation, it all depends on your project size, schedule and product complexity.
ASAIK, in general, testers usually break down their automation based on their needed and the resources available.
Here is a brief ideas on different type of tests (It vary depends on the complexity of the product)
|
|
When to run? |
How long to run? |
What covered? |
Purpose |
Run Contexts |
|
Build Verification Tests (BVT) |
After each build |
50% build time |
Pri 0 issues |
Sanity check for build quality |
Default |
|
Unit Tests |
Before check in |
5 -30 minutes |
API level functionalities |
Prevent new check-in causing regression |
Default |
|
Buddy Tests |
Before check in |
4-8 hours |
Areas around the check in |
Ensure the quality of the check-in |
Manual
(Random) |
|
Nightly Tests |
Every night |
< 24 hours |
Pri 0 - 1 issues |
Keep product quality in known state |
Rotate |
|
Weekly Tests |
Every week |
< 3 days |
Pri 0 - 3 issues |
Keep product quality in known state |
Rotate |
|
Full Suite Tests |
Each Sign off/
milestone |
2-4 weeks |
Pri 0 - 3 issues
on all contexts |
Sign off/Verification |
All |
|
Code Coverage Tests |
Each major milestone |
2-4 weeks |
Found out what had not been covered |
Identify missing coverage and fill it |
All |
|
User Scenario |
Feature complete |
1-2 hours each |
User scenario of individual feature |
Ensure feature complete meet customer expectation |
Manual
(random) |
|
Demo Scenario |
Each major milestones |
4-8 hours each |
User scenario across multiple features |
Prevent cross feature integration issues |
Manual
(random) |
|
End to End Scenario |
When product is dogfood-able |
1-2 weeks |
User scenario from End to End |
Ensure best End to End user experience |
Manual
(random) |
