As software development is getting more and more agile, the need is to reduce the effort required in the system testing phase and start testing early in the cycle. How can we reduce test effort? By automating the tests. When can we automate tests, especially UI automation? We need a stable UI and an application ready for the test code development right? Ohhh, does that sound ridiculous?
How can we develop a UI automation suite, when the dev team is not even ready with the UI design, and we don’t have a stable UI to for developing the automation suite? Yes, we can achieve it if we follow a well-structured, layered and loosely coupled automation solution.
The idea here is to separate out the layer of specific actions on the UI from the scenarios and tests. The specific UI actions can only be identified once the UI is ready and stable which can be achieved only later in the cycle. But what about the tests and scenarios, these would be known to us earlier when the requirements are baselined. The chances of the scenarios and tests to change over the cycle is very less when compared to the UI.
Before we go into the details, let’s look at the anatomy of a typical test suite for UI automation.
A test automation solution will mostly consist of these components as mentioned above either well-structured or layered or mixed up in a clutter. The worst case that I have seen in my experience is that all of these layers are not well separated and there are cross references among these.
A better level of maturity that we can see is where the layers are well organized as below.
This is the most common form of test automation suites than we can see in the industry now. There will be a tests layer at the top which interacts to test data layer either by using the tool capabilities or a custom implementation. The test layer interacts with the scenarios layer which has a high level implementation of scenarios and not a set of detailed actions. The scenarios layer will use the actions layer to realize the actions needed for a scenario. This might look very familiar to all who have done a structured test automation framework. This is good, but would you be able to do the following with this approach
1) Start developing the automation suite before even the UI design is finalized
2) Remove any layer without code changes spread across other layers(loosely coupled)
3) Spend minimal effort in the System Testing phase for completing the automation suite and use the suite effectively to run the tests
In order the achieve this, we need to think of how we can make the solution loosely coupled, easily extendable and maintainable. We can achieve this by
Design the solution with interfaces to set the contract on which each layers will interact and use a factory pattern to provide the concrete implementation of the components.
So the tests layer only knows the signatures of methods or in fact the contracts it can use on the scenarios to run the test. The scenarios layer knows only about the contracts on action methods. It doesn’t know how the actions are going to be implemented. We can plug in any implementation of the scenarios or actions component to the tests. So the business logic tests can be same for different implementations. Even the test data, whether it be dynamic or static should be abstracted by test data adapter layer which provides the data required for the tests. Tests layer will never know how this data is created or fetched for it.
Let’s take a simple example, suppose our application under development is a simple calculator as a windows application. During the requirements analysis phase itself we know that the business logics that we need to test would be Add, Subtract, Multiply and Divide. To make it simple, let’s think these are the only logics that is going to be implemented. Could we identify the tests and test scenarios from this phase. Absolutely yes, let’s describe few high level tests
1) Add two numbers and verify the result
2) Multiply two numbers and verify the result
3) Subtract one number from another and verify the result
4) Divide one number by another and verify the result
Now think of some scenarios we would need in the application to realize these functionalities. I am writing these below as scenario method signatures
1) int Add( int firstNumber, int secondNumber)
2) int Subtract(int firstNumber, int secondNumber)
3) int Multiply(int firstNumber, int secondNumber)
4) int Divide(int dividend, int divisor)
Did we think about any detail about implementation of the calculator so far? No right. We don’t even know whether it is going to be a windows application or a web application. We just know that the application should be capable of realizing the calculator functionalities that we understood from requirements. Now think of the data that you would need for your multiple positive and negative tests. We still don’t need to know implementation for this.
At this stage, we can develop 90% of the test automation suite, the tests, the scenarios, and the test data adapter module. Plug in the specific implementation of the actions layer component when the UI is ready.
So far so good, but without having an actions layer in place, how can we test whether our solution is working. We may have a good idea of the UI design as we progress through the implementation but we may not have a working UI. The answer is use a mock actions layer which does something like writing to a log or console. For instance, when the scenario calls for a click of button, the mock action will just write a string to log, “ClickSubmitButton method called. Button click simulated by mock”. You may also return back mock values to the scenarios for the test to succeed. In this way, the entire solution can be tested with mocks on how the test works and the flow of actions. Think of a case when the implementation is totally changing where a web application or modern UI application is developed for the same business logic. Still your test automation suite works well for the test part. Create a new implementation of scenarios layer and plug it in. The same set of tests can run for the new implementation.