In this blog, I will be writing about the integration of test runner with diagnostic data adapters. This integration allows test runner to collect a lot of interesting data in the background while testing is done. This data is automatically attached to the bug filed from the test runner. The amount of data is so rich that, in most cases, the developer should be able to know the cause of the bug by just looking at this data. This should also considerably reduce the amount of back and forth between developers and testers on how to repro the bug.

At the heart of rich bug data is an extensible data adapter framework that gathers interesting data during test execution. By default, a few data adapters are enabled. You can enable more data adapters based on your need or create your own data adapter if you have a special need to collect your own custom data.

I will not get into specifics of what each data adapter does. However, I will deliberate more on the integration of data adapters with test runner. For now consider data adapters as some kind of abstract data providers.

How does test runner interact with data adapters?

When you run a manual test from MTLM, you will get to the test runner.

image

Below are the operations you perform on the test runner and how interactions happen with the data adapters:

User Action What does test runner do?
What do data adapters do?
Starts the “first test” in the list
(Please notice the emphasis on the first test. This is considered as the start of the test run)
Initialization event is sent Data adapters initialize themselves. For instance, the intellitrace and test impact data adapters start instrumenting new applications only after initialization is complete.
Start the test Start test event is sent Data adapters start data collection.
Pause the test Pause test event is sent Data adapters pause data collection.
Resume the test Resume test event is sent Data adapters resume data collection.
File bug The test runner pauses data collection, and requests data from the data adapters  
    The data adapters return data. These are typically files/streams. The stream data are serialized to file by the data adapter framework. The files are stored physically on the same machine in which the test runner is running.
  The data adapter files are attached to test result and linked to the bug.  
  The test result is saved. This starts uploading the data adapter files to the server in background. You can do all other operations, while this upload is happening in the background.  
End the test End test event is sent Data adapters stop data collection
  Data is requested from data adapters and attached to the test result The data adapters return data to test runner.
Save the test The test result is saved and all attachments start uploading to the server in background  
Close test runner Session end event is sent The data adapter dispose any resources that they may be using

There are a few things to notice here.

1. All data adapter attachments exist on the test result. They are just referenced from the bugs. So if you delete a test result or a test result attachment, bugs that refer to these attachments will have stale links. Be very careful while deleting test results. Ensure that they are not linked to any bugs.

2. The data adapter attachments can be quite large. For instance, the intellitrace adapter logs can run into many mega bytes. These attachments are uploaded to server in the background when you file the bug or you save the test. As a result, you can continue running other tests while these attachments are being uploaded in the background. This uploading continues to happen even after you close the test runner and return back to MTLM. For more information on this, refer this blog.

3. The data adapter attachments are all saved first on the machine on which the test runner is running and then uploaded to the server. Even in remote data collection scenario, the data adapter logs are first saved on the test runner machine and then uploaded to the server.