In this post, I will talk about how I normally debug the automated tests failure on a virtual lab environment.
1. Most of our automated test cases are executed as part of the end-to-end workflow. So I first check the build summary to find out how many test passed/failed. In the example below, it is showing that out of 4 tests, 2 passed and 2 failed.
2. Then I will click on the “View Test Results” link shown above in the red box which will launch Microsoft Test Manager (MTM) showing the details of the test run and the test cases which failed and their failure message etc.
3. To understand what’s going on, I typically do the following: -
4. If the test logs are not sufficient to find out the cause, I typically add more logs in the test case and re-run the test and follow the 3.3 step again. This way the logging of the test case also improves so that if the test fails next time, it is easier to debug.
5. To re-run the test case in the above step, normally I tend to queue the workflow again using the binaries from my local drop location. However there is a little variant here where instead of rerunning the workflow, it is possible to login to the machine and run the failed test manually. Once a test case gets executed on a virtual environment, the test/dependent binaries gets deployed in a temporary folder on the virtual machine. So it is possible to just replace the test binary containing better logs in that temporary folder and then manually run the test case using MSTest. To do that, following steps should be followed.
C:\Windows\ServiceProfiles\NetworkService\AppData\Local\VSEQT\QTAgent\1095180f-805b-4c75-aa50-794f8145f8ee\RigTemplate1\Deployment> “C:\Program Files\Microsoft Visual Studio\Common7\Ide\MsTest.exe” /testcontainer:mytestproject.dll /test:mytestmethod
Note: – Ideally you should be logged in to the lab machine as the account under which the test agent was running so that the run parameters are exactly the same. But by default the agent is configured to run as “network service”, so you will not be able to login under that account. In that case you can run the test case under different account and ensure that it works fine. (If under different account test works fine, then the failure should be because of permission issues etc, in that case you can run the test case under network service using psexec tool.)
6. Additionally you can do remote debugging of your test case as well. You can copy the remote debugger monitor on the lab machine, start it and then connect to it using your visual studio, attaching to the agent execution process (QTAgent32.exe). Now you can put breakpoint on your test code and debug.