Web Test Authoring and Debugging Techniques for VS 2008
In Visual Studio 2008, many new features were implemented that eliminate the top issues found in the VS 2005 Web Test recorder, as covered in the white paper Web Test Authoring and Debugging Techniques.
While many areas have been addressed, there will still be times when record/playback fails or does not give the desired result. I’ll be writing a series of blog articles on how the recorder works, “knobs” in the registry that will let you control what does and does not get recorded, and problems you may encounter. Finally, I’ll introduce new debugging techniques that will enable you to find and fix your tests.
New Recorder Technologies in VS 2008
The VS 2008 recorder introduced two key new technologies that eliminate the majority of the problems encountered in the VS 2005 recorder:
1) The new recorder now picks up all requests, whereas the 2005 recorder did not record AJAX request or certain types of popup requests, and
2) The new recorder has a feature to detect dynamic parameters and automatically add the extraction rules and bindings to the test to properly correlate them. A common class of these parameters was dynamic parameters on the query string, such as a session id.
3) If a page has a redirect, the Expected Response URL records the page that was redirected to. In VS 2005, a redirect to the error page was not detected as an error. In VS 2008 the recorder adds the Response URL validation rule to catch this and flag it as an error.
Filtering Requests
Even though the recorder now captures all requests, some requests are still filtered when the web test is generated.
First of all, “dependent” requests are filtered. Web Test has a feature to “Parse Dependent Links”, in which resources on a page such as images, java scripts sources, and css references are not recorded, but at run time the web test engine parses the response, finds all references to dependents, and then fetches them. This works the same way in VS 2008. When the parser runs, it looks for all IMG, SCRIPT, and LINK tags to find the dependent resources and fetch them.
However, a web page can also download content using java script. There was a “hole” in the VS 2005 recorder, in that it could not pick up these requests. A couple of examples of this are mouse over images that are fetched via java script, or images fetched in a mapping program via AJAX calls.
By default, the recorder is configured to filter “static” content such as images and css files. You can override what gets recorded using these registry settings below. These settings are the default (set in code, these registry entries aren’t present after install):
Windows Registry Editor Version 5.00
[HKEY_CURRENT_USER\Software\Microsoft\VisualStudio\9.0\EnterpriseTools\QualityTools\WebLoadTest]
"WebTestRecorderMode"="exclude"
"ExcludeMimeTypes"="image;application/x-javascript;application/x-ns-proxy-autoconfig;text/css"
"ExcludeExtensions"=".js;.vbscript;.gif;.jpg;.jpeg;.jpe;.png;.css;.rss"
You can see that these settings will filter out images, java script source files, and css files. If you want to record everything, you can simply set “ExcludeMimeTypes” and “ExcludeExtensions” to the empty string.
The recorder will also work in an “include” mode, where you specify a list of mime types and extensions to include. For example, the default include list is:
Windows Registry Editor Version 5.00
[HKEY_CURRENT_USER\Software\Microsoft\VisualStudio\9.0\EnterpriseTools\QualityTools\WebLoadTest]
"WebTestRecorderMode"="include"
"IncludeMimeTypes"="text/html;text/xml;text/xhtml;application/xml;application/xhtml+xml;application/soap+xml;application/json"
"IncludeExtensions"=""
Note you have to include the xml, soap, etc., in order to pick up ajax calls.
Folding in Additional Requests
One thing the recorder is not smart about is that requests picked up by the low level recorder are always treated as top-level requests. If you do set the recorder to include images and such, it does not try to figure out which request the dependent came from and store it under the appropriate top-level request. Instead any additional requests are recorded as top-level requests.
Also, asynchronous requests are recorded at the top level of the web test and will be played back synchronously. We hope to add an “Async” property to top-level requests in our next release that will enable you to more accurately simulate the request pattern generated by the browser for AJAX requests.
Note that dependents are fetched in parallel over two connections in the same way the browser fetches them.
Filtering HTTP Headers
HTTP header filtering works in a similar way to request filtering.
Normally in a web test, most HTTP headers are set from the browser template file. Here are the contents of the IE7 browser template file:
<Browser Name="Internet Explorer 7.0">
<Headers>
<Header Name="User-Agent" Value="Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)" />
<Header Name="Accept" Value="*/*" />
<Header Name="Accept-Language" Value="{{$IEAcceptLanguage}}" />
<Header Name="Accept-Encoding" Value="GZIP" />
</Headers>
</Browser>
Notice the MSIE 7.0 in the User-Agent header, which identifies it as IE7.
By default, the recorder only records these additional HTTP headers:
Windows Registry Editor Version 5.00
[HKEY_CURRENT_USER\Software\Microsoft\VisualStudio\9.0\EnterpriseTools\QualityTools\WebLoadTest]
"RequestHeadersToRecord"="SOAPAction;Pragma;x-microsoftajax"
Your application may send additional custom headers. In that case, you can change the recorder settings to add the headers your app sends.
The recorder is set to never record these headers, which are automatically handled in the HTTP engine:
"Authorization", "Proxy-Connection", "Connection", "Host", "Expect", "Content-Length"
Recorder Settings Summary
You can see the new web test recorder is powerful, but the default settings may night be right for your application.
1. The default filtering of static content may mask performance problems in your application. Consider recording additional requests.
2. HTTP Headers that your application depends on may be filtered out. If your application uses custom headers, consider changing your recorder settings to pick them up.
Detecting Dynamic Parameters
A major new feature in VS 2008 is the ability to detect dynamic parameters. A dynamic parameter is a parameter whose value is generated each time a user runs the application, and therefore playback of recorded values won’t work.
The best example of this is a dynamically generated session ID. For apps that support login, each time a user logs in, the server generate s a unique session ID to track the user. This session ID may then be passed back to the server via a cookie, form field, or query string parameter. In VS 2005, Web tests handled cookies and hidden fields. Note there were some bugs in hidden field binding in VS 2005, some of which were fixed in SP1 and some have been fixed in VS 2008.
VS 2008 adds support for two more types of dynamic parameters: query string parameters and form fields (other than hidden fields).
The way it works is at record time, the value of each form post and query string parameter is recorded, and an appropriate extraction rule is computed. Immediately after recording the web test is played back by the correlation tool. During playback, the tool applies the extraction rule and compares the extracted value with the recorded value. If they differ, it flags the parameter as a dynamic parameter.
Dynamic Detection Playback Fails
A subtle problem you may encounter is Dynamic Detection playback failing. As mentioned above, in order to detect dynamic parameters, the web test is played back under the covers. If this playback fails you may or may not get a complete list of parameters to promote. You can see whether or not it failed by looking in the result window, where the result of the playback is displayed. If it does fail, fix your test per the guidance below and then re-run dynamic parameter detection from the web test toolbar.
When to not Promote a Parameter to a Dynamic Parameter
There may be times when playback thinks a parameter is dynamic when in fact it is not.
One example of this is when cookies in IE are used to populate form fields. For example, our Web test forums provide an option to save your user name for login. If you record a Web test with this setting turned on, a cookie is used to set the value of the user name so it is automatically filled in. When dynamic parameter detection runs, it looks at the user name value and sees that it is different than the recorded value. Aha! A dynamic parameter! If you accept this as a dynamic parameter, the web test engine will scrape the value out of the response rather than playing back what you typed in, clearly not the desired behavior.
To avoid problems like this, clear your browser cookie cache prior to recording.
Playback Doesn’t Work, Now What?
In general, the problem is that the http requests sent by the web test engine are somehow different than the requests sent by IE. The first challenge is figuring out what IE is sending over http. There are different tools for doing this, include Fiddler (http://www.fiddler2.com/), or NetMon. There is also a new feature in the web test recorder for generating a recording log from a recording. Turning this on is probably the easiest way to see what IE is sending.
Common Problems to Look For
Common problems we’ve seen customers encounter:
1) Missing custom HTTP headers. See the comments on HTTP headers in the recorder section above.
2) Cookies saved on your local machine. A good example of this is cookies stored on your machine to automatically fill out the user name. See the section on Detecting Dynamic Parameters above.
Using the Recorder Log
To turn on the recorder log, set these registry entries:
Windows Registry Editor Version 5.00
[HKEY_CURRENT_USER\Software\Microsoft\VisualStudio\9.0\EnterpriseTools\QualityTools\WebLoadTest]
"CreateLog"=dword:00000001
"RecorderLogFolder"="c:\\RecorderLogs\"
This will result in a log file for each recording session. Open the log file and find the failing request, then carefully compare all parts of the request in the request log to the request in web test playback:
· URI
· Query string parameters and values
· HTTP headers, including custom headers and cookies
· Post body
Once you identify the difference, you can go back to the web test to fix the problem. Some areas to look out for:
· Missing custom http headers in the web test
· Incorrectly handled dynamic parameters (including
o parameters marked as dynamic that aren’t, or )
o Incorrect extraction rules
Using Fiddler
Because of the underlying technology, there may be times when either the web test recorder or web test playback viewer does not accurately reflect what is actually getting sent over the wire in subtle ways. There may even be times when the web test recorder interferes with the requests sent by IE. To get a true picture of what is getting sent, you can use a tool like Fiddler or NetMon.
One example of this is in VS2005, web test playback always showed cookies being sent in a single header, when in fact they were sent in multiple http headers.
You can also configure Fiddler to run while the web test is running to capture the http traffic the web test is sending. To do this, you need to create a web test plugin and add this code to the constructor:
this.Proxy = "http://localhost:8888";
WebProxy webProxy = (WebProxy)this.WebProxy;
webProxy.BypassProxyOnLocal = false;
Web Test Logging
Web test playback is a great tool for seeing what is going on in the web test engine. However, there may be times when you just want to dump the entire session to a text file. One limitation in Web test playback is there is no way to search across all the requests and responses in the session. We have developed a web test logging sample plugin that will do just that and plan to release it to CodePlex soon at http://www.codeplex.com/TeamTestPlugins.
Conclusion
Record playback in VS 2008 has been vastly improved over VS 2005. The recorder now has the capability to pick up all requests sent from IE, the Dynamic Parameter Detection feature catches the most common cases for dynamic parameter correlation, the Response URL validation rule, and a number of bugs have been fixed to make playback more reliable. Even with these new features, there may be times where the recorder by default isn’t capturing the meaningful requests for your application, and you will want to tweak the recorder settings to record additional requests.
There may also be times where record/playback will fail, and you will need to debug the web test to figure out what is going wrong. The addition of the recorder logging feature will make this easier.