Event ordering in Web Tests

edglas — Thu, 28 Aug 2008 18:49:49 GMT

Web tests have a number of extensibility points before and after a request. What order do they run in?

The hooks are:

WebTestPlugin.PreRequest

WebTestRequestPlugin.PreRequest

ExtractionRule.Extract

ValidationRule.Validate

Here is a sequence diagram that shows what order things fire:

There is no guaranteed order of firing for WebTestPlugin.PreRequest and RequestPlugin.PreRequest. Also if you have >1 Extraction or Validation rule on a request there is no guaranteed order they are run in.

Also note that Pre and Post request events do not fire by default for dependents. You can hook the PostRequest call on a top level request and iterate through dependents to wire up event handlers if you need that functionality.

Ed.

New templates for plugins and rules

edglas — Tue, 19 Aug 2008 10:30:35 GMT

Inspired by templates developed by Dennis Bass of the services test labs, I added new item templates to the codeplex.com\TeamTestPlugins project.

If you install these on your machine it makes creating new plugins much easier:

If you work with web and load tests you'll find these very handy, and they are xcopy-installed on your machine (you'll have them working in less than 5 minutes).

Now to get these in our next release. :)

Ed.

Catching errors as they happen: WebTest ValidateResponseUrl

edglas — Tue, 24 Jun 2008 16:20:55 GMT

In VS 2008, we added a new Validation Rule ValidateResponseUrl.

In VS 2005, a very common problem was that a page would redirect to the error page, but the web test engine did not automatically recognize this as an error. Then downstream in the playback another page would fail, usually with "hidden field blah not found in the context" error, as a side effect of the earlier error. Many times these errors would happen in a load test, but since it wasn't trapped as an error there was no error log for the page to figure out what was going on.

Many users hit this problem in 2005, and since the "side-effect" error was reported, but not the actual error, it was very confusing. This is problem is covered in both Web Test Authoring and Debugging Techniques, as well as Sean Lumley's great posts: Debugging Web Tests and Debugging Load Test Errors.

In order automatically catch redirects to the error page (which is exactly the case Sean documents in the Debugging Load Test Errors post), we 1) added the Expected Response URL property to web test requests, and 2) added test-level validation rules that apply to every request, and 3) automatically add the Response URL validation rule at the test level. Now if a page redirects to the error page, that page will fail. This is essential in a load test as it enables you to get the error details for that page.

Why does my web test pass if I run it, but fail if I run it in a load test?

The new rule can lead to unexpected failures in a load test.

In VS 2008, we also beefed up our vUser context in a load test. One thing the context now carries from iteration to iteration is the set of cookies for a user (thus emulating the browser cookie store). Let me take a typical example of how this can trip up your web test in a typical shopping cart type app. Record a web test as following:

Browse to an item and put it in your cart
Go to checkout
Checkout requires login, so app redirects to login
Login, which sets a cookie
Login redirects to checkout
Checkout

When you run this in a load test, the vUser context will be carried from iteration to iteration, and the context contains any cookies set in a previous iteration. So the second iteration for the vUser will look like this:

Browse to an item and put it in your cart
Go to checkout
Checkout requires login, but now the user is already logged in (cookie is set) so app goes straight to checkout
The ValidateResponseUrl rule fails the web test because it is expecting to redirect to the login page

To fix this, you must explicitly log your user in during recording rather than counting on the redirect. If you only want the user to login once you can use the vUser Init and terminate functions in the load test test mix settings.

Ed.

Ed Glas's blog on VSTS load testing : Validation Rule

Event ordering in Web Tests

New templates for plugins and rules

Catching errors as they happen: WebTest ValidateResponseUrl