Sean Lumley's Blog

Debugging Custom Validation/Extraction rules, Web Test and Load Test Plug-ins

2009-11-17T02:24:00Z

Most of my recent posts have been about new features in VS 2010. This post is going to show you how to debug your custom rules. This applies to VS 2005, VS 2008 and VS 2010.

First let’s create a Test Project
Then Right click on the project node and select Add –> Web Test
Record a simple web test
Now add a new class to the project. We can re-use the sample extraction rule from this earlier post: Sample Extraction Rule Copy and paste the code from this sample into the new class file.
Compile the project
Right click on a request that has a post parameter and select the new rule:
Enter a context parameter name, name of the form field to extract and the index of the form field to extract. I am going to enter _EVENTVALIDATION. Then click OK.
The test is ready to run. If you click the play button in the web test editor the test will run and the you will see the results in the Web Test viewer. But what if your custom rule/plugin isn’t working as expected. When I playback my test, my rule failed to find the _EVENTVALIDATION field.

Debugging the rule

First we need to place a break point in the custom rule. For an extraction rule, place a break point in the extract method.
Now we need to start the test. This time we don’t want to just click play. If you hit the drop down on the web test editor play button, you will see that there are 4 options. This time choose Debug Test.
When the test runs, it will hit your break point.
Now we can step through and see why the rule failed.
In my case I can see that when I typed in the form field, I missed a _. Now if I correct my mistake and rerun, I can see the value extracted into the context.

This shows you how to debug web tests rules/plugins. But the same applies to load test plugins. The load test editor play button is also a split button. So just place your break point in the load test plug-in and choose debug test instead of play test.

I hope this helps in debugging your custom rules.

VSTS 2010 Feature: API for Processing Web Test Results

2009-11-11T15:49:00Z

One request which we have received a number of times is something like, “The Web test playback UI is great for when you are in VS, but I need to share these results with others. Can I generate a report from the web test result?” In VS 2005 and 2008, the web test result was stored in a trx file which was an XML file. We did not have a public API for processing the web test results. In VS 2010, we have made some changes in this area.

First, the web test results are no longer completely stored in the trx file. The trx file has reference to a .webtestresult file which has the actual result. Look under you TestResults directory which is located in your TestProject directory by default. In this directory, you will see a number of trx files. If you then search into the folder that corresponds with the trx and then navigate into the IN directory, you will find the .webtestresult file. This file will have the name of the web test that was being executed. So if you ran WebTest1.webtest, you will find a result file names WebTest1.webtestresult.

We have made the API for processing .webtestresult files public. So now you can process this file and create your own custom reports for web test results. I am going to walk you through a demo for how to do this. We will create a simple console app that will read in the webtestresult file and then create an XML document. This will show you how to process the result file so you can generate your own custom reports.

WebTestResult Model

There are a few classes that you will have to become familiar with. These are all in the Microsoft.Visual Studio.TestTools.WebTesting namespace.

WebTestResultDetailsSerializer – This class will be used to read the .webtestresult file.
WebTestResultDetails – This class is what the serializer will return. This contains the full result. It contains a collection of iterations. Each iteration contains a collection of WebTestResultUnit objects.
WebTestResultUnit - This is the base class for page, transaction, comments, loops, and conditional results.
WebTestResultPage – This contains information about a page including the request, response, dependent Urls, redirected Urls, timing info etc.
WebTestResultTransaction – This contains information about a transaction such as name and response time. It also contains a collection of WebTestResultUnit objects which are contained within the transaction.
WebTestResultCondition – This contains information about a web test condition such as condition being tested and whether or not the condition was met. It also contains a collection of WebTestResultUnit objects which are contained within the condition.
WebTestResultLoop – This contains information about a web test loop such as the loop condition. It contains a collection of WebTestResultLoopIteration objects. It has one object for each iteration of the loop.
WebTestResultLoopIteration – This contains information about a particular loop iteration. It also contains a collection of WebTestResultUnit objects which are contained within the loop iteration.
WebTestResultComment – This contains information about a web test comment.

Sample Console Application

Now that we have gone over the basic classes we will be using, let’s create a sample app which will create a custom report.

First launch VS and create a new Console Application project.
Right click on the project in solution explorer and select properties.
Make sure the target framework is set to .NET Framework 4 and not .NET Framework 4 client profile
Right reference node and select Add Reference…
In the Add Reference dialog select the Microsoft.Visual Studio.QualityTools.WebTestFramework.dll
Right click the project node and add a new class called ReportGenerator. Here is the code for that class. Let’s go over what is happening.
- GenerateReport – This method takes the webtestresult file. It creates and instance of the serializer which will then return the WebTestResultDetails. My sample is going to return an xml document, so I am also creating an Xml document here which I will be building up. Then you can see that it starts iterating over the Iterations. For each iteration it then iterates over the children in the iteration which are all WebTestResultUnit objects. It passes the WebTestResultUnit to ProcessResultUnit
- ProcessResultUnit – This method needs to figure out what kind of result unit it is and then do the appropriate thing. For example, for pages, it will Create a page node with the url and response times as attributes. Then it will iterates through the redirected requests and the dependent requests for the page. For a transaction, it will create a transaction node with the name and response time of the transaction. It will then iterate over the children of the transaction which are all WebTestResultUnit objects and call ProcessResultUnit for each of the children

using System.Xml;
using Microsoft.Visual Studio.TestTools.WebTesting;

namespace ConsoleApplication1
{
    internal class ReportGenerator
    {
        public ReportGenerator()
        {

}

        public XmlDocument GenerateReport(string file)
        {
            //create the serialzer
            WebTestResultDetailsSerializer serializer = new WebTestResultDetailsSerializer();

//deserialize the webtest result
WebTestResultDetails details = serializer.Deserialize(file);

            //create root element for the document
            XmlDocument doc = new XmlDocument();
            XmlElement elem = doc.CreateElement("WebTestResultDetails");
            doc.AppendChild(elem);

            //now we need to loop through each iteration
            foreach (WebTestResultIteration iteration in details.WebTestIterations)
            {
                XmlElement iterationElement = doc.CreateElement("Iteration");
                elem.AppendChild(iterationElement);
                //loop through and process each child. A Child could be a
                //transaction, page, comment, innertest, etc.
                foreach (WebTestResultUnit unit in iteration.Children)
                {
                    ProcessResultUnit(unit, doc, iterationElement);
                }
            }

return doc;
}

        private void ProcessResultUnit(WebTestResultUnit unit, XmlDocument doc, XmlElement elem)
        {
            if (unit is WebTestResultPage)
            {
                XmlElement pageElement = doc.CreateElement("Page");
                elem.AppendChild(pageElement);

                WebTestResultPage page = unit as WebTestResultPage;
                pageElement.SetAttribute("Url", page.RequestResult.Request.UrlWithQueryString);
                pageElement.SetAttribute("ResponseTime", page.RequestResult.Response.Statistics.MillisecondsToLastByte.ToString());
                //process redirects
                if (page.RedirectedPages.Count > 0)
                {
                    //create the Redirects element
                    XmlElement redirectsElement = doc.CreateElement("Redirects");
                    pageElement.AppendChild(redirectsElement);
                    foreach (WebTestResultPage redirect in page.RedirectedPages)
                    {
                        ProcessResultUnit(redirect, doc, redirectsElement);
                    }
                }

                //process each dependent
                if (page.RequestResult.DependantResults.Count > 0)
                {
                    //create the DependentRequests element
                    XmlElement dependentsElement = doc.CreateElement("DependentRequests");
                    pageElement.AppendChild(dependentsElement);
                    foreach (WebTestRequestResult request in page.RequestResult.DependantResults)
                    {
                        XmlElement dependentElement = doc.CreateElement("DependentRequest");
                        dependentsElement.AppendChild(dependentElement);
                        dependentElement.SetAttribute("Url", request.Request.UrlWithQueryString);
                    }
                }
            }
            else if (unit is WebTestResultTransaction)
            {
                WebTestResultTransaction transaction = unit as WebTestResultTransaction;

                //create the transaction element
                XmlElement transactionElement = doc.CreateElement("Transaction");
                elem.AppendChild(transactionElement);
                transactionElement.SetAttribute("Name", transaction.Name);
                transactionElement.SetAttribute("IsIncludedTest", transaction.IsIncludedTest.ToString());
                transactionElement.SetAttribute("ResponseTime", transaction.ResponseTime.ToString());

                //now we need to process each child of the transacction which can any resultunit type.
                //so iterate through children and call ProcessResultUnit
                if (transaction.Children.Count > 0)
                {
                    XmlElement children = doc.CreateElement("Children");
                    transactionElement.AppendChild(children);
                    foreach (WebTestResultUnit child in transaction.Children)
                    {
                        ProcessResultUnit(child,doc,children);
                    }
                }
            }
            else if (unit is WebTestResultCondition)
            {
                WebTestResultCondition condition = unit as WebTestResultCondition;

                //create the Condition element
                XmlElement conditionElement = doc.CreateElement("Condition");
                elem.AppendChild(conditionElement);
                conditionElement.SetAttribute("Name", condition.ConditionStringRepresentation);
                //now we need to process each child of the condition which can any resultunit type.
                //so iterate through children and call ProcessResultUnit
                if (condition.Children.Count > 0)
                {
                    XmlElement children = doc.CreateElement("Children");
                    conditionElement.AppendChild(children);
                    foreach (WebTestResultUnit child in condition.Children)
                    {
                        ProcessResultUnit(child, doc, children);
                    }
                }
            }
            else if (unit is WebTestResultLoop)
            {
                WebTestResultLoop loop = unit as WebTestResultLoop;

                //create the Loop element
                XmlElement loopElement = doc.CreateElement("Loop");
                elem.AppendChild(loopElement);
                loopElement.SetAttribute("Name", loop.LoopStringRepresentation);

                //now we need to process each child of the loop which can any resultunit type.
                //so iterate through children and call ProcessResultUnit
                if (loop.Children.Count > 0)
                {
                    XmlElement children = doc.CreateElement("LoopIterations");
                    loopElement.AppendChild(children);
                    foreach (WebTestResultUnit child in loop.Children)
                    {
                        ProcessResultUnit(child, doc, children);
                    }
                }
            }
            else if (unit is WebTestResultLoopIteration)
            {
                WebTestResultLoopIteration loopIteration = unit as WebTestResultLoopIteration;

                //create the LoopIteration element
                XmlElement loopIterationElement = doc.CreateElement("LoopIteration");
                elem.AppendChild(loopIterationElement);
                loopIterationElement.SetAttribute("Number", loopIteration.IterationNumber.ToString());
                loopIterationElement.SetAttribute("IsConditionalRuleMet", loopIteration.IsConditionalRuleMet().ToString());

                //now we need to process each child of the loop which can any resultunit type.
                //so iterate through children and call ProcessResultUnit
                if (loopIteration.Children.Count > 0)
                {
                    XmlElement children = doc.CreateElement("Children");
                    loopIterationElement.AppendChild(children);
                    foreach (WebTestResultUnit child in loopIteration.Children)
                    {
                        ProcessResultUnit(child,doc,children);
                    }
                }
            }
            else if (unit is WebTestResultComment)
            {
                WebTestResultComment comment = unit as WebTestResultComment;

                //create the Comment element
                XmlElement commentElement = doc.CreateElement("Comment");
                elem.AppendChild(commentElement);
                commentElement.SetAttribute("Text", comment.Comment);
            }
        }
    }
}

Now go back to Program.cs file. This class just needs to know where the results file is and where to store the output file. So we will check to make sure we get this information and then pass it along the report generator. This class looks like this:

using System;
using System.Xml;
using System.IO;

namespace ConsoleApplication1
{
    class Program
    {
        static void Main(string[] args)
        {
            if (args.Length != 2)
            {
                Console.WriteLine("Usage: ReportGenerator <WebTestResultFile> <OutputFile>");
                return;
            }

            if (!File.Exists(args[0]))
            {
                Console.WriteLine("WebTestResult file does not exist");
                return;
            }

            if (File.Exists(args[1]))
            {
                Console.WriteLine("Output file already exists. Please specify a new name.");
                return;
            }

ReportGenerator generator = new ReportGenerator();
XmlDocument report = generator.GenerateReport(args[0]);

            XmlTextWriter writer = new XmlTextWriter(args[1], null);
            writer.Formatting = Formatting.Indented;
            report.Save(writer);
        }
    }
}

Now compile your code. to run it, open a command prompt and go to directory that the exe was compiled into. You would run it with something like: ReportGenerator c:\Webtest1.webtestresult c:\NewReport.xml

Here is a sample output created from one of my webtests:

<?xml version="1.0"?>
<WebTestResultDetails>
<Iteration>
    <Transaction Name="Transaction1" IsIncludedTest="False" ResponseTime="88.384">
      <Children>
        <Transaction Name="WebTest8.Storecsvs" IsIncludedTest="True" ResponseTime="56.825">
          <Children>
            <Page Url="http://sampleserver/Storecsvs" ResponseTime="28829">
              <Redirects>
                <Page Url="http://sampleserver/Storecsvs/" ResponseTime="27996">
                  <DependentRequests>
                    <DependentRequest Url="http://sampleserver/Storecsvs/IBuySpy.css" />
                    <DependentRequest Url="http://sampleserver/Storecsvs/images/grid_background.gif" />
                  </DependentRequests>
                </Page>
              </Redirects>
            </Page>
          </Children>
        </Transaction>
        <Page Url="http://sampleserver/Storecsvs/productslist.aspx?CategoryID=15&selection=1" ResponseTime="27737">
          <DependentRequests>
            <DependentRequest Url="http://sampleserver/Storecsvs/images/sitebkgrdnogray.gif" />
            <DependentRequest Url="http://sampleserver/Storecsvs/IBuySpy.css" />
          </DependentRequests>
        </Page>
        <Page Url="http://sampleserver/Storecsvs/ProductDetails.aspx?productID=394" ResponseTime="316">
          <DependentRequests>
            <DependentRequest Url="http://sampleserver/Storecsvs/IBuySpy.css" />
            <DependentRequest Url="http://sampleserver/Storecsvs/images/sitebkgrd.gif" />
          </DependentRequests>
        </Page>
      </Children>
    </Transaction>
    <Comment Text="Sample Comment" />
    <Page Url="http://sampleserver/Storecsvs/AddToCart.aspx?ProductID=394" ResponseTime="251">
      <Redirects>
        <Page Url="http://sampleserver/Storecsvs/ShoppingCart.aspx" ResponseTime="0">
          <DependentRequests>
            <DependentRequest Url="http://sampleserver/Storecsvs/images/sitebkgrd.gif" />
            <DependentRequest Url="http://sampleserver/Storecsvs/IBuySpy.css" />
          </DependentRequests>
        </Page>
      </Redirects>
    </Page>
    <Loop Name="Loop ( Repeat 1 times )">
      <LoopIterations>
        <LoopIteration Number="1" IsConditionalRuleMet="True">
          <Children>
            <Page Url="http://sampleserver/Storecsvs/ShoppingCart.aspx" ResponseTime="27729">
              <DependentRequests>
                <DependentRequest Url="http://sampleserver/Storecsvs/IBuySpy.css" />
                <DependentRequest Url="http://sampleserver/Storecsvs/images/sitebkgrd.gif" />
              </DependentRequests>
            </Page>
          </Children>
        </LoopIteration>
        <LoopIteration Number="2" IsConditionalRuleMet="False" />
      </LoopIterations>
    </Loop>
    <Page Url="http://sampleserver/Storecsvs/ShoppingCart.aspx" ResponseTime="542">
      <DependentRequests>
        <DependentRequest Url="http://sampleserver/Storecsvs/images/sitebkgrd.gif" />
        <DependentRequest Url="http://sampleserver/Storecsvs/IBuySpy.css" />
      </DependentRequests>
    </Page>
</Iteration>
</WebTestResultDetails>

I hope this helps getting you started in creating your own custom web test result reports. There is plenty of other information available to you. My sample just gives you an idea of how to parse through the objects. More or less anything that you see in the web test playback UI is available in the result objects.

VSTS 2010 Feature: Load Testing Run Comparison Report in Excel

2009-11-07T00:43:51Z

This blog post is going to be about one of my favorite new features added for Beta2. In Beta1 we added excel integration for reporting on Load Test Reports. In beta1, the report that was added was a trend report which would give you a feel for how certain counters changed from run to run. Check out this blog post for more info on that report: Excel Trend Report

The new report type which has been added gives a more detailed comparison of 2 runs. Let’s walk through creating the new report and looking at the output.

1) Launch Excel and click on the load test ribbon. The click the Load Test Report Button.

2) In the first page of the wizard enter the database server and select the database. Then click next.

3) On second page of the wizard choose “Create a Report” and click Next.

4) On third page, select Run Comparison as the report type. Then click next.

5) On fourth page, Enter a name for the report and choose a load test that you want to use for report generation. Click next.

6) The fifth page is where you select the runs to compare. Select 2 runs and click next.

7) The sixth page is where you select the counters that you want included on the report. You will notice that there are some counters which have been pre-selected. These are just some default counters. Select whichever counters you want.

8) Then click finish and the report will be generated.

Now let’s take a look at what was generated for you.

1) Table of Contents – This page gives you a list and links to each page in the report.

2) Runs – This page gives a little info about the 2 runs which were selected.

3) The next three sheets give a comparison of each test, page and transaction in the load test.

4) Test Comparison – The page and transaction sheets will be very similar to this. The 2 charts at the top will show the top 5 tests which had the largest performance improvement and top 5 tests which showed the worst regressions. The table underneath the chart will show all tests in the load test. It lists out the scenario and test name. Then gives the Baseline value and Comparison value followed by a % change from baseline. There is also a color scale added to help you quickly identify problems.

5) Page Comparison – This is very similar to the test sheet, but gives a breakdown of each page. The other difference is that it includes the response goal value and gives you a % change from goal. So if you set page goals, they would appear here.

6) Transaction Comparison – Again, this is similar to test and page sheets.

7) Machine Comparison – This page will compare counters between the two runs. As you can see from the screen shot below, it lists the Machines used in the Baseline and Comparison runs as well as the roles for the machine. The role is important because in order for two machines to be compared the roles need to match. For example, if MachineA was the IIS server in Baseline and was SQL Server in Comparison run, then the report will not compare that machine between the 2 runs. You will still have the counters for that machine listed, but it will not line them up for a comparison. Now if MachineA was the IIS server in Baseline run and MachineB was IIS role for comparison run, those 2 machines will be compared. This is useful if you will be doing many runs, but are not guaranteed the same set of machines for every run you do. As you can see from my report, TeamTestWeb1 played the role of IIS and SQL Server for both of my runs which is why they were compared.

8) Error Comparison – This page give you a breakdown of errors in each run and % change between the runs.

As mentioned in the previous blog on excel reporting, the report definition that you generate is saved in the load test database. So you can come back later and generate the report again, or use that as a template to generate other run comparison reports.

I hope you find this report useful and let us know what you think.

VSTS 2010 Feature: Making it easy to debug web tests using Fiddler

2009-11-03T13:41:00Z

VSTS 2010 Beta2 is out and there are a number of blog posts I want to write on the features now available to you. Check out Ed Glas’s blog post on the full set: Beta2 Features

The first small feature that I am going to talk about is better integration with Fiddler. If you are not familiar with Fiddler, it is Web Debugging Proxy and is very useful when debugging web tests. You can find Fiddler here: Fiddler There are times when debugging a web test that it is helpful to get another view of the data that is being sent to the servers you are testing. It has been possible to use fiddler for these purposes since VS 2005, but we have made it much easier. In VS 2005 or VS 2008 or even VS 2010 Beta1, you had to write a small bit of code to correctly hook up to fiddler. The code you would have to add looked like the following and needed to be added in the constructor of a coded test or a in a web test plug-in:

this.Proxy = "http://localhost:8888";
WebProxy webProxy = (WebProxy)this.WebProxy;
webProxy.BypassProxyOnLocal = false;

With the beta2 release we have made this even easier. Now to get Fiddler to capture the traffic from your web tests, you just need to do the following:

Launch Fiddler
Play your web test

That is it. No code to write or any extra configuration. We have only made this work when you are running a web test by itself. If the web test is running as part of a load test, the traffic will not be routed through fiddler. You could still use the old method to get this to work if needed.

I will be writing a few more blog posts in the coming weeks on the new features, so please try them out and let us know what you think.

VSTS 2010 Feature: Load test virtual user activity visualization

2009-06-09T17:30:00Z

This is another post about new VS 2010 web/load testing features. You can get the beta here: beta download

In this post I am going to talk about a new load test feature called “Load test virtual user activity visualization”. This feature is a new way to look at load test data. In this feature we will provide a look at what users are doing during a load test. It will help answer the question, “I see a spike in CPU or a drop in requests/sec, what tests or pages are running during this spike or drop? Also are they passing or failing?”. Prior to VSTS 2010, we could answer this question, but not easily. Now you will be able to just right click on a graph in the load test analyzer and select Go to User Detail.

Configuring Load Test to Collect Full Details

In order to show the detail view, you need to configure your load test to collect full details. In this mode we collect detailed information about every test, page and transaction. This option was available in VS 2005 and VS 2008, but once the data was collected, the only way to access this info was by SQLqueries. In VS 2005 and 2008 this was not the default behavior. In VS2010, the default behavior is to collect full details. If you are upgrading a VS 2008 or 2005 load test, do the following to enable full detail collection.

1) Open the load test in the load test editor

2) Click on the Run Settings node

3) Bring up the property browser

4) Click “Timing Details Storage” and select “All Individual Details”

Accessing User Detail View

The user detail view is only available once the test is complete. After the test finishes, you will see the following dialog:

Choose Yes to enable the post run options. There are 2 ways to get the user detail view. First you can click the User Detail button on the toolbar.

Or you can right click on a graph and select “Go to user detail”. If you use this option, the user detail view will auto zoom to the part of the test that you right clicked on. So if you click on about the 30 second mark, then the detail view will have the 30 second mark plus/minus another number of seconds.

User Detail View

Main Screen

I executed a load test with 2 web tests that access the IBuySpy website. One test just browses the site and the other will browse and place an order. The tests also have transactions. The load test has 25 users executing tests for ten minutes. When you access the user detail view you will see the following:

Let go over what you are looking at. Each horizontal row represents a virtual user in the test. Each line in a row represents one unit of whatever you are currently looking at: Test, Page, or transaction. By default the initial view will display test. So each line in the row represents a test. The length of the line represents the duration of the test and the color represents which test was executed. Look at the legend on the left to see what each color represents.

Legend

Now let’s go over what is in the legend. Here is what it looks like:

At the top of the legend is a combo box that lets you choose what type of result you want to look at: Test, Page or transaction. Change the combo to have the view refreshed into a new result type. The first 2 options below the combo are “Show only results with errors” and “Show only results with logs”. If you check Show only results with errors, the graph will remove successful results. This can be useful, because you might see that one test is failing often during time that tests/ sec drop or CPU spikes. This is what this looks like when selected:

As you can see, when this happens you are left with just checkout tests. So this indicates that the browse test seems to be working, but there appears to be a problem with the checkout process.

The next 2 options are “Highlight Errors” and “Highlight results with logs”. Many times the tests that are causing errors will cause the spike in cpu. Or maybe you will notice that a certain test fails while another test is running or immediately after another test has executed.

When you click highlight errors, each test that failed will be turned red. This will help you get an idea of what tests are failing and it might show a pattern that indicates why a certain condition such as a spike in CPU is occurring. Here is what this looks like:

This now shows us which tests are failing, but it also shows us that there are Checkout tests which are passing. So now we know that the checkout process sometimes works, but when we run it under load, it will also fail.

The other option is highlight tests with logs. If you select this, each test that has as test log will be turned green. This will show you tests that you can get additional information on. Check out this blog for how to enable test log collection: Test Log Collection.

Next you will notice that each test/transaction/page has a check box next to it. If you uncheck the check box, that test/transaction/page will be removed from the view. For example,

I unchecked the Browse test, so now I see just Checkout tests.

Tooltips

Another nice part of this view is that you can get very detailed information in a tooltip by hovering a line. For example, when I hover the top line in the view, I see the following:

As you can see, you get information about the user, scenario, test, outcome, network , start time, duration and machine executed on. Also if there is a test log associated with the result, you will see a hyper link with test log. If you click the test log hyper link, the log for that test will appear. For a web test, the web test result will be opened in the web test playback UI. If it was a unit test, you will have the unit test result viewer opened. For example if I click on the above hyper link, I see:

So now I can get the full web test result and can see the failure. So my checkout test hits a primary key violation during the checkout request. So this is really nice when you click highlight errors, hover a result with an error and then pull up the failed test.

Here is what the tooltip looks like if I hover a page. You can see from here that you get the URL as well as what browser was being emulated. Also you can drill into the test log for the page if a log was collected

Scrolling and Zooming

You may have noticed the scroll/zoom bar at the bottom. This works exactly like the scroll/zoom bar on the regular graphs. You can scroll through different test intervals and zoom down to one sample interval or expand it to show the entire test. For example, here is my entire test run.

This gives you the 10,000 foot view of what is happening in your test. For example, you might see a particular iteration of a test that takes much longer than other iterations of same test. Again you can highlight errors and see the following:

This shows me that my checkout tests passed initially and then they all failed.

You can also highlight a part of the view and zoom to that area. So if I highlight the following:

Then my graph will automatically zoom to this area.

You will also see a vertical scroll bar if there are too many users to display on the window.

Try it out and let us know how you like it

I will follow this post up with other examples of how to use this view and what patterns you may see, such as deadlock type problems. But hopefully you see the value in now being able to answer the question of what is happening at a certain point in the load test. You can quickly jump to the user detail and see the tests/pages/transactions, then even pull up a test log for the part of the test you are looking at. Please provide feedback once you have a chance to try this out.

Technorati Tags: VSTS,Web Test,VSTS2010,Visual Studio 2010

VSTS 2010 Feature: Extensible recorder plugins for modifying recorded web tests (including adding custom dynamic parameter correlation)

2009-06-04T15:15:00Z

This is another post about new VS 2010 web/load testing features. You can get the beta here: beta download.

In this post I am going to talk about a new feature that can help with web test recording. The feature is extensible recorder plug-ins for modifying recorded web tests. Basically we are giving you the opportunity to modify the recorded web test after you click stop on the web test recorder bar but prior to the web test being fully saved back to the web test editor. So what problems does this help with? The main one is performing your own custom correlation. In VS 2008 we added a process which runs post recording that attempts to find dynamic fields. You can read this blog post for more information: http://blogs.msdn.com/slumley/pages/web-test-correlation-helper-feature-in-orcas.aspx

This process still exists, but this process does not always find all dynamic fields for an application. So if we did not find the dynamic fields in your application you had to manually perform the correlation process. Here is a blog post that goes into detail about the manual process: http://blogs.msdn.com/slumley/pages/how-to-debug-a-web-test.aspx Also there are cases that our correlation process does not find the dynamic values, such as dynamic values in the URL.

At a high level, you have to:

1) Determine what parameters are dynamic

2) Then for each parameter find the first occurrence of this in a response body.

3) Add an extraction rule to pull the value out of the response and add it to the context

4) Then modify each query string or form post parameter that needs this value by changing the value to pull the value out of the context.

This new feature allows you to write your own plug-in which can perform correlation or modify the web test in many ways prior to it being saved back to the web test editor. So once you figure out that certain dynamic variable have to be correlated for each of your recordings, you can automate the process. To demonstrate how this works, I am going to write a recorder plug-in which will perform the correlation that I manually walked through in my previous post. Please quickly read that: http://blogs.msdn.com/slumley/pages/vs-2010-feature-web-test-playback-enhancements.aspx

Overview

Create the plug-in

Recorder plug-ins follow the same pattern as WebTestPlugins or WebTestRequestPlugins. To create a plug-in, you will create a class that extends WebTestRecorderPlugin and then override the PostWebTestRecording method:

public class Class1 : WebTestRecorderPlugin

{

public override void PostWebTestRecording(object sender, PostWebTestRecordingEventArgs e)

{

base.PostWebTestRecording(sender, e);

}

Modify the web test

The event args will give you 2 main objects to work with: the recorded result and the recorded web test. This will allow you to iterate through the result looking for certain values and then jump to the same request in the web test to make modifications. You can also just modify the web test if you wanted to add a context parameter or maybe parameterize parts of the URL. If you do modify the web test, you also need to set the ReocrdedWebTestModified property to true. e.RecordedWebTestModified = true;

Deploy the plug-in

After compiling the plug-in, you will need to place the dll in 1 of 2 spots:

1) Program Files\Microsoft Visual Studio 10.0\Common7\IDE\PrivateAssemblies\WebTestRecorderPlugins

2) %USERPROFILE%\My Documents\Visual Studio 10\WebTestRecorderPlugins

Executing the plug-in

After you deploy the plug-in, you will need to restart VS for the plug-in to be picked up. Now when you create a web test, you will see a new dialog. The dialog will display all of the available plug-ins that can be executed. Select your plug-in and hit ok. Once you are done recording your web test, the plug-in will be executed.

Creating the Sample Plug-in

First a quick review of the correlation that we are going to automate. Here is the screen shot from correlation tool after I recorded my web test against a reporting services site.

We are going to correlate the ReportSession parameter.

1) Create a class library project

2) Right click references and select Add Reference

3) Choose Microsoft.VisualStudio.QualityTools.WebTestFramework

4) Here is the code for my plug-in:

using System.ComponentModel;

using Microsoft.VisualStudio.TestTools.WebTesting;

using Microsoft.VisualStudio.TestTools.WebTesting.Rules;

namespace RecorderPlugins

{

[DisplayName("Correlate ReportSession")]

[Description("Adds extraction rule for Report Session and binds this to querystring parameters that use ReportSession")]

public class CorrelateSessionId : WebTestRecorderPlugin

{

public override void PostWebTestRecording(object sender, PostWebTestRecordingEventArgs e)

{

//first find the session id

bool foundId = false;

foreach (WebTestResultUnit unit in e.RecordedWebTestResult.Children)

{

WebTestResultPage page = unit as WebTestResultPage;

if (page != null)

{

if (!foundId)

{

int indexOfReportSession = page.RequestResult.Response.BodyString.IndexOf("ReportSession");

if (indexOfReportSession > -1)

{

//add an extraction rule to this request

// Get the corresponding request in the Declarative Web test

ExtractionRuleReference ruleReference = new ExtractionRuleReference();

ruleReference.Type = typeof(ExtractText);

ruleReference.ContextParameterName = "SessionId";

ruleReference.Properties.Add(new PluginOrRuleProperty("EndsWith", "&ControlID="));

ruleReference.Properties.Add(new PluginOrRuleProperty("HtmlDecode", "True"));

ruleReference.Properties.Add(new PluginOrRuleProperty("IgnoreCase", "True"));

ruleReference.Properties.Add(new PluginOrRuleProperty("Index", "0"));

ruleReference.Properties.Add(new PluginOrRuleProperty("Required", "True"));

ruleReference.Properties.Add(new PluginOrRuleProperty("StartsWith", "ReportSession="));

ruleReference.Properties.Add(new PluginOrRuleProperty("UseRegularExpression", "False"));

WebTestRequest requestInWebTest = e.RecordedWebTest.GetItem(page.DeclarativeWebTestItemId) as WebTestRequest;

if (requestInWebTest != null)

{

requestInWebTest.ExtractionRuleReferences.Add(ruleReference);

e.RecordedWebTestModified = true;

}

foundId = true;

}

else

{

//now update query string parameters

WebTestRequest requestInWebTest = e.RecordedWebTest.GetItem(page.DeclarativeWebTestItemId) as WebTestRequest;

if (requestInWebTest != null)

{

foreach (QueryStringParameter param in requestInWebTest.QueryStringParameters)

{

if (param.Name.Equals("ReportSession"))

{

param.Value = "{{SessionId}}";

}

5) Let’s review parts of the class.

a. Iterate through the result to find first page with ReportSession. This code fragment iterates through each of the recorded objects and searches the response body for ReportSession.