Applies to
· Microsoft® Visual Studio® Team System
· Performance Testing
· Web Testing
Summary
This article explains how to create a web test plug-in to automate scenarios where the numbers of form post parameters for a web request are dynamic in nature. The web test plug-in is a custom extraction rule which extracts a required set of values and stores it in an array. The plug-in is a library (DLL file) that can be reused by adding a reference in your web test project.
Contents
· Objectives
· Overview
· Example scenario
· Summary of steps
· Resources
Objectives:
· Learn how to create your own web test plug-in
· Learn when and how to use this web test plug-in
Overview:
Simulating virtual user load on the web application is one of the key approaches used in Performance Testing. Creating the required virtual user load requires an engineer to create scripts that automate user actions. Visual studio team system helps you create scripts with ease with the help of its record – playback feature and the dynamic correlation feature added newly in Visual Studio Team System 2008. The Dynamic correlation feature identifies the dynamic Form Post or Query String values and automatically binds them, however when the number of form post parameters for a web request change dynamically, an engineer would have to write a custom plug-in to handle such a case.
Example Scenario:
Consider a page with a drop down, a view button and a submit button. Now a user selects an item from the drop down and clicks on view. This displays a set of items on the page depending on how many items are associated with it. Each of these items is associated with a checkbox. The user needs to select all the checkbox and hit on the submit button. Below is snapshot taken from the HTTP watch tool showing the POST data on hitting the submit button. Each of these form post parameters represent a check box selected. When the user wants to select all and submit, the number of form post parameters sent to the server change with initial item selected from the drop down. Also we cannot simply create an extraction rule since we do not know what drop down item the user would select and how many values needs to be extracted.
Summary of steps:
1) Create a web test plug-in
2) Add the web test plug-in
Step 1: Create a web test plug-in
a) Create a new project and choose a class library template.
b) Add a reference to Microsoft.VisualStudio.QualityTools.WebTestFramework.dll
c) Create a class that inherits the ExtractionRule class. The ExtractionRule class lies in the Microsoft.VisualStudio.TestTools.WebTesting namespace. Make use of this namespace in your class library.
d) In the above example shown, each of these form post parameters follow a particular pattern hence a regular expression can be used.
e) Create a string private property. This would be used to set and get the user input in the form of a regular expression.
f) Override the Extract method. The code in this method does the core functionality of the custom extraction rule. In this method content from response body is extracted based on the regular expression and stored in a list. This list is then added to the web test context.
g) Build this project and a DLL would be created.
The final code of the plug-in should look something like this
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using Microsoft.VisualStudio.TestTools.WebTesting;
using System.Text.RegularExpressions;
namespace ExtractRegex
{
public class ExtractRegex : ExtractionRule
{
private string regularexpression;
public string MyRegularExpression
{
get
{
return this.regularexpression;
}
set
{
this.regularexpression = value;
}
public override void Extract(object sender, ExtractionEventArgs e)
{
List<String> lst = new List<String>();
Regex rg = new Regex(MyRegularExpression);
MatchCollection mcoll=rg.Matches(e.Response.BodyString, 0);
for (int i = 0; i < mcoll.Count; i++)
{
lst.Add(mcoll[i].ToString());
}
e.WebTest.Context.Add(this.ContextParameterName, lst);
}
}
}
Step 2: Add the web test plug-in to the recorded web test
a) Create a web test for a given user scenario by using the record and playback functionality of Visual Studio Team System.
b) Add a reference to the DLL created for the plug-in project.
c) Taking a look at the HTTP traffic for the user scenario through HTTP watch tool, we would see that all the form post parameters would be built by extracting certain values from the response body.
d) Choose the request whose response needs to be extracted. Right click and select add extraction rule. A new custom rule ExtractRegex will be seen as shown below.
a) Specify the Context Parameter Name and the Regular Expression to extract. In this case we shall specify “newMajorVersionGrid:_ctl[0-9]*:selectEditCheckBox” for the regular expression and “MyModules” as the context parameter name.
b) Convert your web test to a coded web test by right clicking on the web test name and selecting “generate code”
c) In the coded web test, identify the request which requires us to build the form post parameters. The extracted values are already stored in the current web test context. These values can be obtained by iterating through the web test context variable. The entire web test request will look as stated below. The highlighted code shows how the form post parameters have been built.
WebTestRequest request6 = new WebTestRequest("https://acetesting/AddNewMajorVersion.aspx");
request6.Method = "POST";
request6.Headers.Add(new WebTestRequestHeader("Cookie", "GPLocale=1033;"));
FormPostHttpBody request6Body = new FormPostHttpBody();
request6Body.FormPostParameters.Add("productLineDropDown", "24"); request6Body.FormPostParameters.Add("existingMajorVersionDropDown", "6"); request6Body.FormPostParameters.Add("newMajorVersionDropDown", "7");
foreach (string s in (List<string>)this.Context["MyModules"])
{
request6Body.FormPostParameters.Add(s, "on");
}
request6Body.FormPostParameters.Add("cloneRelationshipCheckbox", "on");
request6Body.FormPostParameters.Add("cloneAvailability", "on");
request6Body.FormPostParameters.Add("saveNewMajorVersionButton", "Save");
request6.Body = request6Body;
yield return request6;
request6 = null;
d) Run and validate your web test.
Just like the current approach makes use of regular expression as a user input, a similar procedure/methodology can be applied to create plug-ins with various input parameters such as HTMLAtrribute, HTML tags, StartsWith and EndsWith Text etc.
Before you start debugging make sure you have the symbols path set appropriately. Below link shows you how to configure symbol path
http://blogs.msdn.com/johan/archive/2007/11/13/getting-started-with-windbg-part-i.aspx
Once you have the right dump with you open the crash dump through windbg and load sos.dll using the command .load sos. Windbg by default will show the native stack and for you to look into managed stack sos.dll is required to be loaded.
The next thing I need to determine is whether the leak is in managed heap or unmanaged heap. Type in the command !eeheap -gc.
As you can see in the above snapshot, out of the total dump size of 1.2 GB, the GC heap size is 1.15 GB which clearly indicates the leak is in managed code.
Now I do a !dumpheap -stat to give the overall summary of space usage of .NET objects. This shows System.Byte[] to be the largest consumer of memory(786 MB) as shown below.
Lets try to dump some of the System.Byte[] objects using !dumpheap -type System.Byte[] to get the object addresses. Next thing we need to determine is what is causing this objects to stay alive and not get garbage collected in other words we need to find out the references of these objects. Run the !gcroot <address> command on one of the System.Byte[] objects consuming highest memory.
I checked the code and the only suspect here looked like OperationContextScope. This class implemented the idisposable interface and was instantiated several times in the code and never being disposed off. To confirm that OperationContextScope was indeed the culprit I tried to figure out the total memory usage of this object. In order to do that you need to get the methodtable address of it. Use the command !name2ee <assemblyname> <Object>. In this case I typed in
!name2ee System.ServiceModel.dll System.ServiceModel.OperationContextScope
A method table contains all the objects instances of a particular type(OperationContextScope), so the next i did was to dump the method table to get all the object addresses for OperationContextScope.
Type in !dumpheap -mt 5094bf84
I tried to find the size of some of the objects using !objsize <address> and found that they were in the range of 20-60 MB and they totalled upto 750 MB and there were many more smaller objects still alive. This confirmed that OperationContextScope object was indeed the cause of OutOfMemoryException and needs to be disposed as soon as it is done using with.
Background : SQL server 2005 introduces a new isolation level called the row versioning with Read Committed Snapshot. This setting is useful in case where there are reader-writer conflicts only. With this setting on, SQL server maintains versions of each row of data in tempdb. Readers do not acquire a shared lock on the data being read and hence helps update operations to take place without any blocking. While reading the engine retreives the most recent committed version of the row.
How to : Enabling Read Committed Isolation using row-versioning is a simple database level configuration setting. Use the below statement to turn it on
ALTER DATABASE MyDatabase SET READ_COMMITTED_SNAPSHOT ON
Case Study : In one of the recent engagements, we had a situation where several SQL deadlocks would occur under load which would eventually show up as an error in the event viewer on the web server causing the end user workflow to fail. Looking at the Deadlock graph in the SQL profiler it was found that the deadlocks were due to reader writer conflicts. We tested the application both with the default Read Committed and Read Committed Snapshot on for the same workload to see if it would eliminate deadlocks. The testing results looked pretty impressive as shown below.
| READ_COMMITTED |
READ_COMMITTED_SNAPSHOT |
| 6 deadlocks within 30 minutes |
0 deadlocks within an hour |
Also the interesting fact about this feature is that it can be enabled without effecting the application.
Tradeoffs : As always, there are certain tradeoffs of setting this option on.
1) Impacts read performance due to chains of row versions
2) Impacts update performance
3) Increases Tempdb space usage.
Consider testing your application to check for the performance impact of turning this setting on.
PS : Thanks to my collegues Jimmy May and Eddie lau for their help on this one
Background : When assemblies are authenticode signed, the signed assemblies need to be verified by the certificate authority. When CA certificate is not present on the same machine the assemblies require network or internet access. If the signed assemblies are installed on machine where CA certificate is not on the same machine and does not have network/internet access the .NET thread might timeout waiting to connect. Below is one case study I have presented. There are other ways to avoid this performance issue including performing strong name signing of assemblies, placing the CA certificate on the same machine
How to : Use the following setting in the Machine.config in the runtime tag
<configuration>
<runtime>
<generatePublisherEvidence enabled="false"/>
</runtime>
</configuration>
Case study : In a recent engagement, the site home page took ~54 seconds to load on IIS reset. As usual, looking into the resource usage pattern(CPU,IO and memory) we didn’t find anything alarming. The best solution was to get a memory dump of the worker process(w3wp.exe) during this time and analyse to see what was going on. Looking at the memory dump and analysing each thread, one thread seemed worth looking into. Looking into the thread stack given below we observed that .NET assembly’s security was being checked and call was being made to verify the certificate revocation list.
00000000`03a0b550 000007fe`faf12e58 cryptnet!CryptRetrieveObjectByUrlWithTimeout+0x263
00000000`03a0b850 000007fe`faf145fa cryptnet!CryptRetrieveObjectByUrlW+0x20c
00000000`03a0ba40 000007fe`faf1b826 cryptnet!RetrieveObjectByUrlValidForSubject+0x14a
00000000`03a0bb70 000007fe`faf141cb cryptnet!RetrieveTimeValidObjectByUrl+0x2de
00000000`03a0bc60 000007fe`faf128b0 cryptnet!CTVOAgent::GetTimeValidObjectByUrl+0x2e3
00000000`03a0bdc0 000007fe`faf124dd cryptnet!CTVOAgent::GetTimeValidObject+0x7cc
00000000`03a0bf90 000007fe`faf115a4 cryptnet!FreshestCrlFromCertGetTimeValidObject+0x61
00000000`03a0c000 000007fe`faf1b6e3 cryptnet!CryptGetTimeValidObject+0xb0
00000000`03a0c080 000007fe`faf1233d cryptnet!GetTimeValidCrl+0x4bb
00000000`03a0c1c0 000007fe`faf1201e cryptnet!GetBaseCrl+0x7d
00000000`03a0c250 000007fe`faf11e24 cryptnet!MicrosoftCertDllVerifyRevocation+0x238
00000000`03a0c3a0 000007fe`fcc06143 cryptnet!CertDllVerifyRevocation+0x28
00000000`03a0c3f0 000007fe`fcc0629c crypt32!VerifyDefaultRevocation+0x398
00000000`03a0c4e0 000007fe`fcc066bd crypt32!CertVerifyRevocation+0x144
00000000`03a0c5e0 000007fe`fcc063fa crypt32!CChainPathObject::CalculateRevocationStatus+0x48d
00000000`03a0c710 000007fe`fcbeccc8 crypt32!CChainPathObject::CalculateAdditionalStatus+0x2e2
00000000`03a0c7a0 000007fe`fcbec86b crypt32!CCertChainEngine::CreateChainContextFromPathGraph+0x443
00000000`03a0c920 000007fe`fcbebf32 crypt32!CCertChainEngine::GetChainContext+0x8b
00000000`03a0c9e0 000007fe`fb9246b8 crypt32!CertGetCertificateChain+0x100
00000000`03a0ca80 000007fe`fb92445a wintrust!_WalkChain+0x2b4
00000000`03a0cb50 000007fe`fb921e47 wintrust!WintrustCertificateTrust+0xea
00000000`03a0cbc0 000007fe`fb921057 wintrust!_VerifyTrust+0x347
00000000`03a0cde0 00000642`ffaf8031 wintrust!WinVerifyTrust+0x70
00000000`03a0ce20 00000642`7f862c5f mscorsec!GetPublisher+0x139
00000000`03a0cf40 00000642`7f57710b mscorwks!PEFile::CheckSecurity+0x40df57
On checking with the developer again we found that the server did not have access to the internet and the assemblies were authenticode signed. Since there was no internet access .NET thread would wait to connect and ultimately timeout without doing the security check. We enabled the setting as shown above. Boom!!!! The problem was fixed and the overall home page load time on iisreset came down from ~54 seconds to ~10 seconds.
Situation: The web application hosted on a server in India is performing well for end users in India. But the users in Europe are having a tough time accessing the web pages due to extremely slow response times. How would you reduce the risk considering you don’t have enough budgets to have a geographically distributed network to support your end users and keep them happy? J
Solution: Two most important things to note that helps improve perf over WAN is:
ü Minimize the overall data transfer between web server and the client.
ü Minimize the number of HTTP requests for each user action.
Here are some of my favourite tips and tricks to enable your app perform in the above mentioned manner.
ü HTTP Compression: Reduces the overall data transferred between web server and the client by compressing static/dynamic pages.
The link below shows how to enable static and dynamic compression.
http://technet.microsoft.com/en-us/magazine/cc160755(TechNet.10).aspx
Note: Enabling Compression for dynamic pages slightly increases the overall CPU utilization on the web server and this depends from app to app. Hence the application has to be thoroughly tested in order to maintain a right balance.
ü Use Pagination: Consider displaying data records on a page by page basis to reduce the data transferred for a single user action. For example, get only 10 records at a time instead of fetching everything. Also, consider fetching only the required columns for each record from the database.
ü Enable Content Expiration: This helps in reducing the unnecessary 304 round trips from client to the web server. This might be only useful for sites where static content do not change too frequently(at least a day)
So how does this work? Consider a page whose static content is already cached on the client. Whenever the browser requests that page, it checks to see if there is any change in static content on the server. If there is no change then the request returns with HTTP status code 304 and the browser takes the content directly from the cache. Hence, in order to reduce these unnecessary 304 round trips and to prevent the browser from checking for any change on the server we set content expiration on static content for a particular duration depending on the type of web application.
See the following on how to enable content expiration.
http://support.microsoft.com/kb/311006
ü Image Clustering: Consider combining smaller images into one image wherever feasible to reduce the number of HTTP request made to the server. Also several java scripts can be combined together instead of fetching them separately from the web server.
ü Enable Connection Keep-Alive: This allows the web server to keep the connection open to the client. Hence the client can download multiple files without any hassles of opening and closing the connections each time. Although this option is enabled by default in IIS 6.0 it is always better to check for it.
Hope you like this post :)
For those guys who havent heard about Loop unrolling here is a brief description about it. Loop unrolling is a way of optimizing the code by manipulating the iteration variable in the body of the loop thus reducing the number of checks each time the loop is iterated. Consider this following code.
for(int i=0;i<100;i++)
{
myfunc();
}
If you try to convert this code to assembly language you would see around half of the instructions were due to the loop overhead(i.e for conditional branches and jumps). Hence to optimize this, I would change it to
for(int i=0;i<100;)
{
myfunc();i++;
myfunc();i++;
myfunc();i++;
myfunc();i++;
}
This reduces the number of conditional branches and loops to 1/5th and hence reduces the overhead. Infact there are some compilers that perform loop unrolling. The Intel C/C++ compiler for linux gives you an option to specify at compile time the number of times the loop has to be unrolled. However the amount of loop unrolling and operations inside the loop body has to be carefully chosen. More unrolling may lead to increase in the register usage to store temporary variables.
You can try out timing the above piece of code to find out the difference using the technique specified in my earlier article
Lets start off with a simple, yet quite useful article on Timing Managed code. I will be using this in my future articles to test performance of certain pieces of code. A lot of sites mention the use of WIN32 functions QueryPerformanceCounter and QueryPerformanceFrequency. They provide timing results with nano-second accuracy. Here's a good article if you guys wanna try out timing managed code using these functions. http://msdn2.microsoft.com/en-us/library/ms979201.aspx
However, using these functions is quite inconvenient. Starting with .NET 2.0 comes the StopWatch class(also included in .NET 3.0). Since all Windows systems that support .NET also include a high-resolution performance counter, you can pretty much count on the Stopwatch being a high-resolution timer. If you’re unsure, check the static Stopwatch.IsHighResolution property, which will tell you whether the timer is based on a high-resolution performance counter.
Check out http://msdn2.microsoft.com/en-us/library/system.diagnostics.stopwatch(vs.80).aspx for information on StopWatch class. The only problem with StopWatch class is that you need to reset the timer everytime you use start and stop methods.Thats because StopWatch accumulates timing values.
Here is the code
long Frequency, Ticks, TotalTime;
Stopwatch sw = new Stopwatch();
Frequency = Stopwatch.Frequency;
sw.Start();
// your code goes here
sw.Stop();
Ticks = sw.ElapsedTicks;
TotalTime=1000000L*Ticks/Frequency;
Console.WriteLine("Total time in microseconds "+TotalTime);
Here, Frequency is returned as ticks/sec. So if you want time in nano-seconds use 10^9 instead of 10^6 in the TotalTime formula.
Also StopWatch frequency depends on the installed hardware and operating system, hence the Frequency value remains constant while the system is running.
