Alik Levin's : Test Phase

How To Generate Unit Test Using WCF Load Test – Quick Steps

alikl — Mon, 29 Jun 2009 09:13:31 GMT

This is quick summary of steps for creating WCF Unit Tests using WCF Load Test available for free on Codeplex. This quick summary created based on the Lab materials that ship with the tool.

Quick Resource Box

WCF Load Test

Summary of steps

Step 1. Configure message tracing in the app.config file on client side.
Step 2. Run WCF client to invoke the remote methods and generate trace file.
Step 3. Generate Unit test based on trace file.

Step 1. Configure message tracing in the app.config file on client side.

Select the Diagnostics folder.
Under Message Logging click Enable Message Logging.
Click Log Level and check Service messages. The other options can be unchecked.
Click ServiceModelMessageLoggingListener and change the file name to be “WCFClient.svclog”.
Under the Message Logging folder enable LogEntireMessage.
Save the configuration file.

Step 2. Run WCF client to invoke the remote methods and generate trace file.

Run your WCF client. Make sure WCFClient.svclog generated. This log file will be used in the next step to generate Unit Tests

Step 3. Generate Unit test based on trace file.

Create a new test project in Visual Studio.
Add a reference to the following assemblies:
- System.ServiceModel
- System.Runtime.Serialization (version 3.0.0.0)
In the directory containing the test project create a file called SampleConfig.xml with the following contents:

<?xml version='1.0' encoding='utf-8' ?>
<WcfUnitConfiguration xmlns='http://microsoft.com/wcfunit'
                           testMethodMode='ScenarioMethodOnly'
                           operationTimerMode='IncludeOperationTimers'>
       <assembly fileName="C:\Client\bin\Debug\ConsoleClient.exe"/>
     <soapActions soapActionMode='Include'>
     </soapActions>
</WcfUnitConfiguration>

Open an SDK command prompt and change the directory to the one containing the test project.
Execute the following command:
svcutil /o:proxy.cs /config:app.config http://localhost:8090/service?wsdl
Run the command-line tool using the following command:
“c:\program files\wcfunit\wcfunit” CompileTimeScenario <trace file> SampleConfig.xml where the <trace file> is the path to the WCFClient.svclog file created in the previous exercise.
Add generated files (CompileTimeScenario.cs and CompileTimeScenario.stubs) to the test project.
Rename CompileTimeScenario.stubs to CompileTimeScenario.Stubs.cs.
Also add the proxy.cs and app.config files located in the client project folder to the test project.
Build the solution and a Unit Test called CompileTimeScenario should appear in the Test View.

DebugView - Free Simple Tool To Quickly Identify Performance Problems

alikl — Tue, 11 Nov 2008 18:12:53 GMT

DebugView is well known free tool from Systinternals (Microsoft). Vast majority of developers is aware of its existence. Nevertheless, I've decided to write this post following my practice of journaling my technical accomplishments during engagements with the customers. Yesterday the tool helped us identify serious performance problem without installing profiler or even taking memory dumps.

Customer Case Study

The customer complained about "bad performance" of web pages. The application was pretty simple one - ASP.NET pages accessing SQL Server for simple queries. The SQL Server was installed on the same machine where ASP.NET app resided. The pages took up to ten seconds to render.

Analysis

After quick code review we found out that the code called function that traversed all server controls using recursion. This function was called several times during the page lifecycle. I presented the team with my assumption that this code is a potential culprit of the "bad performance". The opposite assumption I faced was "but there is only so few controls on the page, it cannot consume too much time."

We decided to add single line of code to the function in question:

System.Diagnostics.Trace.WriteLine ("Calling Recursive Function");

Then we fired up DebugView and ran the page.

The team was amazed by what DebugView presented interactively. "Calling Recursive Function" line kept piling up on and on. Each call took from 50 to 100 milliseconds but when multiplied to overall number of calls (~700) it gave us clear understanding what caused the performance hit.

Conclusion

Instrument your application properly - simple yet powerful technique that helps you avoid calling expensive consultants onsite.

Instrument your application and surprise your end users by high performing application.

Related Materials

This post is made with PracticeThis.com plugin for Windows Live Writer

ASP.NET Performance: Dynamically Loaded Assemblies Cause Application Recycles (Problem and Solution)

alikl — Wed, 08 Oct 2008 16:21:41 GMT

In my speak - dynamically loaded assemblies are those assemblies that were compiled during run time dynamically via CodeProvider like CSharpCodeProvider directly or by using types that use this class internally. Assemblies that are loaded dynamically using reflection via Load/LoadFrom method are out of the scope of this post.

Customer Case Study

The customer complained on periodic restarts/recycles of the ASP.NET application. We observed relative entries in the Event Log that showed up systematically on timely basis. We also observed memory utilization growth in Task Manager. After reaching 500 MB of memory utilization the application would recycle spawning another w3wp.exe to accept new incoming requests while draining the old w3wp.exe.

Analysis

After short discussion with the dev team about the design of the application we thought that the memory leak might be caused by improper usage of XmlSerializer that generate dynamic assemblies. Tess published fantastic walk through specifically dedicated to this case. Using either perfmon (.NET CLR Loading\Current Assemblies) or Process Explorer (see pic below)we observed unusual number (thousands) of loaded assemblies (notice Assemblies column). Also, notice the csc.exe in red - this is CSharp compiler that is invoked on each request:

We decided to take a memory dump to deeply investigate the case. Following are the steps that we took while analyzing the dump using WinDBG to identify the root cause:

Step 1 - Dumping memory heap to identify object allocated on heap

This is the fragment of the long list of objects. Our attention was caught by unusually large number of reflected assemblies.

!dumpheap –stat
9,019       216,456 System.Reflection.Assembly
112         4,032 System.Xml.Serialization.TempAssembly
104         5,408 System.Xml.Serialization.TypeDesc

After 3 minutes the number of dynamic assemblies is larger by more 350 assemblies (from subsequent dump):

!dumpheap –stat
9,379       225,096 System.Reflection.Assembly
114         4,104 System.Xml.Serialization.TempAssembly
102         5,304 System.Xml.Serialization.TypeDesc

Step 2 - Dumping appdomains to identify loaded assemblies

Another cross check to make sure we are dealing with tons of loaded assemblies.

!dumpdomain -stat
    Domain              Num Assemblies   Size Assemblies    Name
0x793f15d8                       1                       2,142,208        System Domain
0x793f2aa8                       56                    16,012,288      Shared Domain
0x000ab7d8                      2                       2,498,560        DefaultDomain
0x000d3368                  9,018                  55,447,040      /LM/W3SV......

Total 4 Domains, Total Size 76,100,096

Step 3 - Dumping all dynamic assemblies

How many of the assemblies are dynamic? (dda stands for dumpdynamicassemblies)

!dda

Domain: 0x793f15d8
-------------------
Domain: .
-------------------
Domain: DefaultDomain
-------------------
Domain: /LM/W3SVC/1/ROOT/......
-------------------
Assembly: 0x19058818 [RegexAssembly133_0] Dynamic Module: 0x16f4c220 loaded at: 0x0 Size: 0x0((null))
Assembly: 0x19058818 [RegexAssembly133_0] Dynamic Module: 0x190696a0 loaded at: 0x0 Size: 0x0((null))
Assembly: 0x19103ee8 [-0g5u8-v] Dynamic Module: 0x1920d6f8 loaded at: 0x19911000 Size: 0xc000((null))
Assembly: 0x190c9a40 [cvmmynwf] Dynamic Module: 0x190dc0d0 loaded at: 0x19a71000 Size: 0x4000((null))
Assembly: 0x1911bad8 [0ikhy_lx] Dynamic Module: 0x1911aa98 loaded at: 0x19f21000 Size: 0xc00((null))
.......
Assembly: 0x43199720 [nv1lvdiy] Dynamic Module: 0x431b3190 loaded at: 0x4cf61000 Size: 0xc00((null))
Assembly: 0x2d2bf008 [rk6dabem] Dynamic Module: 0x2d2bf258 loaded at: 0x4cf71000 Size: 0xc00((null))
--------------------------------------

Total 8,911 Dynamic Assemblies, Total size: 0x1d5b600(30,782,976) bytes.

Step 4 - Saving dynamic assembly to physical DLL

Save assemblies to the filesystem

!savemodule 0x2d2bc4c8 C: \0x1c344438.dll

To save all the assemblies to the file system use the following command:

!dda -save C:\MODULES

Step 5 - Using Reflector to reverse engineer the DLL:

Use Reflector to inspect the implementation/source code of the dynamic assemblies.

Step 6 - Using Reflector to find ExpressionEvaluator class

Try to locate the class in the static assemblies hopefully hitting the code that generates it:

Step 7 - Bingo! Each constructor for ExpressionEvaluator invokes compiler

ICodeCompiler compiler = new CSharpCodeProvider().CreateCompiler();
...
CompilerResults results = compiler.CompileAssemblyFromSource(options, builder.ToString());
...
this._Compiled = results.CompiledAssembly.CreateInstance("NavServices._ExpressionEvaluator");

This is actually the code that causes Process Explorer to show csc.exe under w3wp.exe (see red line in the first pic). And this is the code that caused number of loaded assemblies to grow. And this is the code that caused the application restarts.

Acknowledgements

During this investigation the following resources were used. Big THANK-YOU goes to:

Tess - .NET Memory Leak: XmlSerializing your way to a Memory Leak
Tom - Dynamic Assemblies and what to do about them
Johan - Getting started with windbg - part I
Cuko - OutOfMemory & CompiledAssembly

Related Materials

This template is made with PracticeThis.com plugin for Windows Live Writer

ASP.NET Data Binding Performance – Collection Is More Expensive Than Datatable

alikl — Fri, 01 Aug 2008 14:40:00 GMT

In my previous post - Best ASP.NET Performance Winner For Data Binding - Hands Up To Response.Write() – I’ve conducted several simple performance tests for data binding to GridView in ASP.NET page. What surprised me most is that eliminating massive loops and collection enumerations did not help in reducing CPU utilization. When we measured the execution times for both scenarios, DataBind() method revealed the secret.

Reporting Execution Times

We used System.Diagnostics.Trace to report execution times.

UseCustomCollection.aspx.cs

   1: System.Diagnostics.Trace.WriteLine("1. STARTING");

2:

   3: DataGrid datagrid = SampleServices.GenerateDynamicDataGridControl();

   4: this.Controls.Add(datagrid);

5:

   6: System.Diagnostics.Trace.WriteLine("2. GRID CREATED AND ADDED TO PAGE");

7:

   8: MyCollection myCollection = (MyCollection)SampleServices.GenerateCollection(200);

9:

  10: System.Diagnostics.Trace.WriteLine("3. CUSTOM COLLECTION CREATED");

11:

  12: datagrid.DataSource = myCollection;

  13: datagrid.DataBind();

14:

  15: System.Diagnostics.Trace.WriteLine("5. DONE");

UseDataTable.aspx.cs

   1: System.Diagnostics.Trace.WriteLine("1. STARTING");

2:

   3: DataGrid datagrid = SampleServices.GenerateDynamicDataGridControl();

   4: this.Controls.Add(datagrid);

5:

   6: System.Diagnostics.Trace.WriteLine("2. GRID CREATED AND ADDED TO PAGE");

7:

   8: MyCollection myCollection = (MyCollection)SampleServices.GenerateCollection(200);

9:

  10: System.Diagnostics.Trace.WriteLine("3. CUSTOM COLLECTION CREATED");

11:

  12: DataTable datatable = SampleServices.ConvertCollectionTableIntoDataTalbe(myCollection);

13:

  14: System.Diagnostics.Trace.WriteLine("4. CUSTOM COLLECTION CONVERTED INTO DATATABLE");

15:

  16: datagrid.DataSource = datatable;

  17: datagrid.DataBind();

18:

  19: System.Diagnostics.Trace.WriteLine("5. DONE");

Note, that UseCustomCollection.aspx.cs misses step “4. CUSTOM COLLECTION CONVERTED INTO DATATABLE”.

Collecting Execution Times

We used DebugView to collect reported execution times.

Analysis

Applying simple mathematics we can see that converting Collection to Datatable takes 0.00081015 seconds. This is the gain we get when skipping this step in UseCustomCollection.aspx.cs.

Now lets examine DataBind() in both cases:

UseCustomCollection.aspx.cs 0.00268023 seconds
UseDataTable.aspx.cs 0.00215912 seconds

I know it is not a huge improvement for binding datatable vs. collection, but the tests we conducted always showed this gap. That is why eliminating the type transformation from collection to datatable that included enumeration and looping did not really help and we ended up with similar results of ~65% CPU utilization:

UseDataTable.aspx	UseCustomCollection.aspx

Conclusion

Binding custom collection is expensive performance wise since internally it uses reflection and reflection is expensive thing to do. Looping is expensive performance wise too but it is cheaper than reflection.

Related Materials

Best ASP.NET Performance Winner For Data Binding - Hands Up To Response.Write()

alikl — Thu, 31 Jul 2008 15:29:58 GMT

To achieve best performance you need to make decisions based on trade-off between coolness, coding productivity, and personal engineering values. I never thought I would be recommending my customer considering using old fashion Response.Write() in his Internet facing ASP.NET web application in order to significantly improve the application’s performance.

Customer Case Study

During load/stress testing customer’s ASP.NET web application we identified high CPU utilization (up to 90%). After quick investigation we noticed that %Time in GC performance counter is less than optimal. Our assumption was that the application uses memory allocation techniques that are less than optimal. From GC Performance Counters:

"First thing you may want to look at is “% Time in GC”... What is a health value for this counter? It’s hard to say. It depends on what your app does. But if you are seeing a really high value (like 50% or more) then it’s a reasonable time to look at what’s going on inside of the managed heap."

Another resource we used is timeless patterns & practices’ Chapter 15 — Measuring .NET Application Performance:

.NET CLR Memory\% Time in GC

"…The most common cause of a high value is making too many allocations, which may be the case if you are allocating on a per-request basis for ASP.NET applications. You need to study the allocation profile for your application if this counter shows a higher value."

So we headed to looking into the code and this is what we found out.

Analysis

During performance code inspection we identified massive usage of collections. The collections were used to transfer the data between the logical layers and then the collections were transferred into datatables to be bindable for DataGrid (yes, it is .Net 1.1 app).

Eureka! We just spotted 3 performance anti-patterns. Massive memory allocation, massive loops, massive type conversions. I’ve shown it to 4 very respected professionals and everyone was saying the same – current situation is pure performance anti-pattern. Here are few suggestions that came up:

Bind collections directly to DataGrid eliminating additional memory allocations and loops.
Create Datatable directly from XML skipping collection creation step eliminating additional memory allocations and loops.
Use Xslt transformation transforming original Xml into Html table using Xslt elminating memory allocations and loops for both collections and datatables.
Use Response.Write() as it’s suggested by patterns & practices:

"Use the Response.Write method. It is one of the fastest ways to return output back to the browser."

Case close? Not really...

Secretly I’ve built Visual Studio 2003 project with these implementations and ran simple stress test using TinyGet utility. The results left us all a bit surprised.

Converting Collection To Datatable (Current Situation)

The code:

   1: //create custom collection

   2: MyCollection myCollection = (MyCollection)SampleServices.GenerateCollection(200);

3:

   4: //convert collection to datatable

   5: DataTable datatable = SampleServices.ConvertCollectionTableIntoDataTalbe(myCollection);

6:

   7: //bind datatalbe to dynamically created datagrid

   8: datagrid.DataSource = datatable;

   9: datagrid.DataBind();

The stress test:

tinyget.exe -srv:192.168.50.68 -uri:/dynamiccontrolsloadingrelease/UseDataTable.aspx -loop:100 -threads:15

The result:

Bind Collection Directly To Grid

The code:

   1: MyCollection myCollection = (MyCollection)SampleServices.GenerateCollection(200);

2:

   3: //bind datatalbe to dynamically created datagrid

   4: datagrid.DataSource = myCollection;

   5: datagrid.DataBind();

The stress test:

tinyget.exe -srv:192.168.50.68 -uri:/dynamiccontrolsloadingrelease/UseCustomCollection.aspx -loop:100 -threads:15

The result:

Create Datatable From Xml

The code:

   1: string xml = SampleServices.GenerateXml(200);

2:

   3: StringReader theReader = new StringReader(xml);

   4: DataSet theDataSet = new DataSet();

   5: theDataSet.ReadXml(theReader);

6:

   7: datagrid.DataSource = theDataSet.Tables[0].DefaultView;;

   8: datagrid.DataBind();

The stress test:

tinyget.exe -srv:192.168.50.68 -uri:/dynamiccontrolsloadingrelease/LoadXmlIntoDataTable.aspx -loop:100 -threads:15

The result:

Use Xslt Transformation To Create Html Table

The code:

   1: Xml1.DocumentContent = SampleServices.GenerateXml(200);

   2: Xml1.TransformSource=@"xsl.xml";

The stress test:

tinyget.exe -srv:192.168.50.68 -uri:/dynamiccontrolsloadingrelease/XmlXslTransformation.aspx -loop:100 -threads:15

The result:

Use Response.Write()

The code:

   1: MyCollection myCollection = (MyCollection)SampleServices.GenerateCollection(200);

2:

   3: // Put user code to initialize the page here

   4: Response.Write("<table>");

5:

   6: foreach(MyModelItem item in  myCollection)

   7: {

8:

   9:     Response.Write("<tr>");

  10:     Response.Write("<td>" +  item.Address  + "<td>");

  11:     Response.Write("<td>" +  item.City  + "<td>");

  12:     Response.Write("<td>" +  item.Education+ "<td>");

  13:     Response.Write("<td>" +  item.Family  + "<td>");

  14:     Response.Write("<td>" +  item.Name  + "<td>");

  15:     Response.Write("</tr>");

  16: }

  17: Response.Write("</table>");

The stress test:

tinyget.exe -srv:192.168.50.68 -uri:/dynamiccontrolsloadingrelease/ResponseWrite.aspx -loop:100 -threads:15

The result:

Sample Visual Studio 2003 Project

Interested in testing it yourself? Grab the source code from my SkyDrive here:

Conclusion

After conducting this simple test these are the conclusions I’ve made:

“Don't be afraid to challenge the pros, even in their own backyard." - How to Get Things Done - Colin Powell Version
Testing IS DA thing. Assumptions are good but nothing speaks louder than facts.
Test early - avoid massive rework afterwards. Create POC's (Proof of concept) early in architecture/design stages.
Best performance comes on expense of productivity and coolness.

Related Materials

Security Code Review – String Search Patterns For Authorization Vulnerabilities

alikl — Thu, 24 Jul 2008 22:53:10 GMT

These are the questions and the search criteria I use to identify authorization vulnerabilities in the code beyond web.config <authorization> node.

How does the code protect access to page classes?

Attributes

Search for PrincipalPermission attributes. If there is no match, the code does not perform standard authorization checks.

findstr /S /I "PrincipalPermission" *.cs

Empirical checks

Search for empirical IsInRole calls. If there is no match, the code does not perform standard authorization checks.

findstr /S /I "IsInRole" *.cs

Rolemanager

Search for empirical IsUserInRole calls for Rolemanager API. If there is no match, the code does not perform standard authorization checks.

findstr /S /I "IsUserInRole" *.cs

Does the code use Server.Transfer?

When the code uses Server.Transfer it may improve performance but potentially it may pose a threat of elevation of privileges, more info is here Performance Gain - Security Risk

findstr /S /I "Transfer" *.cs

Security Code Review – String Search Patterns For Authentication Vulnerabilities

alikl — Mon, 21 Jul 2008 15:39:59 GMT

This post contains string search patterns that can help identifying authentication vulnerabilities during security code inspection for your ASP.NET application. Most common vulnerability is about insecurely manipulating credentials in the code. The question we want to actually ask is:

Are you passing clear text credentials?

The associated threat is identity theft or identity spoof that can be achieved by disclosing the credentials or/and tampering it.

What to Search for and Why

Credentials are usually required when accessing a down stream resource – database, web service, active directory, MQSeries, or any other. This information can be easily obtained from the architecture document. Following are possible searches that can lead you to the hotspots to nail potential authentication vulnerabilities:

DB Connections

findstr /S /I ".Open( " *.cs

Web Services

findstr /S /I ".Credentials =" *.cs

LogonUser API – usually used for impersonation

findstr /S /I "LogonUser" *.cs

IIdentity usage

This one is my favorite. This search pattern is actually trying to spot the anti-pattern of identifying end user. The assumption here is that when there is no matches for that search then the solution either does not identifies the requests or uses home grown solution which might be potential vulnerability in both cases.

findstr /S /I “.Identity" *.cs

Other than above searches it is good idea to review the web.config file for potential clear text credentials.

Got more suggestions for search patters to identify potential authentication vulnerabilities? - Please, share!

Use FREE Tools From IIS Resource Kit To Warm Up Your ASP.NET 1.1 Application By Batch Compilation

alikl — Sun, 20 Jul 2008 18:24:42 GMT

Have you noticed that when ASP.NET web application is accessed for the first time the response is slow? The reason for such behavior is batch compilation that occurs on the first hit.

ASP.NET batch compilation is the process of compiling ASP.NET markup (content of aspx files) into temporary dll’s. Compilation requires invoking compiler (csc.exe for C#) – that is pretty heavy activity. Process Explorer shows it clearly:

ASP.NET batch compilation occurs on per folder basis. Said that, if your application divided into multiple sub-folders that contain ASP.NET pages each time any of the folders accessed for the first time the batch compilation is invoked.

Note that starting with ASP.NET 2.0 compilation model has changed. Also, there is a tool Aspnet_compiler.exe that allows pre-compile your ASP.NET web application to improve performance.

Customer’s case study

Customer’s web application is built with ASP.NET 1.1. It is divided into multiple subfolders reflecting logical modules that are hosted across about 20 application pools. The application connects to Oracle database.

QA team complains that the application responds slowly each time any of the modules (subfolders) accessed for the first time.

Using Process Explorer and profiler we identified three main latency points:

Creating the application pool – w3wp.exe.
Batch compiling the application for each subfolder.
Creating Oracle connection pool when Oracle is accessed for the first time.

The solution

We decided to create a Warmer – solution that will hit each subfolder’s page in unattended manner thus warming up the application before the first user hits it.

For the solution we used free tools from IIS resource kit:

LogPrser.exe to identify the URL’s of the pages to hit.
TinyGet.exe to actually hit the pages identified by LogParser.

To identify what pages to hit we took IIS log files from QA environment and than we ran the following query using LogParser:

LogParser.exe "SELECT DISTINCT STRCAT('XXX', cs-uri-stem) AS cs-uri-stem-strcat INTO 'C:\result.txt' FROM 'C:\yourIISlogFile.log' WHERE INDEX_OF(cs-uri-stem, 'aspx') > 0" -o:w3c

Notice XXX – it has nothing to do with XXX rated content rather it is a placeholder to replace it with tinyget command.

Open resulting yourIISlogFile.log file in Notepad, hit Ctrl+H for “Replace” and replace all occurrences of XXX with the following command:

tinyget -srv:www.YourServer.com -uri:

yourIISlogFile.log before the Replace:

yourIISlogFile.log after the Replace:

Remove the header and save the file with BAT extension - your Warmer is ready for action. Run it each time you deploy new version.

Do not forget to remove old temporary files in ASP.NET temporary folder:

C:\Windows\Microsoft.NET\Framework\<<NET FX VERSION>>\Temporary ASP.NET Files\

CAUTION. This action may potentially corrupt your application if you do not provide proper exception handling. On one hand it is good check to make. on other hand – be aware of it and do not do it on production sites unless you are completely sure it will not corrupt the application.

Related materials

Security Code Review – String Search Patterns For Finding Input Validation Vulnerabilities

alikl — Fri, 11 Jul 2008 14:24:53 GMT

Well defined set of search patterns helps significantly reduce time (cost) when performing security code inspections. This post focuses on input validation vulnerabilities commonly found in ASP.NET web applications.

SQL Injection and Cross Site Scripting (XSS) String search patterns

SQL Injections and XSS attacks are most common that exploit improper data access and lack of output encoding. Following are the how-to’s on finding these vulnerabilities:

Input Validation vulnerabilities String Search Patterns

To search and find security vulnerabilities you start asking questions or better yet create a list of the questions. Here is the example how - Generate Your Own Security Code Review Checklist Document Using Outlook 2007.

Use search utility similar to FindStr to perform your searches (look at Performing Text Searches). When Visual Studio is available then you can use it - Visual Studio 2005 As General Code Search Tool. Any other search tool is just fine. Following are the most common questions and search patterns.

Does the code rely on client-side validation?

If the code does not use Validators or Regex there is a potential vulnerability. Review each control how it is validated for type, length, range, string format. In the searches I assume there is no inline code and developers use code behind technique to separate markup from code.

ASP.NET pages

findstr /S /I ".Validator" *.aspx

User Controls

findstr /S /I ".Validator" *.ascx

Source code

findstr /S /I "Regex" *.cs

Is the code susceptible to canonicalization attacks?

Review that there is no external input involved in building paths and file names.

findstr /S /I “File" *.cs

findstr /S /I “Path" *.cs

Does the code validate data from all sources?

Using Cookies and QueryStrings poses a risk of the tampering threat (review STRIDE Explained to understand threats). If there is a use of Params property there is a chance for CSRF attack - Cross-Site Request Forgery Attack explained

Cookies

findstr /S /I “Cookies" *.*

Query Strings

findstr /S /I “QueryString" *.*

Params

findstr /S /I “Params" *.*

Does the code use MapPath?

If there is a usage of MapPath review that it does not use external input parameters and it is restricted to access only application file space. Make sure its third parameter set to false.

findstr /S /I “MapPath" *.*

How To Mitigate Input And Data Validation Vulnerabilities

Below are detailed step-by-step guidelines for writing code that is not vulnerable to SQL Injections and XSS attacks:

How To: Prevent Cross-Site Scripting in ASP.NET

How To: Protect From Injection Attacks in ASP.NET

How To: Protect From SQL Injection in ASP.NET

How To: Use Regular Expressions to Constrain Input in ASP.NET

Microsoft Anti-Cross Site Scripting Library V1.5

Share Your Practices

If you’ve got more search patterns to suggest – please do so! Let’s make the World [Wide Web] a more secure place together.

My Related Posts

ASP.NET Performance Sin - Serving Images Dynamically (Or Another Reason To Love Fiddler)

alikl — Fri, 02 May 2008 16:12:19 GMT

Serving images dynamically may cause performance hit. Dynamically served images require more HTTP requests which violates Steve Souders' performance rule #1 - Make Fewer HTTP Requests. The latency is also caused by parallelism (or parallel downloading) limitations as described in detail here Performance Research, Part 4: Maximizing Parallel Downloads in the Carpool Lane

Static Images

Below are the series of images that served dynamically and static.

Static images displayed using GridView's ImageFiled column type. ImageField generates the following HTML mark-up:

<img src="IMAGES/Birds/icon-penguin.gif" style="border-width:0px;" />

Browser interprets it as a static image and is ready to cache it for further reuse.

Serving Images Dynamically

Below is the sample code that implements dynamic image serving. I witness in the field different variation but the pattern (I'd call it anti-pattern) remains the same. ASP.NET and HTML mark-up that is usually part of repeater control looks similar to the following:

<img src="ServeImage.ashx?FN = 
<%#DataBinder.GetPropertyValue(Container.DataItem, "Image")%>" />

ASHX file's code that actually serves the image looks similar to this:

public void ProcessRequest(HttpContext context)
{
    string imageFileName = 
           context.Request.MapPath(@"IMAGES\" + context.Request.QueryString["FN"]);

    context.Response.ContentType = "image/jpeg";
    context.Response.WriteFile(imageFileName);
    context.Response.Flush();
    context.Response.Close();

}

Network Analysis

Using one of my most favorite tools - Fiddler - it is easy to reveal browser's view on the traffic:

There is expiration attribute attached to static images while dynamically served images do not have such attribute.

Subsequent call the the same page that gets the same images reveals the following:

All dynamically served images are not cached and utilize the network on each request.

Further investigation shows, using Fiddler's P and C fantastic feature, that overall network utilization caused by these dynamically served images is about 350 KB, which could be saved by caching the images.

Recommendations

Avoid serving images dynamically. Follow best practices outlined at Exceptional Performance:

My relative posts

Performance Sin - Chatty Database Access And Loops (Plus Another Free Performance Tool)

alikl — Mon, 28 Apr 2008 15:31:08 GMT

Chatty database access is the surefire way for slow performance caused by resources starvation that might even lead to denial of service. Following is a real world case.

Customer

Service Unavailable message is consistently observed when there are more than 150 users access the web site. We think IIS cannot handle more than 150 users. What would you suggest?

Support

Let's see what "Service Unavailable" means. "IIS cannot start any new worker processes because of limited system resources...". Let's see the code.

The code

The code was calling on DB Access function that was put inside for loop that was called inside event handler. The event handler was called on each GridVew's row creation. Meaning, DB access in nested loop. In case of small GridVew of 10 rows and 10 cells the database would be accessed 100 times for each request. 150 concurrent users would create significant load of 15,000 connections:

SQL Server Profiler

After running SQL Server profiler it became clear that database access should be significantly improved. Following is the number of SQL Commands performed as a result of single request:

Conclusion

For improved performance avoid chatty data base access. Apply caching techniques instead.

More free performance tools

If you use SQL Server 2005 Express which does not come with built-in profiler you might find useful the following free tool:

Profiler for Microsoft SQL Server 2005 Express Edition

My related posts

Free Performance Tool - Analyze IIS Logs Like A Pro With Funnel Web Analyzer

alikl — Mon, 21 Apr 2008 13:16:00 GMT

These free performance tools will save you time and money identifying performance bottlenecks. Your customers will thank you for building fast and responsive applications.

Funnel Web Analyzer 5.0 for Windows

Download the tool here.

Analyzer gives insight into everything from server load and customer usage to intranet analysis. It allows you to gain vital feedback on visitor behaviour and preferences, so you can more accurately customize your site to meet the needs of your clients

How- to use

After you download and install the tool - run it. Just drag and drop log files onto it and then click on View icon:

You will be presented with nice array of summary and detailed reports:

Usage Scenarios

There are few scenarios I can see this tool would be very useful:

Developer analyzes the module she developed during development phase just before submitting for holistic performance testing.
IT support personnel probes production applications to get a quick view on potential performance hotspots.
Performance team analyzes the data after running load/stress tests.

My related posts

Quickly Find And Fix Cross Site Scripting (XSS) Vulnerabilities In Your ASP.NET Application.

alikl — Mon, 17 Mar 2008 15:56:00 GMT

Want to quickly check your ASP.NET Web application for Cross Site Scripting (XSS) vulnerability?

It is pretty easy with the knowledge and tools you already have. This post describes how to quickly find and fix most of XSS vulnerabilities in your code.

Why XSS vulnerabilities are possible

XSS vulnerabilities are possible when un-sanitized data printed out on the page. From what I witness when I do security code inspections most cases can be summarized to two most common:

Using DataBinder.Eval function:

<%#DataBinder.Eval(Container.DataItem, "TEXT") %>

Assigning to Text property of the control:

Label1.Text = TextBox1.Text;

[Update 20.7.08] Assigning to Text property of the control:

<%=myStringGoesHere...

How to quickly find XSS vulnerabilities

Above patterns are easily identifiable using any strings search utility. I use Visual Studio 2005 As General Code Search Tool to find such vulnerabilities. When Visual Studio is not an option, just use FindStr, here is an example - Code Inspection - First Look For What To Look For.

Run your search for ".Eval(" and then for ".Text =". You might want to modify slightly it as some folks omit space before "=" or other minor changes.

Use searches similar to these:

findstr /S /I ".Text =" *.cs
findstr /S /I ".Eval(" *.aspx
findstr /S /I ".Eval(" *.ascx
[Update 20.7.08] findstr /S /I "<%=" *.aspx

Ran your search yet? What do you see? Scared?

How to quickly fix XSS vulnerabilities

The fix is pretty simple - just apply Html Encoding to both cases. The best is using freely available Microsoft Anti-Cross Site Scripting Library V1.5. Note that ASP.NET’s Server.HtmlEncode is not the safest one as it only encodes <,>,",& characters which is not sufficient to protect against all possible attacks.

My related posts

Stress Test ASP.NET Web Application With Free WCAT Tool

alikl — Sun, 09 Mar 2008 10:59:00 GMT

Building ASP.NET web applications? Plan to serve thousands of users? Would you like to see how your application would behave [misbehave] under stress?

Use simple-to-use and freely available WCAT tool to generate the load and get detailed report for expected throughput (requests/sec) and other important performance-wise information.

Summary of steps

Install WCAT
Create configuration files
Run the test
Examine results

Next section describes each step in details. Note, this post is a how-to, a jump start - not the guidelines or best practices of how to use the tool.

Install WCAT

Download and install Internet Information Services (IIS) 6.0 Resource Kit Tools. For the purpose of our exercise there is no need to install all the tools included with the resources kit, only WCAT.

Create configuration files

There are three textual files one needs to create and configure (you can give any name and extension of your choice):

script.txt - this file defines the requests, in other words what pages to request and how to request it. Following is an example of simple script.txt file:

NEW TRANSACTION
    classId = 1
    NEW REQUEST HTTP
        Verb = "GET"
        URL = "http://localhost/BankingShmanking/Default.aspx"

distribution.txt - defines weights among different requests. For example, if I need to generate request to page1.aspx twice as to page2.aspx, I will define it in this file. In case of loading only one page, the file is meaningless. Following is an example of simple distribution.txt file (1 refers to classId in script.txt file, and 50 is that 50% of the load should got this file which is meaningless since there is only one page to request, thus it will get the whole 100% load):

1 50

config.txt - determines the duration of the test, number of clients that will generate the requests against the web application. Following is the example of simple config.txt file:

Warmuptime 5s
Duration 30s
CooldownTime 5s
NumClientMachines 1
NumClientThreads 20

Save the files in "C:\Program Files\IIS Resources\WCAT Controller" folder.

Run the test

To run the stress test open command prompt by opening Run window(Windows + R) type cmd and hit Enter. Change current directory to "C:\Program Files\IIS Resources\WCAT Controller>" and run the following command to test the page hosted on the localhost:

wcctl -c config.txt -d distribution.txt -s script.txt -a localhost

then open second command prompt, change current folder to "C:\Program Files\IIS Resources\WCAT Client" and run the following command to actually launch the virtual client's requests from local machine:

wcclient.exe localhost

Examine results

The results are displayed interactively in the command line windows

The tool also generates log file that includes logged metrics - look for it in "C:\Program Files\IIS Resources\WCAT Controller" folder.

WCAT tool is actively developed by IIS team and recently they released new version of the tool - WCAT 6.3, download it from here, free.

Related materials

This post was mainly based on this article - IIS 7.0 Output Caching
WCAT: Easy, Magical, Stress Testing for IIS Web Applications
WCat 6.3 (x86)

My related posts

Performance Sin - Using Exceptions To Control Flow

alikl — Sat, 02 Feb 2008 16:19:50 GMT

Want to spot coding anti-patterns from performance perspective without actually looking in the code?

One of the common performance coding anti-patterns I’ve noticed lately is using Exception Handling to control program flow.

The anti-patterns

Most common anti-pattern is just using exception handling to control flow, in some cases it was even nested exception handling – that means the exception is thrown anyway.

In other cases there were empty “catch” exception statements. That means that precious cycles .Net consumes to handle the exception spent for nothing.

The last case was where exception handling was done to catch simple types parsing. That was done on each request.

How to identify Exception Handling anti-pattern

To identify exception handling anti-pattern set “.NET CLR Exceptions/# of Excepts Thrown” perf counter. If you see the graph constantly climbs on each request chance are Exception Handling is used to control the flow which is performance anti-pattern:

Look at the relevant source code to spot try/catch blocks. If the source code is not available use Reflector to reverse engineer the compiled assemblies into C# sources – I used it very successfully during my latest performance review.

Best practices

Do not use exception handling to control the flow. Try to reduce catching exceptions to only most upper component/class. Catching exceptions is expensive from both CPU and memory perspective.

Use TryParse method instead Parse method to avoid throwing exceptions .

Use simple "if" statement to check for nulls.

Tools

Use perfmon to spot the anti-pattern with Exception handling. Run perfmon in command line and add “.NET CLR Exceptions/# of Excepts Thrown” counter. Then run few scenarios to see the graph.

Use Practice Checker for static code analysis. The tool scans the code and reveals excessive usage of exception handling.

My related posts

Performance Code Review Tool – Practices Checker

Alik Levin's : Test Phase

How To Generate Unit Test Using WCF Load Test – Quick Steps

Summary of steps

Step 1. Configure message tracing in the app.config file on client side.

Step 2. Run WCF client to invoke the remote methods and generate trace file.

Step 3. Generate Unit test based on trace file.

DebugView - Free Simple Tool To Quickly Identify Performance Problems

Customer Case Study

Analysis

Conclusion

Related Materials

ASP.NET Performance: Dynamically Loaded Assemblies Cause Application Recycles (Problem and Solution)

Customer Case Study

Analysis

Step 1 - Dumping memory heap to identify object allocated on heap

Step 2 - Dumping appdomains to identify loaded assemblies

Step 3 - Dumping all dynamic assemblies

Step 4 - Saving dynamic assembly to physical DLL

Step 5 - Using Reflector to reverse engineer the DLL:

Step 6 - Using Reflector to find ExpressionEvaluator class

Step 7 - Bingo! Each constructor for ExpressionEvaluator invokes compiler

Acknowledgements

Related Materials

ASP.NET Data Binding Performance – Collection Is More Expensive Than Datatable

Reporting Execution Times

Collecting Execution Times

Analysis

Conclusion

Related Materials

Best ASP.NET Performance Winner For Data Binding - Hands Up To Response.Write()

Customer Case Study

Analysis

Converting Collection To Datatable (Current Situation)

Bind Collection Directly To Grid

Create Datatable From Xml

Use Xslt Transformation To Create Html Table

Use Response.Write()

Sample Visual Studio 2003 Project

Conclusion

Related Materials

Security Code Review – String Search Patterns For Authorization Vulnerabilities

Related posts

Security Code Review – String Search Patterns For Authentication Vulnerabilities

What to Search for and Why

Related posts

Use FREE Tools From IIS Resource Kit To Warm Up Your ASP.NET 1.1 Application By Batch Compilation

Customer’s case study

The solution

Related materials

Security Code Review – String Search Patterns For Finding Input Validation Vulnerabilities

SQL Injection and Cross Site Scripting (XSS) String search patterns

Input Validation vulnerabilities String Search Patterns

Does the code rely on client-side validation?

Is the code susceptible to canonicalization attacks?

Does the code validate data from all sources?

Does the code use MapPath?

How To Mitigate Input And Data Validation Vulnerabilities

Share Your Practices

My Related Posts

ASP.NET Performance Sin - Serving Images Dynamically (Or Another Reason To Love Fiddler)

Static Images

Serving Images Dynamically

Network Analysis

Recommendations

My relative posts

Performance Sin - Chatty Database Access And Loops (Plus Another Free Performance Tool)

Customer

Support

The code

SQL Server Profiler

Conclusion

More free performance tools

My related posts

Free Performance Tool - Analyze IIS Logs Like A Pro With Funnel Web Analyzer

Funnel Web Analyzer 5.0 for Windows

How- to use

Usage Scenarios

My related posts

Quickly Find And Fix Cross Site Scripting (XSS) Vulnerabilities In Your ASP.NET Application.

Why XSS vulnerabilities are possible