Alik Levin's : Planning Phase

ASP.NET Performance By Design: Takeaways From PDC

alikl — Mon, 03 Nov 2008 12:06:35 GMT

During PDC, there were 5 dedicated sessions for improving performance in .Net titled "Performance By Design". The presenters are Rico Mariani, Vance Morrison, and Mark Friedman. These guys live and breathe performance. Although I did not make to get to PDC, I was following after what's going on there. Fortunately, Vance published all slides on his blog. These are my takeaways from the slides he published.

Performance by Design Intro

By Rico Mariani

My favorite is the first slide that sets the expectations - Performance is about Culture:

Part 1 - Teaching Performance Culture
Part 2 - General Topics about Managed Code

I am so happy to see it! There is a perception about performance in the field - it is about tools. There is another perception - "we can fix performance after we build the app". I've been through few situations where such approach resulted in extra budget to "fix the performance issues", missed deadlines, and frustrated customers (and stakeholders). Rico says there are very few rules to follow:

Rule #1 - Measure
Rule #2 - Do your homework

Rico explains Performance Culture very simply but in powerful way:

Budget. An exercise to assess the value of a new feature and the cost you’d be willing to pay:
- Begin by thinking about how the customer thinks about performance
  - Responsiveness
  - Capacity
  - Throughput
  - Cost of Entry
- Identify the resource the customer views as critical to this system
- Choose the level of performance we want to deliver (do we need an “A+” or is a “D” good enough)
- Convert this into what resource usage needs to be to succeed
- Don’t think about the code, think about the customer
Plan. Validate your design against the budget, this is a risk assessment
- You can’t plan without a budget, so get one
- Use best practices to select candidate algorithms
- Understand their costs in terms of the critical resource
- Identify your dependencies and understand their costs
- Compare these projected costs against the budgets
- If you are close to budget you will need much greater detail in your plans
- Identify verification steps and places to abort if it goes badly
- Proceed when you are comfortable with the risk
Verify. Measure the final results, discard failures without remorse or penalty, don’t make us live with them
- The budget and the plan drive verification steps
- Performance that cannot be verified does not exist
- Don’t be afraid to cancel features that are not meeting their budgets – we expect to lose some bets
- Don’t inflict bad performance on the world

Bottom line - performance is about culture - tools only support it. Performance cost you either way - either you invest in it or not. Manage it as you'd manage any other risk.

My Related Posts

CPU Optimization for .NET Applications

By Vance Morrison

Vance supports Rico's Rule #1 - Measure. He provides several practical options:

Low Tech: System.Diagnostics.Stopwatch
Medium Tech: MeasureIt (Automates Stopwatch)
Medium Tech: Use ETW (Event Tracing for Windows)
Higher Tech: Sample Based Profiling.

Vance admits that ETW (Event Tracing for Windows) "...is not easy to use End-to-End:

We are working it
We will have more offerings in next release
It is a complete talk just by itself
If you need logging NOW you CAN use EventProvider, xperf
If you can wait a year, it will be significantly nicer.
If there is interest, we can have an ‘Open Space’ discussion"

My biggest takeaway from this session is that I know unforgivably too little about ETW - need to ramp up myself on it.

My Related Posts

ASP.NET Performance: High CPU Utilization Case Studies And Solutions

Memory Optimization for .NET Applications

By Vance Morrison

My favorite slide is #14:

"Fixing Memory Issues: Prevention!

Fixing Memory Issues is HARD
- Usually a DESIGN problem: Not Pay for Play
- Using every new feature in your app
  XML, LINQ, WPF, WCF, Serialization, Winforms, …
- Initialize all subsystems at startup
GC Memory Are your Data Structures
- Tend to be designed early
- Hard to change later
Thus it Pays to Think about Memory Early!"

What does it mean to think about memory early? The slide deck is packed with explanation about the measurement tools and the theory behind GC. I'd also expect to see few code samples - both patterns and anti-patterns. Mid-life crisis drill down would work for me.

My Related Posts

Parallelism for .NET Applications

By Vance Morrison

The only takeaway (beside good tips) is that Parallel Computing is getting into mainstream. From the field I see more and more demand for multithreaded work, I observe customers buying strong servers but do not utilize it to its capacity while asking to improve performance. I liked the structure of the session, especially "How .Net Can Help..." slides that offer practical tips and implementation suggestions to improve performance by parallelism.

My Related Post

ASP.NET Web Application Performance

By Mark Friedman

This is really huge slide deck - 110 slides... that covers tons of stuff.

I was looking for unusual stuff. I found it. Turns out Visual Round Trip Analyzer (VRTA) was released to the web. The tool was internal for some time and now it is available for the masses. Good news!

I also liked the slides about ETW for IIS. Especially the ETW Trace reporting tool , which is Excel ;). One statement made me feel alert - "Caching the same data in multiple places tends to be wasteful". Not the statement itself rather the relation to Velocity, which is MS distributed cache mechanism. Need to dig deeper. Overall, the slide deck is packed with very useful and practical recommendations spanning multiple technologies like ASP.NET, AJAX, and WCF.

My Related Posts

This template is made with PracticeThis.com plugin for Windows Live Writer

ASP.NET Data Binding Performance – Collection Is More Expensive Than Datatable

alikl — Fri, 01 Aug 2008 14:40:00 GMT

In my previous post - Best ASP.NET Performance Winner For Data Binding - Hands Up To Response.Write() – I’ve conducted several simple performance tests for data binding to GridView in ASP.NET page. What surprised me most is that eliminating massive loops and collection enumerations did not help in reducing CPU utilization. When we measured the execution times for both scenarios, DataBind() method revealed the secret.

Reporting Execution Times

We used System.Diagnostics.Trace to report execution times.

UseCustomCollection.aspx.cs

   1: System.Diagnostics.Trace.WriteLine("1. STARTING");

2:

   3: DataGrid datagrid = SampleServices.GenerateDynamicDataGridControl();

   4: this.Controls.Add(datagrid);

5:

   6: System.Diagnostics.Trace.WriteLine("2. GRID CREATED AND ADDED TO PAGE");

7:

   8: MyCollection myCollection = (MyCollection)SampleServices.GenerateCollection(200);

9:

  10: System.Diagnostics.Trace.WriteLine("3. CUSTOM COLLECTION CREATED");

11:

  12: datagrid.DataSource = myCollection;

  13: datagrid.DataBind();

14:

  15: System.Diagnostics.Trace.WriteLine("5. DONE");

UseDataTable.aspx.cs

   1: System.Diagnostics.Trace.WriteLine("1. STARTING");

2:

   3: DataGrid datagrid = SampleServices.GenerateDynamicDataGridControl();

   4: this.Controls.Add(datagrid);

5:

   6: System.Diagnostics.Trace.WriteLine("2. GRID CREATED AND ADDED TO PAGE");

7:

   8: MyCollection myCollection = (MyCollection)SampleServices.GenerateCollection(200);

9:

  10: System.Diagnostics.Trace.WriteLine("3. CUSTOM COLLECTION CREATED");

11:

  12: DataTable datatable = SampleServices.ConvertCollectionTableIntoDataTalbe(myCollection);

13:

  14: System.Diagnostics.Trace.WriteLine("4. CUSTOM COLLECTION CONVERTED INTO DATATABLE");

15:

  16: datagrid.DataSource = datatable;

  17: datagrid.DataBind();

18:

  19: System.Diagnostics.Trace.WriteLine("5. DONE");

Note, that UseCustomCollection.aspx.cs misses step “4. CUSTOM COLLECTION CONVERTED INTO DATATABLE”.

Collecting Execution Times

We used DebugView to collect reported execution times.

Analysis

Applying simple mathematics we can see that converting Collection to Datatable takes 0.00081015 seconds. This is the gain we get when skipping this step in UseCustomCollection.aspx.cs.

Now lets examine DataBind() in both cases:

UseCustomCollection.aspx.cs 0.00268023 seconds
UseDataTable.aspx.cs 0.00215912 seconds

I know it is not a huge improvement for binding datatable vs. collection, but the tests we conducted always showed this gap. That is why eliminating the type transformation from collection to datatable that included enumeration and looping did not really help and we ended up with similar results of ~65% CPU utilization:

UseDataTable.aspx	UseCustomCollection.aspx

Conclusion

Binding custom collection is expensive performance wise since internally it uses reflection and reflection is expensive thing to do. Looping is expensive performance wise too but it is cheaper than reflection.

Related Materials

Best ASP.NET Performance Winner For Data Binding - Hands Up To Response.Write()

alikl — Thu, 31 Jul 2008 15:29:58 GMT

To achieve best performance you need to make decisions based on trade-off between coolness, coding productivity, and personal engineering values. I never thought I would be recommending my customer considering using old fashion Response.Write() in his Internet facing ASP.NET web application in order to significantly improve the application’s performance.

Customer Case Study

During load/stress testing customer’s ASP.NET web application we identified high CPU utilization (up to 90%). After quick investigation we noticed that %Time in GC performance counter is less than optimal. Our assumption was that the application uses memory allocation techniques that are less than optimal. From GC Performance Counters:

"First thing you may want to look at is “% Time in GC”... What is a health value for this counter? It’s hard to say. It depends on what your app does. But if you are seeing a really high value (like 50% or more) then it’s a reasonable time to look at what’s going on inside of the managed heap."

Another resource we used is timeless patterns & practices’ Chapter 15 — Measuring .NET Application Performance:

.NET CLR Memory\% Time in GC

"…The most common cause of a high value is making too many allocations, which may be the case if you are allocating on a per-request basis for ASP.NET applications. You need to study the allocation profile for your application if this counter shows a higher value."

So we headed to looking into the code and this is what we found out.

Analysis

During performance code inspection we identified massive usage of collections. The collections were used to transfer the data between the logical layers and then the collections were transferred into datatables to be bindable for DataGrid (yes, it is .Net 1.1 app).

Eureka! We just spotted 3 performance anti-patterns. Massive memory allocation, massive loops, massive type conversions. I’ve shown it to 4 very respected professionals and everyone was saying the same – current situation is pure performance anti-pattern. Here are few suggestions that came up:

Bind collections directly to DataGrid eliminating additional memory allocations and loops.
Create Datatable directly from XML skipping collection creation step eliminating additional memory allocations and loops.
Use Xslt transformation transforming original Xml into Html table using Xslt elminating memory allocations and loops for both collections and datatables.
Use Response.Write() as it’s suggested by patterns & practices:

"Use the Response.Write method. It is one of the fastest ways to return output back to the browser."

Case close? Not really...

Secretly I’ve built Visual Studio 2003 project with these implementations and ran simple stress test using TinyGet utility. The results left us all a bit surprised.

Converting Collection To Datatable (Current Situation)

The code:

   1: //create custom collection

   2: MyCollection myCollection = (MyCollection)SampleServices.GenerateCollection(200);

3:

   4: //convert collection to datatable

   5: DataTable datatable = SampleServices.ConvertCollectionTableIntoDataTalbe(myCollection);

6:

   7: //bind datatalbe to dynamically created datagrid

   8: datagrid.DataSource = datatable;

   9: datagrid.DataBind();

The stress test:

tinyget.exe -srv:192.168.50.68 -uri:/dynamiccontrolsloadingrelease/UseDataTable.aspx -loop:100 -threads:15

The result:

Bind Collection Directly To Grid

The code:

   1: MyCollection myCollection = (MyCollection)SampleServices.GenerateCollection(200);

2:

   3: //bind datatalbe to dynamically created datagrid

   4: datagrid.DataSource = myCollection;

   5: datagrid.DataBind();

The stress test:

tinyget.exe -srv:192.168.50.68 -uri:/dynamiccontrolsloadingrelease/UseCustomCollection.aspx -loop:100 -threads:15

The result:

Create Datatable From Xml

The code:

   1: string xml = SampleServices.GenerateXml(200);

2:

   3: StringReader theReader = new StringReader(xml);

   4: DataSet theDataSet = new DataSet();

   5: theDataSet.ReadXml(theReader);

6:

   7: datagrid.DataSource = theDataSet.Tables[0].DefaultView;;

   8: datagrid.DataBind();

The stress test:

tinyget.exe -srv:192.168.50.68 -uri:/dynamiccontrolsloadingrelease/LoadXmlIntoDataTable.aspx -loop:100 -threads:15

The result:

Use Xslt Transformation To Create Html Table

The code:

   1: Xml1.DocumentContent = SampleServices.GenerateXml(200);

   2: Xml1.TransformSource=@"xsl.xml";

The stress test:

tinyget.exe -srv:192.168.50.68 -uri:/dynamiccontrolsloadingrelease/XmlXslTransformation.aspx -loop:100 -threads:15

The result:

Use Response.Write()

The code:

   1: MyCollection myCollection = (MyCollection)SampleServices.GenerateCollection(200);

2:

   3: // Put user code to initialize the page here

   4: Response.Write("<table>");

5:

   6: foreach(MyModelItem item in  myCollection)

   7: {

8:

   9:     Response.Write("<tr>");

  10:     Response.Write("<td>" +  item.Address  + "<td>");

  11:     Response.Write("<td>" +  item.City  + "<td>");

  12:     Response.Write("<td>" +  item.Education+ "<td>");

  13:     Response.Write("<td>" +  item.Family  + "<td>");

  14:     Response.Write("<td>" +  item.Name  + "<td>");

  15:     Response.Write("</tr>");

  16: }

  17: Response.Write("</table>");

The stress test:

tinyget.exe -srv:192.168.50.68 -uri:/dynamiccontrolsloadingrelease/ResponseWrite.aspx -loop:100 -threads:15

The result:

Sample Visual Studio 2003 Project

Interested in testing it yourself? Grab the source code from my SkyDrive here:

Conclusion

After conducting this simple test these are the conclusions I’ve made:

“Don't be afraid to challenge the pros, even in their own backyard." - How to Get Things Done - Colin Powell Version
Testing IS DA thing. Assumptions are good but nothing speaks louder than facts.
Test early - avoid massive rework afterwards. Create POC's (Proof of concept) early in architecture/design stages.
Best performance comes on expense of productivity and coolness.

Related Materials

Pitfalls With .Net Multithreading And COM Objects – Threads Must Have Compatible Apartment Models (MTA vs. STA)

alikl — Fri, 20 Jun 2008 10:09:11 GMT

Be alert when implementing multithreading .Net in conjunction with COM objects. Thread apartment models matter.

.Net threads have Multi Threaded Apartment (MTA) model by default. COM objects have Single Thread Apartment (STA). Calling on COM objects on .Net threads that you spawn may cause unpredicted result.

Multithreading in .Net is easily implemented based on either Thread or ThreadPool objects. Thread.Start() method spawns new thread which has Multi Threaded Apartment (MTA) model . ThreadPool.QueueUserWorkItem(myMethod) queues myMethod to be executed on available thread managed by it.

Thread object and COM

When spawning your own threads with Thread.Start() set Thread’s apartment model to STA if you plan calling COM object on it:

        1: Thread t = new Thread(DoWork);

       2: t.SetApartmentState(ApartmentState.STA);

       3: t.Start();

Note, spawning your own threads with Thread.Start() utilizes resources – mainly CPU, spawning too much threads may cause performance hit. Consider using ThreadPool object.

ThreadPool object and COM

All threads managed by ThreadPool objects are MTA threads. Apartment model of ThreadPool’s threads cannot be changed. Do not call COM objects on ThreadPool’s threads.

Possible designs and solutions

ASP.NET. Calling COM objects in ASP.NET pages configure AspCompat=true:

“Most COM components will work with ASP.NET…”

“…The AspCompat attribute forces the page to execute in STA mode. The runtime throws an exception if the compatibility tag is omitted and an STA component is referenced on the page.”

Client side (like WinForms)multithreading. Use ThreadPool with its MTA threads. Spawn another thread with Thread.Start() setting its apartment model to STA.
Custom server side multithreading. I’d consider using Remoting infrastructure. In case your server application should serve messages that are not covered by Remoting’s built in mechanisms I’d consider building my own Channel sink.
And the last but not the least. Do not forget to call Marshal.ReleaseComObject(obj) to dispose the COM objects created on your the threads.

Related materials

ASP.NET Performance Engineering - Stress Test Your Architecture, Design, And Code

alikl — Mon, 05 May 2008 13:31:00 GMT

Field experience proves - the earlier performance is tackled in development lifecycle the better results achieved. Below are most frequent practices that were most helpful in my engagement with the customers.

Architecture/Design phase

Set performance scope using Performance Frame
Set performance objectives - Performance Testing Objectives Document Template
Generate and distribute performance engineering principles document to the dev team (this sample was generated using Guidance Explorer)
Conduct performance design inspection (this sample document was generated using Guidance Explorer)

Coding phase

Conduct static code analysis for performance issues - Performance Code Review Tool – Practices Checker
Conduct manual performance code inspection using checklist document (this sample was generated using Guidance Explorer)
Analyze program's flow early in development - Use Sysinternals DebugView To Diagnose The Application

Unit testing phase

Conduct performance smoke test for your application - Stress Test ASP.NET Web Application With Free WCAT Tool
Analyze IIS behavior - Identify ASP.NET, Web Services, And WCF Performance Issues By Examining IIS Logs
Analyze Database behavior using SQL Server profiler - How To: Use SQL Profiler and Using SQL Server Profiler
Analyze Client Side JavaScript behavior - Profiling JavaScript With Ajax View Tool: Spot Poor Performance Client Script In No Time
Analyze ASP.NET behavior - Use Performance Counters Templates To Streamline Performance Analysis
Analyze HTTP traffic from end user's angle - Improve Web Application Performance By Reducing Number Of Http Requests - Fiddler To The Rescue

Performance Sins (performance anti-patterns)

My related posts

ASP.NET Performance Sin - Serving Images Dynamically (Or Another Reason To Love Fiddler)

alikl — Fri, 02 May 2008 16:12:19 GMT

Serving images dynamically may cause performance hit. Dynamically served images require more HTTP requests which violates Steve Souders' performance rule #1 - Make Fewer HTTP Requests. The latency is also caused by parallelism (or parallel downloading) limitations as described in detail here Performance Research, Part 4: Maximizing Parallel Downloads in the Carpool Lane

Static Images

Below are the series of images that served dynamically and static.

Static images displayed using GridView's ImageFiled column type. ImageField generates the following HTML mark-up:

<img src="IMAGES/Birds/icon-penguin.gif" style="border-width:0px;" />

Browser interprets it as a static image and is ready to cache it for further reuse.

Serving Images Dynamically

Below is the sample code that implements dynamic image serving. I witness in the field different variation but the pattern (I'd call it anti-pattern) remains the same. ASP.NET and HTML mark-up that is usually part of repeater control looks similar to the following:

<img src="ServeImage.ashx?FN = 
<%#DataBinder.GetPropertyValue(Container.DataItem, "Image")%>" />

ASHX file's code that actually serves the image looks similar to this:

public void ProcessRequest(HttpContext context)
{
    string imageFileName = 
           context.Request.MapPath(@"IMAGES\" + context.Request.QueryString["FN"]);

    context.Response.ContentType = "image/jpeg";
    context.Response.WriteFile(imageFileName);
    context.Response.Flush();
    context.Response.Close();

}

Network Analysis

Using one of my most favorite tools - Fiddler - it is easy to reveal browser's view on the traffic:

There is expiration attribute attached to static images while dynamically served images do not have such attribute.

Subsequent call the the same page that gets the same images reveals the following:

All dynamically served images are not cached and utilize the network on each request.

Further investigation shows, using Fiddler's P and C fantastic feature, that overall network utilization caused by these dynamically served images is about 350 KB, which could be saved by caching the images.

Recommendations

Avoid serving images dynamically. Follow best practices outlined at Exceptional Performance:

My relative posts

Securing IIS7 - Windows Server 2008 Security Guide

alikl — Thu, 28 Feb 2008 22:09:42 GMT

Windows Server 2008 Security Guide is out.

It covers many crucial aspects but my favorite of course is IIS7 chapter:

Chapter 6: Hardening Web Services

This chapter provides prescriptive guidance for hardening the Web Server role. The chapter discusses how the Web server role installs Microsoft® Internet Information Services (IIS) 7.0, which has been redesigned into forty modular components that you can choose to install as needed.

It points to the following resources:

How To Consume WCF Using AJAX Without ASP.NET

alikl — Mon, 18 Feb 2008 23:35:21 GMT

How to consume WCF services directly from Html client? How to add AJAX-like functionally to application that does not natively support ASP.NET AJAX like classic ASP, ASP.NET 1.1, or PHP?

WCF that ships with .Net 3.5 provides capability to consume it from any JavaScript enabled client via XML or JSON encoding. There is new built in webHttpBinding that supports either JSON or XML encoded messages to be sent to WCF services.The functionality can dramatically improve performance and user experience.

This post summarizes the steps to create and consume basic WCF service using webHttpBinding binding.

Summary of Steps

Step 1 – Create WCF service.
Step 2 – Configure WCF end point.
Step 3 – Create JavaScript to invoke WCF service.
Step 4 – Test the solution.

Next section describes each and every step in details

Step 1 – Create WCF service

Open Visual Studio and create new WCF service project by choosing "WCF Service Application" template under "Web" project type. Name it Wcf2Ajax. Open IService1.cs file and create OperationContract as follows:

        1: [ServiceContract]

       2:  public interface IService1

       3:  {

       4:  

       5:      [OperationContract]

       6:      string Sum2Integers(int n1, int n2);

Open Service1.svc.cs file and add public method that accepts to integers and returns the sum of it. This is the functionality that will be exposed by the WCF service and consumed by JavaScript enabled client:

       1: public class Service1 : IService1

       2: {

       3:     public string Sum2Integers(int n1, int n2)

       4:     {

       5:         int result = num1 + num2;

       6:  

       7:         return result.ToString();

       8:     }

Step 2 – Configure WCF end point

Open web.config file and add <binding> section to <system.serviceModel section. Add <webHttpBinding> to binding section:

<system.serviceModel>
  <bindings>
    <webHttpBinding>
      <binding name="AjaxBinding"/>
    </webHttpBinding>
  </bindings

Add AjaxBehavior to <behaviors> section to support WCF invocation via AJAX:

<behaviors>
  <endpointBehaviors>
    <behavior name="AjaxBehavior">
      <enableWebScript/>
    </behavior>

Configure WCF service's endpoint to use newly created binding:

<endpoint address="ajaxEndpoint" 
          behaviorConfiguration="AjaxBehavior" 
          binding="webHttpBinding" 
          bindingConfiguration="AjaxBinding" 
          contract="Wcf2Ajax.IService1">

Step 3 – Create JavaScript to invoke WCF service

Add Html file to the solution by right clicking on the solution node in solution explorer and choosing "New Item..." and then "HTML Page" template. Name it WCFConsumer.htm. Add few HTML controls - two text boxes to accept two integers, one pure HTML button to trigger WCF call, and <span> element to present the result:

<input type="text" id="num1" />
<input type="text" id="num2" />
<br>
<input type="button" onclick="CallWcfAjax()" value="Call WCF via AJAX" />
Result <span id="result"></span>

Add <script> block to <header> section and add JavaSctip that builds HTTP request and wires invocation function to some event, say button click:

    <script type="text/javascript">
    function CallWcfAjax()
    {
        var xmlHttp = new ActiveXObject("Microsoft.XmlHttp");

        var url = "Service1.svc/ajaxEndpoint/";
        url = url + "Sum2Integers";

        var body = '{"n1":';
        body = body + document.getElementById("num1").value + ',"n2":';
        body = body + document.getElementById("num2").value + '}';
          
        // Send the HTTP request
        xmlHttp.open("POST", url, true);
        xmlHttp.setRequestHeader("Content-type", "application/json");
        xmlHttp.send(body);

        // Create result handler 
        xmlHttp.onreadystatechange= function X()
        {
        
             if(xmlHttp.readyState == 4)
             {
                  result.innerText = xmlHttp.responseText;
             }
        }
    }
    </script>

Step 4 – Test the solution

Build the solution and navigate to WCFConsumer.htm. Provide two integers to both text boxes and hit "Call WCF via Ajax" button. You should expect for result similar as depicted below:

Parse the result to your needs.

My related post

Related resources

Creating WCF AJAX Services without ASP.NET

Download Visual Studio 2008 project with the sample from my SkyDrive

Design For Operations [DFO] – Problems And Solution Frame

alikl — Sun, 20 Jan 2008 20:02:06 GMT

patterns & practices team maintains Design for Operations [DFO] project on codeplex. The goal of the project focuses on:

“Developing tools and guidance to help enable the development of highly manageable applications on the Windows platform.”

This post summarize my understanding of the project’s problems and solutions frame. Most of the content is direct copy paste from more than 300 pages Manageability Guidance document found here and few interpretations of mine.

Problems Frame

Active players and their concerns

End User.
- Why it is not working?
- Why it works so slow?
- Why I am not allowed to do this operation?
Operator.
- How do I configure this?
- Why it failed without alerts?
- Where all alerts are sent?
- How do I roll back the version?
- What should I do when I se this alert?
Developer.
- How come end users do not understand exception message? – it is simple call stuck dump!
- What do I do with this “Unspecified error” thing?
- What component throws this exception?
- Here is the patch – just drop it to fix the problem.

Operations Challenges

How do I know what is the source of the incident? For example, “It is IIS authentication”.
How do I get detailed information regarding the incident? For example, “SPN is not configured for IIS Application account”.
How do I recognizes the trends that usually lead to incident? “Yesterday we had 10% CPU utilization and today it is 20% - it must mean something”.

Solution Frame

Representing Applications as Managed Entities

A managed entity is any logical part of an application that a system administrator needs to configure, monitor, and create reports about while managing that application or service. Examples of managed entities are a Web service, a database, an Exchange routing group, an Active Directory site, a computer, a server role, a network device, a hardware component, or a subnet.

Model Comprehensive Management Models

Creating a comprehensive management model consists of modeling in a variety of different areas to provide a total system view, including the following:

Configuration modeling. This involves encapsulating all the settings that control the behavior or functionality of an application or system component.
Task modeling. This involves cataloging the complete list of tasks that administrators have to perform to administer and manage a software system or application.
Instrumentation modeling. This involves capturing the instrumentation used to record the operations of a system or application. Instrumentation provides information to the operations team to increase understanding about how the application functions, and to diagnose problems with an application.
Health modeling. This involves defining what it means for a system or application to be healthy (operating normally) or unhealthy (operating in a degraded condition or not working at all). A health model represents logically the parts of an application or service the operations team is responsible for keeping operational.
Performance modeling. This involves capturing the expected baseline performance of an application. Performance counters can then be used to report and expose performance on an ongoing basis, and a monitoring tool can compare this performance to the expected performance.

Building Effective Health Models

An application is considered healthy if it is operating within a series of defined parameters. A number of factors may result in a change in application health, including the following:

Change in application configuration
An application update
A change in an external dependency
A hardware change
A network change
Bad input to the application
Scalability problems
Operator error
Change in deployment
Malicious attack

Steps to handle the problem

Detect a problem.
Verify that the problem still exists.
Diagnose the cause(s) of the problem.
Resolve the problem.
Verify that the problem was resolved.
[My addition] Log the incident and convert it into Knowledge Base gem.

Conclusion

There are few key terms mentioned above - "Modeling", "Design", "Building", "Maintain", "Testing". To me it is absolutely clear that Design For Operations is no different from Security Development Lifecycle or Performance Development Lifecycle. "Operations" is just another important non-functional requirement that needs to be taken throughout the whole development lifecycle to be successfully implemented and deployed in production. It had to be called Operations Development Lifecycle.

My related posts

Chain Of Responsibility Design Pattern – Focus On Security, Performance, And Operations

alikl — Mon, 14 Jan 2008 18:50:19 GMT

The pattern is also called Intercepting Filter, Pipeline, AOP, and may be few more… I am confused by the name for this design pattern.

“Life is really simple, but we insist on making it complicated.” - Confucius

No matter how they call it I like the idea of decoupling actions while processing one after another. Here is the definition from data & object factory:

Avoid coupling the sender of a request to its receiver by giving more than one object a chance to handle the request. Chain the receiving objects and pass the request along the chain until an object handles it.

Intercepting Filter visual from MSDN’s Discover the Design Patterns You're Already Using in the .NET Framework

.Net framework implements Chain Of Responsibility design pattern for many its internal mechanisms. My favorite is HttpModule. I like it so much I decided to build my own pipeline.

This post summarizes my steps I took to create my own simple implementation for this design pattern. I did not do any research purposely online and wanted to get my hands dirty without bias.

The goal

My goal was creating simple code that can be adopted and extended:

The code should be very simple
The code should provide pipeline infrastructure to invoke aspects/filters
The code should allow dynamic invocation of any arbitrary number of aspects
The code should include no optimization to stay simple
The code should demonstrate practical idea and not be state of the art, production ready to go one.

What I needed

Provide generic mechanism that will execute arbitrary logic – the Pipeline.
The logic is encapsulated in decoupled components – Aspects.
Aspects can be independently configured without rebuilding the application.

The design

Summary of steps

Step #1 – Create base classes and interfaces.
Step #2 – Create concrete implementation of specific Pipeline.
Step #3 – Modify config file.
Step #4 – Test the solution.

Following section describes each step in details

Step #1 – Create base classes and interfaces. I needed two bases classes - BasePipeline and IAspect. BasePipeline is responsible to perform generic actions of:

Consulting configuration file.
Loading the configured aspects.
And invoking them one by one.

private void LoadPipelineAspects()
{
    aspects = new List<IAspect>();

    Configuration config =

                  ConfigurationManager.OpenExeConfiguration(

                  ConfigurationUserLevel.None);

    string  piplineConfig =

            config.AppSettings.Settings[pipelineName].Value;

    string[] aspectNames = piplineConfig.Split(',');

    for (int i = 0; i < aspectNames.Length; i++)
    {

        Type t = Type.GetType(aspectNames[i]);

        IAspect aspect=(IAspect)Activator.CreateInstance(t);

        aspects.Add(aspect);

    }
}

IAspect interface serves as a contract between BasePipline and aspect's concrete implementation. Take a look at the design diagram. When the client invokes HandleRequest method on concrete Pipeline implementation it invokes underneath generic implementation of the method of its base - BasePipeline:

public virtual bool HandleRequest(object request) 
{

    foreach (IAspect item in aspects)
    {
        item.ProcessRequest(request);
    }
    return true;
}

Step #2 – Create concrete implementations. Concrete implementations of both types SimplePipeline and SimpleAspect are nothing fancy. SimplePipeline calls into its base but reserves the right to add any additions without interfering with any other concrete implementations:
```
public override bool HandleRequest(object request)
{
    base.HandleRequest(request);

    return true;
} 
```
To indicate what aspect is running I simply spit out aspect's name for aspect's ProcessRequest. That is the place where the whole logic should be performed - DB access, Web Services calls, request modifications etc :
```
public bool ProcessRequest(object request)
{
    Console.WriteLine("processing from SimpleAspect1");
    return true;
}
```

Step #3 – Modify config file. Config file - app.config or web.config in my case - holds the information about what aspects should be called for specific pipeline:
```
<appSettings>
 <add key =
"MyPipeline.SimplePipeline" 
 value ="MyPipeline.SimpleAspect1,MyPipeline.SimpleAspect2"/>
</appSettings>
```
The key is the type of the pipeline and the value is comma separated types of the aspects. When the concrete pipeline is instantiated by the client it already knows it type which serves to consult the config file and identify what aspects to load using reflection. This is how BasePipeline constructor looks like:
```
public BasePipeline()
{
    pipelineName = this.ToString();
    
    LoadPipelineAspects();

}
```
See Step #1 for LoadPipelineAspects() function implementation where reflection magic happens to dynamically load the aspects.

Step #4 – Test the solution. To test the solution follow these steps:
- Create Class library project and reference to the library where Pipeline base classes reside. Implement your own Pipeline while inheriting from BasePipeline.
- Implement few aspects that implement IAspect interface.
- Create simple Windows Console application, add reference to the libraries and add simple code similar to this to Main function:
```
MyPipeline.SimplePipeline sp = 
                        new MyPipeline.SimplePipeline();

sp.HandleRequest(null);
```
- In the config file add modifications as described in Step #3.
- Run the application.

Done.

Any time one needs to change the logic, add/remove aspects - it is just a matter of tweaking config file - that is it.

I've just copied and pasted existing aspects inside the configuration file and run the application again, here is what I get:

I think it is cool.

Focus on Security

From security perspective there are few things to keep in mind. When invoking assemblies with reflection there is immediate risk of luring attacks that result in spoofed assemblies. For more information how to get protected see my related post below - .Net Assembly Spoof Attack.

Focus on Performance

Reflection is considered as slow operation. Everything is relative. In case where reflection is used in performance critical application you should consider performance optimization for reflection. One such optimization is constructor caching as described in More Provider Goodieness

Focus on Operations

From operations perspective it seems ideal case. I witnessed few times the situation where the change was not an option since it required whole rebuild of the application - something that operations team was not happy with and put veto on it.

The trick is constantly asking yourself "What I am optimizing? Security, Performance, or Operations?"

My related posts

Download the sample from my SkyDrive:

Have fun.

ASP.NET AJAX Control Toolkit - Basic Sample For DynamicPopulate Control

alikl — Mon, 31 Dec 2007 07:49:45 GMT

How to dynamically populate the content of a control based on Web Service call triggered by another control? DynamicPopulate extender to the rescue:

DynamicPopulate is a simple extender that replaces the contents of a control with the result of a web service or page method call. The method call returns a string of HTML that is inserted as the children of the target element.

This post to summarize basic steps of using AJAX Control Toolkit's DynamicPopulate extender. Plus customer's case study of how it was implemented with ASP.NET Masterpage for performance and UX improvement.

Summary of steps

Step 1 - Create ASP.NET Web Application
Step 2 - Add server side code
Step 3 - Add DynamicPopulate extender
Step 4 - Add client side script to use the extender behavior
Step 5 - Add client side event to invoke the client script
Step 6 - Test the solution

Following section describes each step in details.

"Then another ask came through - content page must update the sidebar that is part of the Master page...."

Step 1 - Create ASP.NET Web Application. Open Visual Studio 2008. Create new "ASP.NET Web Application" project, found under Web node in "New Project" dialog. Name it DynamicPopulateSample. Add site's Master Page by right clicking on the project in project explorer window, then "Add" -> "New Item...", then choose "Master Page" template from the "New Item" dialog. Leave its default name. Drag Label control on the Master page from the Tool Box. The Label will be dynamically updated. Add new ASPX page by right clicking on the project in project explorer window, then "Add" -> "New Item....", then choose "Web Content Form". Specify it's Master Page by choosing Site1.Master. Name it default.aspx. Add two pure Html radio buttons to default.aspx. These will serve as a trigger to update the Label on the Master Page. Add AjaxControlToolkit.dll to the project's bin folder and add reference to it.

Step 2 - Add server side code. Open default.aspx.cs code behind for default.aspx. Add the following function:

[System.Web.Services.WebMethod]
[System.Web.Script.Services.ScriptMethod]
public static string GetHtml(string contextKey)
{
    // A little pause to mimic a latent call
    // This is the place to perform server side
    // code like DB access
    System.Threading.Thread.Sleep(350);

    return String.Format("Persona: {0}", contextKey);
}

Step 3 - Add DynamicPopulate extender. Register AjaxControlToolkit assembly inside the page. Add the following declaration right after <@Page...> directive

<%@ Register
    Assembly="AjaxControlToolkit"
    Namespace="AjaxControlToolkit"
    TagPrefix="ajaxToolkit"%>

Add AJAX Script Manager to the page:
<asp:ScriptManager ID="ScriptManager1"runat="server"/>

Add DynamicPopulateExtender control to the page:

<ajaxToolkit:DynamicPopulateExtender
          ID="dp"BehaviorID="dp1"runat="server"
          TargetControlID="form1$Label1"
          ClearContentsDuringUpdate="true"
          ServiceMethod="GetHtml" />

Notice TargetControlID is set to "form1$Label1". This is the Label control to be updated with the Html string returned by the server side code I described in Step 2. $ notation means nesting - read "Label1 control inside form1 control". It can be any nesting depth. Save your work and then view in browser to make sure that no exceptions generated.

Step 4 - Add client side script to use the extender behavior. So far, there is Label1 to be updated and it is sitting in the Mater page. There is DynamicPopulateExtender that defines the behavior. There is server side code to handle the request. Now add client script that makes the request to the server:

Notice initialization function call - the last row. It invokes the client side function to call into server side for the first time when the page is rendered into the browser for the first time.

Step 5 - Add client side event to invoke the client script. The last part is adding the event that triggers the client side code to be invoked and thus making call to the server. Locate Html radio buttons that were created during Step 1, add onclick events to call the client side function:

<input type="radio" 
       name="rbFormat" 
       id="r0" 
       value='alik'
       onclick="updateDateKey(this.value);" 
       checked="checked" />click to set 'alik'<br/>
<input type="radio" 
       name="rbFormat" 
       id="Radio1" 
       onclick="updateDateKey(this.value);"
       value='levin' />click to set 'levin'

Step 6 - Test the solution. Save your work and view in browser. Click on the radio buttons and you should see how the Label in master page gets updated with respective values. It all happens with small delays simulated on the server by Sleep function. I think it is cool.

Case Study

I was asked by a customer to offer a solution for very responsive UX [User Experience] while avoiding annoying refresh of the web page. Natural answer was AJAX. The customer also asked to provide the solution for common look and feel. Master pages was my answer. Then another ask came through - content page must update the sidebar that is part of the Master page.... hmm A-ha! Use DynamicPopulateExtender.

My Related posts

Sample VS2008 project that demonstrates DynamicPopulateExtender can be found on my SkyDrive:

Watch UX [User Experience] in the video below:

Video: DynamicPopulateExtender Demo

Enjoy.

ASP.NET 3.5 Extensions: Basic Steps To Create Dynamic Data Web Application - Focus On Security and Performance

alikl — Tue, 18 Dec 2007 13:08:01 GMT

This post walks through the steps I've taken to create simple Dynamic Data Web Application. I just loved the development model for DTO [Data Transfer Object] and Input Validation options.

Summary of steps

Step 1 - Download and install ASP.NET Extensions.
Step 2 - Create New Dynamic Data Web Application in VS2008
Step 3 - Add "LINQ to SQL Classes" file to the project
Step 4 - Test the project
Step 5 - Create Model Class and add validation rules

Following are detailed explanations for each step.

Step 1 - Download and install ASP.NET Extensions. ASP.NET Extensions can be found here - http://asp.net/downloads/3.5-extensions/. I installed it on my Vista machine that has Visual Studio 2008 installed.

Step 2 - Create New Dynamic Data Web Application in VS2008. Open Visual Studio 2008 and choose "Dynamic Data Web Application" template found under Web node. Make sure that the project references System.Web.Extensions assembly version 3.6. If not then remove it and reference the assembly from "C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\ASP.NET 3.5 Extensions\System.Web.Extensions.dll".

Step 3 - Add "LINQ to SQL Classes" file to the project. Right click on the project and choose "Add"->"New Item..." and then choose "LINQ to SQL Classes" template from "Data" node. In server Explorer expand "Data Connections" node, expand desired database, and then expand "Tables" node. Highlight desired tables and then drag them onto "LINQ to SQL Classes" component [double click file with .dbml extension].

Step 4 - Test the project. Right click on default.aspx page and choose "View in Browser". The page should look similar to the picture below. Click on the tables names to see the actual values and master-child view.

Step 5 - Create Model Class and add validation rules. Adding validation rules can be accomplished using .Net attributes declaratively or using partial methods. Create new class. Add "using System.Web.DynamicData;" declaration. Delete the default constructor. Add partial class with the name of desired table, say "Item" or/and "Product".

Attribute based input validation. Add attribute to the class, for example to check range for ListPrice field/column and add the following declaration:
Partial method input validation. Add partial method for desired field change [intelliSense really rocks at this part] and then add validation logic:

Validation rules propagated to both client [for usability] and server [for security], this is how violation of input validation looks in it default view:

Heaven.

Focus on Security

I can create data driven web pages using GridView and DataSets. The drawback is that validation is not that straightforward. On other hand I can create custom collections and manually data bind it - the code is much nicer and cleaner and validation rules are easy to add but there is the need of writing extra code. In the case of Dynamic Data there is fantastic combination of design time productivity and also clean code where validation rules are applied directly to the model. Less room for mistake to introduce security vulnerability.

Focus on Performance

Ready to go templates that are generated with the default Dynamic Data projects already implement AJAX and paging. It reduces dramatically amount of data that round trips over the wire. Large HTML output - including ViewState - is one of the biggest performance vulnerabilities I've noticed recently. AJAX and paging is a great way to overcome this issue.

My related posts

Related materials

The post was inspired and based on David Ebbo's fantastic screencast
ASP.NET 3.5 Extensions Quickstarts
Keeping View State on the Server
Performance Threats

Avoid Manipulating Passwords In Memory - It Is Easy To Reveal

alikl — Sat, 08 Dec 2007 08:55:40 GMT

Revealing clear text passwords in memory seems to be a trivial task. This post describes how to reveal clear text passwords and what countermeasures to apply.

Summary of steps:

Install WinDbg
Attach to process or open dump file
Load SOS .Net extensions for WinDbg
Enumerate threads
Enumerate objects in thread
Dump object's values
Countermeasures and guidelines

Install WinDbg

Download and install WinDbg as described in How to install Windbg and get your first memory dump.

Attach to process or open dump file

WinDbg can analyze both running processes and memory dumps which conveniently can be taken offsite for further investigation. I've created simple console application that accepts user name and password pair as its parameters and stores in local variables in memory:

static void Main(string[] args)
{
string userName = Console.ReadLine();
string password = Console.ReadLine();

Console.ReadLine();
}

Compile and run the application. I called it SecretsInMemory. This is how it looks when running:

Attach WinDbg to the running application by opening File->Attach to a Process:

and press Ok.

Alternatively, we can create dump file - for detailed how-to refer to How to install Windbg and get your first memory dump.

To Investigate resulting dump file in WinDbg open File->Open Crash Dump

Load SOS .Net extensions for WinDbg

To analyze .Net assemblies we need to load .Net extensions by typing .load sos and hitting Enter:

Enumerate threads

Run !threads command to enlist available threads:

and then choose specific thread - use left most column for thread identification as follows ~[thread number goes here]s:

Enumerate objects in thread

Use !dso command to dump all objects in the thread:

Dump object's values

Use !do <object address> to dump specific object's values. Object address is a second column in the list generated by !dso command, the column named "Object" - just copy and paste it:

The password is revealed either by attaching to the process or analyzing a crash file that was taken offsite.

Countermeasures and guidelines

As rule of thumb avoid using custom built identification and authentication mechanisms and leverage those that the infrastructure offers - preferably Windows Integrated authentication. In case where all options exhausted and there is no other way but accept end user credentials, refer to the following article - Using Credential Management in Windows XP and Windows Server 2003. Techniques described in the article allow to leverage built in mechanism of accepting credentials from end user in more secure manner. It also keeps common familiar look and feel across custom application and built in Windows mechanisms leaving less room for end user confusion.

My related posts:

Authentication Hub

Other resources:

Composite Application Block (CAB) Programming Essentials - Crucial For CAB Performance

alikl — Mon, 29 Oct 2007 23:30:16 GMT

Rich Newman posted awesome guides for Composite Application Block (CAB) programming:

Table of Contents: Introduction to CAB/SCSF

Part 1 Modules and Shells
Part 2 WorkItems
Part 3 Introduction to Dependency Injection
Part 4 An Aside on Inversion of Control, Dependency Inversion and Dependency Injection
Part 5 Dependency Injection and the Composite Application Block
Part 6 Constructor Injection in the Composite Application Block
Part 7 Introduction to Services in the Composite Application Block
Part 8 Creating and Using Services in the Composite Application Block
Part 9 The Command Design Pattern
Part 10 Commands in the Composite Application Block
Part 11 Introduction to Events in the Composite Application Block
Part 12 Events in the Composite Application Block
Part 13 Introduction to UIExtensionSites

It is important to understand these core principles. Recently I was involved with a project where CAB was used extensively. Too extensively... The application was actually over-CAB'ed causing performance hit. When we ran the profiler we saw that many functions calls were empty while adding up to execution time. The only solution was redesigning the application and CAB usage.

Lessons Learned:

Do not over CAB
Load modules on demand
Cache static data

Related resources:

Notes on (extreme) Performance requirements for CAB
Mobile Client Software Factory (includes includes CABgen and OBgen for NGEN'ing CAB modules)
How To: load CAB modules on demand

Authentication And Identity Flow When ASP Page Consumes ASP.NET Web Service

alikl — Wed, 05 Sep 2007 19:04:00 GMT

"Classic" ASP has application isolation that is different from ASP.NET. Here is one of the real world scenarios where it might matter.

There is a legacy web application written in ASP and hosted on Win2K3 box (IIS 6.0). It is of course in the process of migration to ASP.NET. As part of the migration process there were several ASP.NET web services factored out of the classic ASP app. These web services are hosted on another Win2K3 box and require windows authentication. Classic ASP must consume these web services while satisfying the requirement of windows authentication. ASP page consumes the web service via .Net COM interop invoking .Net component:

The question here is what is this account that ASP page authenticates to ASP.NET web service on another machine?

It is common mistake assuming that the account is the application pool's one. ASP does not run in the context of the application pool. In case of anonymous access It runs in the context of what defined for anonymous user:

Said that, in order to let ASP page authenticate to ASP.NET web service based on windows authentication one needs to define domain account in above property page for virtual directory where ASP resides. This is the account that will hit the ASP.NET web service.