Thomas Marquardt's Blog

Correct use of System.Web.HttpResponse.Redirect

tmarq — Thu, 25 Jun 2009 20:10:00 GMT

Try very, very hard to avoid using Response.Redirect(url), instead, use Response.Redirect(url, false). Response.Redirect(url), after writing a 302 redirect response to the response buffers, calls Response.End. This is very expensive. The alternative, Response.Redirect(url, false) is fast, but unlike Response.Redirect(url), the lines of code which follow the call to Response.Redirect(url, false) will be executed. More on this later, but first, let me tell you about the horrors of Response.End.

Response.End is a terrible method. It was added in ASP.NET 1.0 for compatibility with classic ASP. In classic ASP, this method terminated script processing, so that the lines of script that followed this call never executed. How do you simulate that in managed code? The only way is to abort the thread, which raises a ThreadAbortException and is outrageously expensive. Normally, that's exactly what Response.End does in ASP.NET, however, if it is called from within an asynchronous handler/module (a.k.a. an asynchronous pipeline event), it will instead perform a synchronous flush of the response buffers (can you say expensive?) and then complete the request by calling Context.ApplicationInstance.CompleteRequest(). Either way, whether you're calling it from a synchronous handler/module or an asynchronous handler/module, Response.End is horribly expensive, and you should avoid it.

Ok, so what if you don't want the lines of code to execute after you redirect? Well, one way to accomplish this is to call HttpApplication.CompleteRequest(), which is accessible from the HttpContext. e.g., call calling Context.ApplicationInstance.CompleteRequest(). It's not the same as aborting the thread, which truly does prevent all subsequent lines of code form running. The lines of code that follow the call to CompleteRequest() will execute, but as soon as the current page or module that calls this completes, the pipeline will jump ahead to the EndRequest event, thereby short circuiting the pipeline. This is usually all you need.

So to summarize...

BAD:

Response.Redirect(url);

GOOD:

Response.Redirect(url, false);

Context.ApplicationInstance.CompleteRequest();

But before I put my pen away, there's one more common practice I've seen that people should be aware of . In classic mode, calling Response.Redirect(url) from Application_Error without calling Server.ClearError is doubly expensive. It causes three exceptions to be thrown, and then either raises a ThreadAbortException or does a synchronous flush and completes the response. And in integrated mode, calling Response.Redirect(url) from Application_Error without calling Server.ClearError does not even work. Instead, you should use the following code, which performs well in both classic mode and integrated mode. If you’re going to redirect from Application_Error, you should do it the following way:

GOOD:

void Application_Error(object sender, EventArgs e)

{

Server.ClearError();

Response.Redirect(url, false);

Context.ApplicationInstance.CompleteRequest();

}

The behavior difference between classic and integrated mode with respect to calling Redirect from Application_Error without calling Server.ClearError is just due to the different environments. You might notice that with a default 3.5 install, if you remove the ASP.NET ScriptModule from the IIS <modules> section, you're suddenly able to call Redirect from Application_Error without calling Server.ClearError. ScriptModule registers a PreSendRequestHeaders handler. In integrated mode, the PreSendRequestHeaders event is implemented differently. As a result, in integrated mode, the exception will be rendered if you try to redirect without clearing the error from Application_Error. I’ve attached a sample that demonstrates the difference between the two pipelines. This sample will demonstrate the difference regardless of whether or not ScriptModule is installed. Just request default.aspx. Then change the value of demonstratePipelineDifference in global.asax, and request default.aspx again. Do this in both classic mode and integrated mode, and observe the behavior.

Thanks,

Thomas

Obtaining more information from an HttpException

tmarq — Tue, 10 Mar 2009 23:44:00 GMT

ASP.NET throws HttpException for numerous types of errors, making it difficult to determine exactly what went wrong programmatically. You can parse the error message, but the message can be localized, and therefore parsing it can be non-trivial. There is, however, a way that you can identify a few specific types of errors.

In v2.0 there is an internal property on HttpException named WebEventCode that returns an int. This value contains additional information which you can use to determine the type of error. Because it is internal you will have to use private reflection to get the value, but the property has been made public in v4.0. Below I've listed the meaning of several of the possible values of this property:

If the HttpException is because the request timed out ("Request timed out."), the value will be System.Web.Management.WebEventCodes.RuntimeErrorRequestAbort.
If it's because the client posted too much data ("Maximum request length exceeded."), the value will be WebEventCodes.RuntimeErrorPostTooLarge.
If it's caused by a ViewState error, the value will be WebEventCodes.RuntimeErrorViewStateFailure.

Again, this property has been made public in v4.0, but for v2.0, you'll have to use private reflection to access it.

-Thomas

Using IIS 7.0 Dynamic Compression with ASP.NET Output Cache

tmarq — Wed, 27 Aug 2008 22:31:00 GMT

This post discusses an efficient way to compress content served by the ASP.NET output cache. In .NET Framework v2.0 SP1, included with Windows Server 2008, the ASP.NET Output Cache includes a new feature that allows cached responses to vary by the response's Content-Encoding header. And IIS 7.0 has efficient compression algorithms for gzip and deflate. Put the two together, and you can efficiently compress responses once, and serve them from the cache many times.

Here's how you can do this on Windows Server 2008:

1. If you haven't already, install the Dynamic Content Compression role for IIS 7.0 using the Server Manager.

2. Step 1 will add the <httpCompression> configuration section to applicationHost.config. Note that only gzip compression is supported by default. However, deflate is also implemented in gzip.dll, so if you add the following to the <httpCompression> configuration section, you'll also have deflate enabled:

<scheme name="deflate" dll="%Windir%\system32\inetsrv\gzip.dll" />

3. Create an application with an ASPX page like the one below. This example page will cache at most 3 responses, one that is gzip compressed, one deflate compressed, and one that is a normal (uncompressed) response.

<%@ Page language="C#" %>
<%@ OutputCache duration="600" varyByParam="None" varyByContentEncoding="gzip;deflate" %>

<script runat="server">
    void Page_Load() {
        Request.ServerVariables["IIS_EnableDynamicCompression"] = "1";
        Response.Write(DateTime.Now);
    }
</script>

4. You must enable the IIS 7.0 "dynamic compression before cache" feature. You can do this by adding the following to your application's web.config file.

<system.webServer>
<urlCompression doDynamicCompression="false" dynamicCompressionBeforeCache="true" />
</system.webServer>

Note that in Step 4, doDynamicCompression="false". If you set this to "true", all responses will be compressed. That is, pages that are not using the output cache, as well as pages that are using the output cache (but may not be using varyByContentEncoding) will be compressed. Our example sets doDynamicCompression to "false" in configuration, but then uses the server variable "IIS_EnableDynamicCompression" to enable it for a specific page. Alternatively, you could use configuration and <location path=""> to limit the scope of compression.

So how does this work? When a request is sent to the server, it may or may not include an Accept-Encoding header, indicating one or more acceptable encodings. For example, Internet Explorer sends "Accept-Encoding: gzip, deflate". When the ASP.NET Output Cache receives the request, if the page is using the VaryByContentEncoding feature, ASP.NET will check if any of the encodings specified by VaryByContentEncoding match the request's Accept-Encoding header. If there is a match, ASP.NET will see if that response is in the cache and serve it. If it's not cached, ASP.NET will render the response and the IIS 7.0 DynamicCompressionModule, which runs during RQ_RELEASE_REQUEST_STATE, will encode the response according to the request's Accept-Encoding header, and it will set the response's Content-Encoding header. For the example request, the response header will be set to "Content-Encoding: gzip". When the ASP.NET OutputCacheModule runs during RQ_UPDATE_REQUEST_CACHE, it will check to see if the Content-Encoding header has been set, and if specified in VaryByContentEncoding, it will cache the response according to that encoding. If the Content-Encoding is not set, the response can be cached as the identity.

This ASP.NET Output Cache feature can be used via the <%@ OutputCache varyByContentEncoding= %> page directive, programmatically via HttpCachePolicy.VaryByContentEncodings, or via configuration outputCacheProfiles><add varyByContentEncoding=””></outputCacheProfiles>.

Here are a few of the implementation details:

The value of the VaryByContentEncoding property is the name of an encoding, or a semi-colon separated list of encodings (e.g. "gzip", "gzip; deflate; compress").
The identity, or plain text, is a special case. You can consider it to be implicitly included in the value of VaryByContentEncoding. E.g., if you specify varyByContentEncoding="gzip", ASP.NET will cache at most two responses, gzip and the identity.
A response can be cached only if the value of its Content-Encoding header is included in VaryByContentEncoding; except for the identity, which is a special case. If the response does not have a Content-Encoding header, it can be cached as the identity.
A response will be served from the cache if it has an acceptable encoding according to RFC 2616, Section 14.3 (qvalues are supported). However, even if the identity is available in the cache, if an acceptable encoding is found in VaryByContentEncoding, but that encoding has not yet been cached, ASP.NET will allow the page to render so that a response for that encoding may be cached.
If the value of the Content-Encoding response header is not in the list, the response is not cacheable. If the Content-Encoding response header is in the list, the response is cacheable (assuming nothing else in its cache policy prevents it from being cached).
If the Content-Encoding response header is not set, the response may be cached as the identity (again, assuming nothing else in its cache policy prevents it from being cached).
If the request's Accept-Encoding doesn't match any cached responses, and is not listed in VaryByContentEncoding, and the identity is available in the cache, then the response served will be the identity.

Regards,
Thomas

ASP.NET Cache can notify you before an entry is removed

tmarq — Tue, 22 Jul 2008 19:32:00 GMT

.NET Framework v2.0 SP2, which will be included with .NET Framework v3.5 SP1, is scheduled to release this summer. This release contains a long awaited ASP.NET Cache feature, which will notify you when an entry becomes invalid and leave the stale entry in the cache while you generate a replacement in the background. Therefore, requests are not blocked attempting to access the cache entry while it is being updated, and the update is done by a single thread. Here's the specification:

A new Cache.Insert overload:

void Cache.Insert( String key,

Object expensiveObject,

CacheDependency dependencies,

DateTime absoluteExpiration,

TimeSpan slidingExpiration,

CacheItemUpdateCallback onUpdateCallback

);

This method inserts an object into the cache, with optional dependencies and expiration policy, and a callback that is invoked before the entry is removed. Objects inserted into the cache with this method should be expensive to generate. You should not use this method for objects that are inexpensive to generate, because there is overhead in the implementation. When the callback is invoked, the application developer should generate a new object, with new dependencies and expiration policy, and return it via the callback’s out parameters. The stale cache entry will be accessible while the callback is executing and the replacement object is generated.

Cache entries inserted by this method are not subject to eviction by the cache memory manager. These objects only become invalid when they expire or a dependency changes, and therefore these objects MUST have some form of invalidation (i.e., it is not possible to insert the entry with a null dependency, NoAbsoluteExpiration, and NoSlidingExpriation).

A new update callback:

void CacheItemUpdateCallback( String key,

CacheItemUpdateReason reason,
out Object expensiveObject,

out CacheDependency dependencies,

out DateTime absoluteExpiration,

out TimeSpan slidingExpiration

);

When the cache entry becomes invalid, ASP.NET invokes this callback, allowing the application developer to generate a replacement cache entry. The developer is passed only two pieces of information: the key for the cache entry, and an enum value indicating why the entry is invalid. The rest of the arguments to the callback are output-only parameters, allowing references to the newly generated object, dependencies, and expiration policy to be passed back and subsequently inserted into the cache by ASP.NET.

Do not modify the existing cache entry. The stale cache entry is still alive and potentially being used by other threads. It is not included in the callback arguments in order to minimize the likelihood of a developer accidentally updating portions of the cache entry. Although this pitfall exists today in the cache API (e.g. a cache entry could be modified in a non-thread-safe manner by an ASP.NET developer), we think leaving the cached value out of the callback arguments will reduce the likelihood of a developer stumbling into this.

To remove the cache entry, simply set the argument expensiveObject to null. Otherwise, set it to the newly generated replacement object. The developer MUST also specify expiration and/or dependency information using the other out parameters. As stated above, the cache entry is not subject to eviction by the cache memory manager, and therefore must have some form of invalidation. It is an error to pass back a non-null value for expensiveObject while setting dependencies to null, absoluteExpiration to NoAbsolteExpiration, and slidingExpriation to NoSlidingExpiration. The cache entry will be removed if this occurs.

If the developer’s callback throws an exception, ASP.NET will suppress the exception and remove the cache entry. The exception needs to be suppressed because cache item updates occur in the background where there is no call path for routing the exception back to user code. The cache entry is removed because without a new cache value, ASP.NET has no way of knowing what the correct behavior is for updating the cached item.

The update callback will not be called if the cached item was explicitly removed via a call to Cache.Remove, or a call to Cache.Insert (Insert removes a pre-existing entry). These operations indicate the developer’s intent to remove the item, so the update callback will not be triggered.

A new update reason enumeration:

public enum CacheItemUpdateReason {

// The item was removed from the cache because either

// the absolute or sliding expiration interval expired.

Expired = 1,

// The item was removed from the cache because an

// associated cache dependency was changed.

DependencyChanged = 2

}

Unlike the CacheItemRemovedReason enumeration, the update reason enumeration does not include values for Removed (see explanation above) or Underused (as mentioned above, the cache entry is not subject to eviction by the cache memory manager).

Regards,

Thomas

If you allocate from the Large Object Heap each time an ASPX page is requested, your performance will be dismal.

tmarq — Tue, 29 Apr 2008 23:12:00 GMT

I've been giving this advice for years internally, and the problem pops up way too often. I'm sure it will be addressed eventually, in a future version of the CLR, but for now, every released version of the CLR has this problem. If you allocate objects larger than 85000 bytes, they will be allocated in the Large Object Heap (LOH). The LOH is not compacted and collection only occurs after a full collection (so gen 2 must be collected in order to collect an object in the LOH). Short-lived and frequent LOH allocations will kill the performance of your application. As an example, if you allocate from the LOH each time an ASPX page is requested, your performance will be dismal.

To know if you’re allocating from the Large Object Heap (LOH), you can use this break point on v2.0 on a multi-proc box, where the server GC is enabled by default:

bp mscorwks!SVR::gc_heap::allocate_large_object ".echo size=; ? poi(@esp+4); k12; gc"

If you want to get fancy, the following will attach ntsd to w3wp.exe, disable break on first chance exceptions, and trace the calls to allocate_large_object to %SYSTEMDRIVE%\NTSD.log

ntsd.exe -y %WINDIR%\symbols;srv*%WINDIR%\Symbols*http://msdl.microsoft.com/download/symbols -xd * -c "bp mscorwks!SVR::gc_heap::allocate_large_object \".echo size=; ? poi(@esp+4); k12; gc\"; g" -G -logo %SYSTEMDRIVE%\NTSD.log -pn w3wp.exe

You can also deterimine if you're allocating from the LOH by profiling a request with the CLR Profiler, which is discussed in an earlier blog post. The Histogram view shows you a histogram of the allocations broken down by size, and you can quickly check to see if there's anything larger than 85000 bytes.

Aside from short-lived LOH allocations, the most common cause of memory related issues in ASP.NET apps is pinning managed memory during long async I/O operations. ASP.NET itself does not do this, but perhaps your application does?

Thanks,

Thomas

ASP.NET File Change Notifications, exactly which files and directories are monitored?

tmarq — Fri, 02 Nov 2007 06:15:00 GMT

Perhaps one of the most loved and hated features of ASP.NET is the ability to detect file changes and automatically recompile or reconfigure the application. In development scenarios it’s very nice because you can access the application in a browser at the same time you’re modifying ASPX, ASAX, DLL, CONFIG, and other application files, and you don’t have to stop the web server process in order to update DLLs or configuration. The changes are picked up immediately, but in order to make this possible we have to recycle the AppDomain. In deployment scenarios, this feature can be a problem. There are many files and folders (e.g. global.asax, web.config, BIN, App_Code, etc) that trigger an AppDomain unload when changed. A subsequent request to the application will cause the AppDomain to be loaded again and the new files will be compiled, if necessary. Restarting the application is expensive due to the disk access and recompilation of files. Many applications also have expensive initialization steps that occur when the AppDomain is loaded, such as accessing SQL to populate local stores with frequently accessed data. So what exactly does ASP.NET monitor and how?

ASP.NET uses the Win32 function ReadDirectoryChangesW to monitor directories and files. Given a directory handle, this function will let you know lots of information about changes to the directory or the files and directories contained within it. By passing different flags, you tell it exactly what information you’re interested in and it informs you when there are changes. ASP.NET monitors directories with the following flags (they are fairly self-explanatory, but you can refer to ReadDirectoryChangesW documentation for details):

FILE_NOTIFY_CHANGE_FILE_NAME
FILE_NOTIFY_CHANGE_DIR_NAME
FILE_NOTIFY_CHANGE_CREATION
FILE_NOTIFY_CHANGE_SIZE
FILE_NOTIFY_CHANGE_LAST_WRITE
FILE_NOTIFY_CHANGE_SECURITY

The above flags are the ones that ASP.NET uses, but not all of them are used for each directory handle that is monitored. So which directories are monitored? By default in ASP.NET 2.0, there are six directory monitors for each application path, and two additional directory monitors for each virtual subdirectory path. This is best explained with an example. Suppose that we have an application with a virtual path of "/vpath" and a physical path of "c:\ppath". If you make a request to a file located at "/vpath/f.aspx", the following directory monitors will be created:

#1) The application physical root path and all of its subdirectories are monitored for subdirectory name changes or deletions. To be more precise, if FILE_NOTIFY_INFORMATION.Action is FILE_ACTION_RENAMED_OLD_NAME or FILE_ACTION_REMOVED, the AppDomain is unloaded. This means that if you rename or delete a subdirectory beneath the application physical root, the application will be restarted. This is discussed in Todd Carter's blog post.