HOWTO: Use the HTTP.SYS Kernel Mode Response Cache with IIS 6

David Wang discusses http.sys's kernel response cache.

WebTransports's WebLog — Fri, 15 Jul 2005 03:48:24 GMT

Check out David Wang's recent post about http.sys's response cache. He describes the configurable registry...

David Wang discusses http.sys's kernel response cache.

Windows Network Developer Platform — Fri, 15 Jul 2005 23:44:28 GMT

Check out David Wang's recent post about http.sys's response cache. He describes the configurable registry...

David Wang discusses http.sys's kernel response cache.

Windows Network Developer Platform — Fri, 15 Jul 2005 23:44:51 GMT

Check out David Wang's recent post about http.sys's response cache. He describes the configurable registry...

re: HOWTO: Use the HTTP.SYS Kernel Mode Response Cache with IIS 6

Rick Strahl — Fri, 12 Aug 2005 10:09:57 GMT

David,

Would it be possible to post some sample code on how to get a URL into the Cache from an ISAPI extension? The docs for HSE_REQ_VECTOR_SEND are sketchy at best and I'm not clear how set up the call and what to pass. Do you still use WriteClient or pass all to ServerSupportFunction or what?

re: HOWTO: Use the HTTP.SYS Kernel Mode Response Cache with IIS 6

David Wang — Fri, 12 Aug 2005 12:52:53 GMT

Yes, I'm planning on posting in another entry source code on how to both insert and invalidate the HTTP.SYS cache from ISAPI using HSE_REQ_VECTOR_SEND. I am just a bit swamped with IIS7 work at the moment (see my recent blog entries).

What aspect of the HSE_REQ_VECTOR_SEND documentation do you think is sketchy?

The earlier versions (which I never knew about nor reviewed) were pretty bad, but the latest version from Feb 2005 should have everything necessary. All the flags are finally there and described, related SSF function calls and structures are linked correctly, and everything is documented. Only thing thing that could be missing is a code sample, but that is not platform documentation.

A "HOWTO" document belongs elsewhere and was an article I planned to write (in the past) and publish through the MSDN/KB system, but I don't need to do that anymore because I'm just going to write it up and blog it now.

//David

re: HOWTO: Use the HTTP.SYS Kernel Mode Response Cache with IIS 6

Tom Keating — Thu, 13 Oct 2005 04:06:18 GMT

I think the IIS kernel caching is what is causing my RSS feed (index.xml) to be cached with old information. I have to restart IIS for the latest RSS feed to diaplay or wait what seems like an eternity for the latest RSS feed to display.

Is there a way of excluding certain files from kermel caching or at least force it to re-cache the file if the file changes?

Thanks

re: HOWTO: Use the HTTP.SYS Kernel Mode Response Cache with IIS 6

David Wang — Thu, 13 Oct 2005 12:17:33 GMT

Tom - Why do you think that kernel caching is the issue? Is the kernel cache hit perf counter increasing?

I presume .xml is handled as a static file on your server. If it is not, then this cannot be an IIS issue since caching behavior depends on the handler.

By default, IIS static file handler uses file-change notification to detect file changes and flush the appropriate cache(s).

There is no built-in declarative mechanism to include nor exclude resources from kernel caching, though it would not be very hard to implement the exclusion logic - just use a IIsWebFile to make a configuration setting on index.xml such that it fails the aforementioned caching criteria.

//David

re: HOWTO: Use the HTTP.SYS Kernel Mode Response Cache with IIS 6

Tom Keating — Thu, 13 Oct 2005 19:43:49 GMT

>> Why do you think that kernel caching is the issue?

If I open the file locally on the IIS server, the file is current. If I go to another PC which has never loaded the index.xml feed (not cached by the browser), it often loads an older version of the file. The only way I can see that happening is if the kernel is holding the file in memory.

I can even DELETE the index.xml file fom the IIS server and IIS will continue to "serve" the file as though it still exists. Weird, eh?

If I then stop/start IIS then I'd get a file not found error on my browser. (or if I didn't delete the file, restarting IIS makes sure the most recent XML file is served)

I assumed since the kernel cache caches heavily accessed files, such as my blog's RSS feed, that the kernel was keeping the file in memory and thus serving it from the kernel instead of the hard disk.

>>Is the kernel cache hit perf counter increasing?

Not sure how to add this to the performance monitor. I'll have to look into this.

>>I presume .xml is handled as a static file on your server

Not exactly sure what you mean. It isn't defined as a MIME type in the IIS Manager. Should it be? I didn't think I had to define the "common" file types under HTTP Headers, MIME Types. I'll ask our main IIS web guru about that.

Thanks for your help.

re: HOWTO: Use the HTTP.SYS Kernel Mode Response Cache with IIS 6

David Wang — Sat, 15 Oct 2005 11:48:09 GMT

Tom - IIS has many, many caches.

And when you suspect a caching issue, you need to determine:
1. What cache(s) are involved
2. What are the insertion and eviction policies of each cache involved

I just do not see how "assuming" behaviors is valid at all, thus I ask you for info proving what caches are involved (without SP1, only perfmon tells you if kernel response cache is involved - SP1 allows you to use ETW Tracing to see when a response is kernel response cached).

A blog is a web application that may configure IIS differently than default, so I assume nothing. For example, assuming the kernel response cache is involved here... then according to its cache insertion policy, either the IIS Static File Handler or an ISAPI inserted the cache entry.

To determine which happened, you need to see if options.xml has a MIME Type and is handled by the IIS Static File handler, or is it scriptmapped (or *-scriptmapped) to an ISAPI DLL. .XML has a default MIME Type so it should download by default unless you have it ScriptMapped and handled by another handler.

See this URL for details of how IIS6 determines which handler to process a requested resource:
http://blogs.msdn.com/david.wang/archive/2005/10/14/HOWTO_IIS_6_Request_Processing_Basics_Part_1.aspx

Thus, I can think of a case where ASP.Net is *-scriptmapped to handle all requests to that application, including options.xml. This means that ASP.Net's cache insertion policy is in effect, and maybe it does not heed file-change notification but rather Last-Modified and @Page directives. In this case, you can modify/delete options.xml all you want, but since it does not trigger anyone to evict the cached entry, it stays there until one of the other two kernel response cache eviction algorithms kick in.

At this point, you just have an unexplained caching phenomenon. Maybe RSS feed is not supposed to change that fast (if it changes fast, then why bother caching it?). Knowing what cache is involved and its insertion/eviction semantics allow you to determine whether something is actually broken or not.

//David

re: HOWTO: Use the HTTP.SYS Kernel Mode Response Cache with IIS 6

Stu Hodgson — Thu, 03 Nov 2005 17:37:29 GMT

Due to an encountered problem with caching it was decided to turn off caching within the registry. Whilst the implemntation uses IIS6 it is running in IIS5 compatability mode. The cache was proven to be Server side with the problem being when the client browser was closed and the same URL hit from another browser on any other netwoked machine cached information was displayed. Is anybody ware of a similar issue.

re: HOWTO: Use the HTTP.SYS Kernel Mode Response Cache with IIS 6

David Wang — Fri, 04 Nov 2005 11:39:13 GMT

Stu - Well, if you say that you rule out client-side cache by using different clients, and you rule out server-side cache by turning it off, then you probably need to look at caches in between the client and server in the network layer, like Proxy Servers and the like.

Now, you did not state WHAT type of content was cached - static or dynamic content (as identified by IIS) - because static content cache is controllable via metabase and/or registry key settings, but dynamic content cache is completely arbitrary and depends on the ISAPI implementation (this blog entry should have made that clear).

//David

re: HOWTO: Use the HTTP.SYS Kernel Mode Response Cache with IIS 6

Stu Hodgson — Thu, 10 Nov 2005 17:27:42 GMT

David- Thanks for your reply this issue was concerning dynamic content and investigation is ongoing around the ISAPI implementation. It is noted that this issue is not encountered with IIS5. The theory is that the third party security runtime ISAPI is not coded correctly for IIS6.

re: HOWTO: Use the HTTP.SYS Kernel Mode Response Cache with IIS 6

Jp Maxwell — Thu, 24 Nov 2005 02:13:08 GMT

David I am having the exact same problem Tom described. I have an XML file that when updated on the hard drive is not updated via http. I have disabled all asp.net & asp processing. I have enabled immediate content expiration. I have removed the "application" making just a straight static web server. It still caches. The XML file is updated on the hard drive every 5 minutes or so. In order for it to update via HTTP I have to do an IISRESET. Any help would be appreciated. FYI, this was a fresh Win 2003 install w/ all updates applied.

re: HOWTO: Use the HTTP.SYS Kernel Mode Response Cache with IIS 6

Jp Maxwell — Thu, 24 Nov 2005 05:01:21 GMT

Please see:

http://www.experts-exchange.com/Web/Web_Servers/IIS/Q_21037463.html?query=IIS6+Cache&clearTAFilter=true

re: HOWTO: Use the HTTP.SYS Kernel Mode Response Cache with IIS 6

fublogs — Tue, 17 Jan 2006 22:22:27 GMT

Recently,I write a iis filter for iis 6.0;

but,i found that event client cancel download

file,but filter can receive "sf_notifiy_send_raw_data" up to 2000 times.

please help me! email:fujiachun@163.com

Thanks!!!!

re: HOWTO: Use the HTTP.SYS Kernel Mode Response Cache with IIS 6

David Wang — Wed, 18 Jan 2006 11:23:13 GMT

fublogs - The behavior is by-design on IIS6 due to HTTP.SYS buffering of the outgoing response. There is no way for an ISAPI Filter to detect if the client cancelled the download or not. There is full discussion of this on the newsgroup microsoft.public.platformsdk.internet.server.isapi-dev :
http://groups.google.com/group/microsoft.public.platformsdk.internet.server.isapi-dev/browse_thread/thread/73cde21658c5be27/118e4c7d608a311e
http://groups.google.com/group/microsoft.public.platformsdk.internet.server.isapi-dev/browse_thread/thread/c9ebb5f37d8f7d61/8c2b6dbca5231633

//David

re: HOWTO: Use the HTTP.SYS Kernel Mode Response Cache with IIS 6

Luke Harris — Mon, 13 Feb 2006 00:36:17 GMT

I was wondering about the use of NPP by HTTP.sys doing HTTP response caching. I guess I might be making a wrong assumption, but I thought...

(1) HTTP.SYS is a kernel mode driver.
(2) Kernel mode drivers can only access NPP.
(3) NPP is restricted to 128Mb RAM on X86 architecture.

If that's right, then 128Mb doesn't seem a lot of space for caching, especially since you wouldn't want to use all of the NPP!

Thanks if you get a chance to address this in your blog

re: HOWTO: Use the HTTP.SYS Kernel Mode Response Cache with IIS 6

David.Wang — Tue, 14 Feb 2006 02:44:34 GMT

Luke - yes, you are making some wrong assumptions... :-)

Kernel mode code can access much more than just NPP.

HTTP.SYS actually uses available physical memory (configurable limit) for the cached response content of the "kernel response cache".

//David

re: HOWTO: Use the HTTP.SYS Kernel Mode Response Cache with IIS 6

Charm — Tue, 21 Feb 2006 21:08:18 GMT

David, would POSTs with 200 0 64 reponses (from IIS log) be cached. What I'm experiencing is these responses are being repeated until a 200 0 0 response is returned. This causes multiple posts within seconds of each other.

I have not been able to find a concrete answer to tell me what the 200 0 64 or 200 0 1236 mean.

I hope you can help shed some light.

re: HOWTO: Use the HTTP.SYS Kernel Mode Response Cache with IIS 6

David.Wang — Tue, 21 Feb 2006 23:48:11 GMT

Charm - I do not think "this causes multiple posts within seconds of each other" has anything to do with caching because:
1. when multiple posts happen, it means that multiple requests were processed and responses generated
2. when cached response happens, it means that request is handled from the cache *without* any processing

What this means is that if caching is involved, request handling never happens, and without request handling you cannot have "multiple post".

64 and 1236 are Win32 error codes.

NET HELPMSG 64 returns:
The specified network name is no longer available.

NET HELPMSG 1236 returns:
The network connection was aborted by the local system.

They both indicate that the connection was prematurely closed, so it really does not indicate failure or success. All you know is that from server side, it thinks everything is ok.

I think the client is multi-posting for you and possibly causing your duplicate posts.

//David

re: HOWTO: Use the HTTP.SYS Kernel Mode Response Cache with IIS 6

Charm — Thu, 23 Feb 2006 20:19:24 GMT

Thanks for the quick response David. After another day of troubleshooting duplicate posts, I have no solid answers. The users swear they only pressed the Submit button once. We're seeing GET and POST requests with the 200 0 64 responses. While I don't care much about the GETs, the POSTs are killers as it will cause the app to post multiple times, creating charging or crediting clients multiple times.

It’s happening at random on two load balanced servers, each with quad CPUs. The app was just migrated from W2K to these W2K3 servers. We did not see this issue in Stage where UAT and Intergration Testing was done.

Is there a way to configure IIS to prevent identical POSTs that have 200 0 64’s in the last 10 seconds or to treat all 200 0 64s as a 500 errors?

re: HOWTO: Use the HTTP.SYS Kernel Mode Response Cache with IIS 6

David.Wang — Fri, 24 Feb 2006 01:21:26 GMT

Charm - The safest and easiest way to prevent multiple POSTs is for your application to synchronize the action itself. People tend to use Session State to do this... and if you load-balance and use out-of-process Session State, users can bounce between the load balanced servers and still not duplicate POST.

In general, this sort of problem is best solved at the user application level. The notion of "preventing identical posts within the last 10 seconds" does not solve the underlying logic problem at the application layer.

//David

re: HOWTO: Use the HTTP.SYS Kernel Mode Response Cache with IIS 6

Rob Gwen — Mon, 15 May 2006 21:20:09 GMT

"A static file (i.e. .htm, .css, .js, .jpg) is NOT guaranteed to be kernel response cached. You could have an [Wildcard] Application Mapping that applies to those extensions"

Hi,
A newbie question: I need kernel caching for .jpg files. To what application should I map?
Thanks...

re: HOWTO: Use the HTTP.SYS Kernel Mode Response Cache with IIS 6

Adi — Tue, 03 Oct 2006 19:10:33 GMT

David,

Just a quick question:

If you use HttpContext.RewritePath in ASP.NET does that mean you can never benefit of kernel mode caching? A yes/no answer would suffice. A more detailed answer would be greatly appreciated.

Thanks,

Adi

re: HOWTO: Use the HTTP.SYS Kernel Mode Response Cache with IIS 6

Dharmesh Shah — Sat, 06 Oct 2007 02:15:14 GMT

Hi David,

In your February 23, 2006 reply - you have mentioned "The safest and easiest way to prevent multiple POSTs is for your application to synchronize the action itself."

I would request you to share the technic/logic to be used to identify the duplicate POST request.

Thanks,

Dharmesh

re: HOWTO: Use the HTTP.SYS Kernel Mode Response Cache with IIS 6

Radha Krishna Prasad — Wed, 02 Jan 2008 13:15:13 GMT

I am not able to access a sql end point (created on a live server) from a proxy server enabled LAN.

I am using Windows Server 2003 SP-1, So do I need any registry edit?

re: HOWTO: Use the HTTP.SYS Kernel Mode Response Cache with IIS 6

David.Wang — Sun, 06 Jan 2008 13:51:00 GMT

Rob - .jpg files are kernel response cached by default unless otherwise configured. Read the URL link in this same blog entry for the full meaning of "otherwise configured".

//David

re: HOWTO: Use the HTTP.SYS Kernel Mode Response Cache with IIS 6

David.Wang — Sun, 06 Jan 2008 13:54:02 GMT

Adi - rewriting the URL breaks all caching, kernel response included, by definition.

IIS cannot be certain that your custom code rewrites URLs consistently to ensure cached responses are also consistent and correct. Thus, if any code attempts to rewrite the URL, caching is disabled.

//David

re: HOWTO: Use the HTTP.SYS Kernel Mode Response Cache with IIS 6

David.Wang — Sun, 06 Jan 2008 13:59:42 GMT

Dharmesh Shah - thanks for the request, but I believe it is an exercise to be completed by the reader.

Please read my BIO on why I do not answer questions posed like your own. If you have a specific question on how to do something, then I will try to help with knowledge -- but I cannot just "do something for you".

//David

re: HOWTO: Use the HTTP.SYS Kernel Mode Response Cache with IIS 6

Brad Laney — Wed, 06 Feb 2008 03:46:15 GMT

Hello,

Your post helped us find our problem well.

I've verified that the pages are being inserted into the HTTP.SYS Response cache, but the problem is it doesn't update.

We are a hosting company, and our clients CSS files do not update during design modifications. Recycling the application pools does not help, I believe restarting IIS doesn't help either (neither of which are acceptable).

I need a way to force the HTTP.SYS to run its algorithm to get new content static pages.

"On-Demand - ISAPI Extension can use the HSE_REQ_GET_CACHE_INVALIDATION_CALLBACK ServerSupportFunction to retrieve a pointer to the on-demand URL revocation function"

Well.. This page is specific to

Microsoft Visual Studio 2008/.NET Framework 3.5

I am running classic ASP and do not have the option to run .NET to do this.

Am I just up the creek on this one?

re: HOWTO: Use the HTTP.SYS Kernel Mode Response Cache with IIS 6

mitsuwo — Wed, 06 Feb 2008 12:11:06 GMT

Hi David,

I have a problem same as brad posted above.

Is There any way to use this future with classic ASP?

Can'I solve the problem by write ISAIP Filter or other programs ?

re: HOWTO: Use the HTTP.SYS Kernel Mode Response Cache with IIS 6

John Hancock — Sat, 26 Apr 2008 11:51:03 GMT

@Brad,

What will help is stopping the website in IIS, connecting to the URL through a browser and restarting it.

Of course, you can't give your hosting clients that kinda access.

We're down to metabase edits for some stuff, but for the css....

Simple way is to use the Custom Headers and write type: 'Cache-Control' value: 'no-cache' after right-clicking and editing properties of the css file you want to refresh every time. Of course, we like our CSS files to get cached in production, so we faff around every time. Oh for a list-view and a checkbox.

John

re: HOWTO: Use the HTTP.SYS Kernel Mode Response Cache with IIS 6

MikeC — Wed, 05 Nov 2008 23:54:46 GMT

David,

I need to flush periodically the entire IIS6/7 http.sys cache programmatically from a filter dll. The cache invalidation function obtained via HSE_REQ_GET_CACHE_INVALIDATION_CALLBACK apparently works on a single URL -- or is there a wildcard to zap all? One other bit of advice I see is to set cache invalidation time to zero, then back to the original value, presumably via CacheControlMaxAge, or some such. But does this work without restarting the service? Many thanks for your informative public-spirited site.

--Mike

re: HOWTO: Use the HTTP.SYS Kernel Mode Response Cache with IIS 6

Southside Steve — Thu, 08 Jan 2009 21:42:31 GMT

Is there any way to monitor cache flush events? I want to see why the Current Files Cached and Current URIs cached continually get flushed. There is no expires: in the headers, the TTL is set higher and the scavenger period is set longer than the frequency with which these things seem to get flushed. (IIS 6.0)

Steve

re: HOWTO: Use the HTTP.SYS Kernel Mode Response Cache with IIS 6

Abhijeet — Mon, 12 Jan 2009 16:02:52 GMT

Is there any way to configure IIS to clear cache at the end of every session? What are pros and cons?

re: HOWTO: Use the HTTP.SYS Kernel Mode Response Cache with IIS 6

David.Wang — Wed, 14 Jan 2009 10:23:03 GMT

Abhijeet - Can you clarify what you are trying to accomplish?

There is no such thing as "session" in HTTP or IIS, so your question has no direct meaning.

//David

re: HOWTO: Use the HTTP.SYS Kernel Mode Response Cache with IIS 6

Alex — Wed, 13 May 2009 01:30:43 GMT

___________________________

Win2K3 + II6 Double Posts:

Trying to uncover the reason why IIS6 would not be logging requests for 2 minutes. An outage reported by an external ping, first thought: connection issue. However a little more digging and the IIS time is specified as the end time by http://www.w3.org/TR/WD-logfile.html (microsoft refers to it as the time the activity occured) so naturaly start time should be calculated by subtracting the time-taken from end time.

Now to prove this I've made a very simple asp script. it sleeps for 10s using the SQL Server.

I ran the script these are my results:

please note connection error the first time it ran

then an automatic re-request which took 17 seconds for a script that is supposed to take 10 seconds.

this produces more questions then answers

I was not able to reproduce this behavior again (however maybe some of the issues we're having could be explained by this misterious behavior), I've tried renaming the script changing the code a bit, but it now always produces a single entry of little more then 10s.

______________

Script Output:

First Run:

Start time: 15:38:03

End time: 15:38:13

Second Run:

Start time: 15:53:45

End time: 15:53:55

____________

IIS logfile:

2009-05-12 20:38:03 W3SVC920801317 GET /inc/sleep.asp - 80 - 76.65.237.13 HTTP/1.1 Mozilla/5.0+(Windows;+U;+Windows+NT+5.1;+en-US)+AppleWebKit/525.19+(KHTML,+like+Gecko)+Chrome/1.0.154.65+Safari/525.19 ASPSESSIONIDAABBRBAB=MIKCIGIAODFAAMDIGHMMJGAE 200 0 64 220 508 10171 2009-05-12 20:38:13 W3SVC920801317 GET /inc/sleep.asp - 80 - 76.65.237.13 HTTP/1.1 Mozilla/5.0+(Windows;+U;+Windows+NT+5.1;+en-US)+AppleWebKit/525.19+(KHTML,+like+Gecko)+Chrome/1.0.154.65+Safari/525.19 ASPSESSIONIDAABBRBAB=MIKCIGIAODFAAMDIGHMMJGAE 200 0 0 260 508 16749 2009-05-12 20:53:55 W3SVC920801317 GET /inc/sleep.asp - 80 - 76.65.237.13 HTTP/1.1 Mozilla/5.0+(Windows;+U;+Windows+NT+5.1;+en-US)+AppleWebKit/525.19+(KHTML,+like+Gecko)+Chrome/1.0.154.65+Safari/525.19 ASPSESSIONIDAABBRBAB=MIKCIGIAODFAAMDIGHMMJGAE 200 0 0 260 508 10062

re: HOWTO: Use the HTTP.SYS Kernel Mode Response Cache with IIS 6

Alex — Fri, 05 Jun 2009 18:39:46 GMT

One more question. Does IIS6 and IIS7 have any simple way of reloading file caches?

At the moment I'm mostly concerned with the cache for the compiled classic ASP files. I'm using NTFS junctions for versioning and instant reverts. However IIS does not detect that the files have changed. The only way that I found how to get IIS6 to reload the files without loosing sessions, is to change the following metabase

properties to 0

AspScriptFileCacheSize="0"

AspMaxDiskTemplateCacheFiles="0"

AspScriptEngineCacheMax="0"

and then change them back.

Surely there must be some simple scriptable interface for controlling various caches in IIS?

re: HOWTO: Use the HTTP.SYS Kernel Mode Response Cache with IIS 6

Daniel Mizrahi — Fri, 07 Aug 2009 18:44:07 GMT

Hi David,

I have been looking for the answer to this for a few days, but haven't been able to get it answer.

I have three environments setup identically (DEV, QA, STAGING) using IIS. We have recently made changes to an RSS generator app where we only added a new parameter in the querystring. We found that in DEV and QA environment RSS is not being cached. But when we moved application to our STAGING environment, response is being cached for the <ttl> period of time specified in the RSS.

Any ideas as to how the cachin and ttl works in this situation? Where is it stored?

Any lights to this issue would be really appreciated.

Thanks

Daniel Mizrahi