Why some ISAPI Filter events trigger multiple times per request

re: Why some ISAPI Filter events trigger multiple times per request

David.Wang — Fri, 09 May 2008 04:47:20 GMT

Mario - modifying the raw response data to perform logical HTTP operations is very difficult to do correctly and impossible to do performantly. SendRawData filter turns off just about all the IIS output caches and optimizations and turns async IO paths synchronous. For example, TransmitFile used to do optimal user-kernel transitions but NOT when SendRawData Filter is involved, even if the filter does nothing.

1. You must buffer the response header at the very beginning SF_NOTIFY_SEND_RAW_DATA and prevent it from being sent -- since you likely have to modify it.

2. Using pfc->pFilterContext to store the buffered response header AND entity body is required for efficiency. If you do not use it, you would have to use global read/write variables, and that introduces read/write locks and thus contention into the critical data output pathway across the entire server... which would be horrendous.

3. SF_NOTIFY_END_OF_REQUEST tells you that the request would not generate any more SF_NOTIFY_SEND_RAW_DATA. However, you would have to WriteClient() your buffered response in that event to flush the response back to the client, which itself would generate SF_NOTIFY_SEND_RAW_DATA... so if you are not careful, you would buffer everything and emit nothing.

//David

re: Why some ISAPI Filter events trigger multiple times per request

Mario — Wed, 07 May 2008 14:04:11 GMT

Thank you David,

That was concise and to the piont, and as mentioned in other posts, this is a duanting task. Would it say to assume the following as I move in this task.

1. Regardless of how many event triggers, I will always have access to the Response header to calculate and modify the appropiate header ( "Transfer-type", "Content-length" )?

2. As per this entry, what is the wisest way to use to ensure safe and efficient ways of storing the entire body , across multiple child exec requests ( SSI Includes ). I would take a guess that as per you above mentioned cautionary notes, PER-CONNECTION would be the preffered method

3. Is there a Event handler that I could use to find out when the EndOfRequest has been reached?

Thank you again

Mario

re: Why some ISAPI Filter events trigger multiple times per request

David.Wang — Wed, 07 May 2008 07:03:20 GMT

Mario - yes, if you want to modify the outbound response, you have to buffer sufficient amount of entity body to make your modification, and depending on the response transfer method (either Content-Length, or Transfer-Encoding: chunked), you will have to make the appropriate modifications.

For example, your code will fail right now if the double \r\n is sent over two HTTP packets.

What you want to do is:

1. Buffer enough raw data over possibly multiple triggers of SF_NOTIFY_SEND_RAW_DATA to determine if the response header has "Content-Length:", "Transfer-Encoding: chunked", or neither header

2a. If Content-Length, then you must continue to buffer everything, perform your modifications, recalculate the new entity body length, and modify "Content-Length" to reflect the new length

2b. If Transfer-Encoding: chunked, then you can easily insert chunks if you're just inserting data. If modifying existing data, then you have to buffer and modify as in 2a, this time having to update the chunk's length header.

In all cases, your buffering has to be a sliding one that crosses two SF_NOTIFY_SEND_RAW_DATA because the thing you are looking to replace *may* come across two different event notifications -- for example, </ht comes on the first event and ml> comes on the second event, so if you want to replace </html>, you have to do this across the two events.

//David

re: Why some ISAPI Filter events trigger multiple times per request

Mario — Tue, 06 May 2008 20:02:23 GMT

Hi David,

I am a little confused with the l differences in PER-CONNECTION and PER_REQUEST. As mentioned, it is a matter of scope, so I am thinking that is I am writing a ISAPI Filter to change elements ( special tags ) in the reponse back to the client, and they are all sliced up into a series of includes ( SSI includes ) , then it would be wise to take the PER-REQUEST Route????

Here is some code, very simple, but it helps to try to show what I am doing. Thoughts on how to make the whole body of the response available, in an effiecent manner ?

Also in a prior blog entry you mentioned content-size is important to note. Does this mean that I am replacing these tags with lets say a paragraph or a few lines of text, I now have to change the response header ( content-length: ) as well? I would assume so.

#include <string>

#include <windows.h>

#include <stdio.h>

#include <stdlib.h>

#include <httpfilt.h>

BOOL WINAPI __stdcall GetFilterVersion(HTTP_FILTER_VERSION *pVer)

{

/* Specify the types and order of notification */

pVer->dwFlags = (SF_NOTIFY_SEND_RAW_DATA);

pVer->dwFilterVersion = HTTP_FILTER_REVISION;

strcpy_s(pVer->lpszFilterDesc, "PreProcessor filter, Version 1.0");

return TRUE;

}

DWORD WINAPI __stdcall HttpFilterProc(HTTP_FILTER_CONTEXT *pfc, DWORD NotificationType, VOID *pvData)

{

CHAR *in;

CHAR out[10000];

DWORD cbBuffer, cbtemp;

signed int newCharCount, contentStartPos;

switch (NotificationType) {

case SF_NOTIFY_SEND_RAW_DATA :

PHTTP_FILTER_RAW_DATA pResponse;

pResponse = (PHTTP_FILTER_RAW_DATA)pvData;

in = (CHAR*)pResponse->pvInData;

cbBuffer = 0;

//Skips the header.

for( ;cbBuffer<pResponse->cbInData-2;cbBuffer++){

if (in[cbBuffer] == '\n' && in[cbBuffer + 2] == '\n'){

cbBuffer +=3;

break;

}

//Verifies that the requested file is an html file.

for (cbtemp = 0; cbtemp < (cbBuffer - 3); cbtemp++)

{

if (in[cbtemp] == '/' && in[cbtemp + 1] == 'h' && in[cbtemp + 2] == 't' && in[cbtemp + 3] == 'm')

{

pfc->pFilterContext = (VOID *)2;

break;

}

if (cbtemp == cbBuffer)

pfc->pFilterContext = 0; /* not an html file */

//Process the file

if(pfc->pFilterContext){

newCharCount = 0;

contentStartPos = cbBuffer;

for ( ;cbBuffer < pResponse->cbInData-2;cbBuffer++){

if (in[cbBuffer] == '<' && in[cbBuffer + 1] == '&'){

for (cbtemp = cbBuffer+2; cbtemp < pResponse->cbInData-2; cbtemp++){

if (in[cbtemp] == '&' && in[cbtemp + 1] == '>'){

break;

}

cbBuffer = cbtemp + 2;

continue;

}else{

out[newCharCount] = in[cbBuffer];

newCharCount++;

}

pResponse->cbInData = contentStartPos + newCharCount+1;

for (cbtemp=0 ;cbtemp < newCharCount; cbtemp++){

in[cbtemp + contentStartPos] = out[cbtemp];

}

break;

default:

break;

}

return SF_STATUS_REQ_HANDLED_NOTIFICATION;

}

Thank you again David.

And if it helps, I think you blog and your reccomendations are both clear, helpful, and insightful.

re: Why some ISAPI Filter events trigger multiple times per request

David.Wang — Wed, 21 Jun 2006 02:57:49 GMT

JC - Which IIS version are you running, and are you running more than 1 ISAPI Filter at either global or site level.

In general, you want to designate your data as either per-request or per-connection and then manage its lifetime appropriately.
- Per-connection memory can be handled with pfc->AllocMem()
- Per-request memory can be handled with PreprocHeaders and Log/EndOfRequest

Also, you need to be aware that pfc->pFilterContext is PER-CONNECTION and NOT PER-REQUEST, which has special implications when servicing keep-alive or pipelined requests. For example:
- keep-alive requests mean that one connection and pFilterContext could be in use for 2 or more requests
- pipelined requests mean that one connection and pFilterContext could be used for 2 or more consecutive requests

Finally, you probably want to minimize the amount of memory creation/deletion done by an ISAPI Filter because it can fragment memory. Pre-allocating memory and then re-using is the preferred approach.

//David

re: Why some ISAPI Filter events trigger multiple times per request

JC — Tue, 20 Jun 2006 15:47:24 GMT

Thank you for your answer. The reason I asked about which events might not be fired is because I have some problems with a filter I've written.

In my filter, I use the filter context for storing data so I can have access to this data throughout the execution of the filter. The data is written to the context when the OnReadRawData event is fired. Then, when OnPreprocHeaders, OnUrlMap, OnSendRawData and OnLog is fired I read the data stored in the filter context. I delete the data from the filter context during the OnLog event.

The problem is that sometimes when OnPreprocHeaders and OnLog is fired, the data cannot be found in the filter context. This happens most often when the web server is under high stress and it seems to be more frequent when simple/static resourses like images and javascripts are requested. I have guessed that the reason for this is one of the following two:

1. OnPreprocHeaders and OnLog is sometimes fired before OnReadRawData
2. OnLog is fired more that once

Do you have any idea why the data is not found in the filter context?

/JC

re: Why some ISAPI Filter events trigger multiple times per request

David.Wang — Thu, 15 Jun 2006 00:43:17 GMT

JC - I am not aware of a scenario where OnLog executes more than once per request because logically, it should execute exactly once per request, just like OnPreprocHeaders. However, some error cases and OnReadRawData event can certainly affect this expectation...

Remember, ISAPI Filters directly manipulate IIS Server-Side behavior, which can change and break expected behaviors.

Listing the possible interactions that result in filter events not firing is beyond the scope of a blog comment. Do you have a specific question in mind.

//David

re: Why some ISAPI Filter events trigger multiple times per request

JC — Wed, 14 Jun 2006 15:15:45 GMT

Hi,

you mention that some requests can be executed more than one time. Do you know if the notification OnLog also can be executed more than once per request?

You state that 'There is no guarantee that every event fires at least once per request'. Does that mean that there are events that sometimes not will be executed/fired? Which events might not be executed?

Thanks in advance!