How many of you are familiar with the following scenario of ISAPI Filters on IIS:

  1. You have just obtained an ISAPI Filter DLL (either you compiled/linked it from source, or the binary was given or purchased)
  2. You figured out whether to configure the DLL as a global or site ISAPI Filter
  3. You restarted IIS to load the global ISAPI Filter (site ISAPI Filter load on-demand on first request for IIS6, immediately on prior IIS versions)

And... the filter does not appear to be loaded nor working. Now:

  1. On IIS6, failure to load/run any configured ISAPI Filter results in failing to start the website, which means that you will see "503 Service Unavailable" for the request
  2. On prior IIS versions, failure to load/run any configured ISAPI Filters are simply ignored by IIS, so it just looks like the filter is not working/running on the request

When you see the above situation when you are trying to install an ISAPI Filter, the best way to troubleshoot is to look in the Windows Event Log for error event entries from either W3SVC or W3SVC-WP regarding a failure to load the HTTP Filter DLL. The "data" of the error is the most important piece of information used to diagnose the failure to load/run an ISAPI Filter.

Here are some of the common error codes and resolutions (one big caveat mentioned at the end...):

Data: 02 00 00 00

Win32 error 2 - NET HELPMSG 2 returns "The system cannot find the file specified."

  • IIS was configured to load an ISAPI Filter, either per-website or globally across all websites, and the ISAPI Filter specified an ISAPI Filter DLL that does not exist.

    Prior to IIS6, this sort of configuration slipped by silently because IIS would ignore and not load the configured ISAPI Filter. However, IIS6 treats this sort of invalid configuration as fatal to the operation of the website or web server and will fail loading because of it (leading to a 503 Service Unavailable error)

    The right way to fix this issue is to determine if the ISAPI Filter is necessary for the operation of your web server. If it is not, then remove the ISAPI Filter; if it is necessary, then locate the correct ISAPI Filter DLL file and correct the configuration.

Data: 05 00 00 00

Win32 error 5 - NET HELPMSG 5 returns "Access is denied."

  • Since IIS uses the process identity to load and run an ISAPI Filter, it means that the process identity failed to have ACL to load the ISAPI Filter DLL. For IIS6 in worker process isolation mode, the process identity is configurable and is at least a member of the IIS_WPG group, while for IIS6 in IIS5 Compatibility Mode and prior IIS versions, the process identity is Local System. So, check for either deny or lack of allow ACLs for the process identity identity on the named ISAPI Filter DLL.
  • The ISAPI Filter DLL returned FALSE from GetFilterVersion() and GetLastError() returns ERROR_ACCESS_DENIED. The reason is completely arbitrary and depends on the ISAPI Filter implementation itself. For example, the ISAPI Filter may have tried to read/write a Registry key or file as the process identity, but unlike prior IIS versions, IIS6 runs as an unprivileged account by default that may fail to read/write with ERROR_ACCESS_DENIED, so the filter errors out by returning FALSE and leaving ERROR_ACCESS_DENIED as GetLastError().

Data: 7E 00 00 00

Win32 error 126 - NET HELPMSG 126 returns "The specified module could not be found."

  • The ISAPI Filter DLL is linked against some resource(s) that are not on the web server.

    One common reason for missing link dependencies is when your DLL is compiled as DEBUG and you do not have the DEBUG CRT/ATL/MFC DLLs on the web server. The DEBUG CRT/ATL/MFC DLLs are installed by Visual Studio, so your filter DLL may magically work on your dev machine but mysteriously "fail" on the production web server.

    Another common reason is when your DLL is linked against a newer version of CRT/ATL/MFC DLLs that are not present on the target OS. For example, DLLs compiled by Visual Studio .NET and later use CRT/ATL/MFC DLLs that are newer than available on the latest current version of Windows, Windows Server 2003. So, the DLL will magically work on your dev machine but mysterious "fail" on the production server.

    The easiest way to diagnose this issue is to use DEPENDS.EXE from the Windows Platform SDK (or from VS.Net) on the problematic DLL on the OS and see which dependency DLLs show up as red. Then, make sure those DLL dependencies are satisfied on the target OS (DEPENDS.EXE will report all dependency DLLs as green).

Data: 7F 00 00 00

Win32 error 127 - NET HELPMSG 127 returns "The specified procedure could not be found."

  • In order to load a DLL as an ISAPI Filter DLL, IIS expects certain method signatures to be exported from the DLL. You must export at least GetFilterVersion and HttpFilterProc (TerminateFilter is optional). Using MS LINK.EXE, you do this by specifying a .DEF file with the /DEF switch using the following content as the .DEF file.
    LIBRARY MyFilterName
    EXPORTS
        GetFilterVersion
        HttpFilterProc
  • If your ISAPI DLL uses an API call that is not actually present on the target OS, you can get this error. This is particularly insidious because you are able to compile/link the DLL successfully, yet the DLL fails to load on an OS which does not implement the API call you linked against.

    For example, suppose your ISAPI uses the RegGetValue() API call introduced in Windows Server SP1 (all platforms) and XP Pro x64. You download and use the latest Microsoft Platform SDK, which contains the advapi32.lib and include files for you to link and use RegGetValue(). Now, you try to run this ISAPI DLL on XP Pro 32bit, which has an older version of advapi32.dll that as documented, does not support RegGetValue(). The end result is error 127 since your ISAPI DLL will be looking for advapi32.dll to provide the RegGetValue() that it needs... which does not exist on XP Pro 32bit.

    The only way to resolve this is to use an API call that actually exists on the target OS. This is one reason why just because a piece of code compiles does NOT mean it actually works... ;-)

Data: Arbitrary Value

While using the data value to diagnose ISAPI Filter install failures usually works, there is one big caveat you need to know. The ISAPI Filter DLL itself can trigger the exact same event log entry with ANY data value, even for reasons unrelated to what I mention above.

  • IIS loads and initializes an ISAPI Filter by first calling LoadLibrary() on the configured filter DLL image file path, then verifying that at least GetFilterVersion() and HttpFilterProc() entrypoints exists, and finally invoking the GetFilterVersion() entrypoint. If this entry point returns FALSE, IIS will log the same "HTTP Filter failed to load" event log entry with WHATEVER value GetLastError() returns.

    In other words, an ISAPI Filter can generate that event if it succeeded or failed for arbitrary reason - as long as the filter returns FALSE, the last set error gets reported.

//David