A Hole In My Head

WDFREQUESTs are for sharing in KMDF v1.9

doronh — Fri, 06 Mar 2009 00:48:14 GMT

In my last post I described why a WDFREQUEST is unique to a particular WDFDEVICE. There is one particular programming pattern where this is not the behavior you want. This pattern is when you have each PDO accepting IO requests which it then forwards on to the parent WDFDEVICE for processing. One great in box example of this is usbhub.sys. Each usbhub PDO receives URBs which are then forwarded to the parent FDO and the FDO is where all IO processing occurs.

If you want to apply this pattern to a KMDF driver written to a v1.7 or earlier and take advantage of WDFQUEUEs you had to send the requests from the PDO to the FDO with WdfRequestSend so that they were represented to the FDO. The easiest way to do this is to create a WDFIOTARGET for the FDO itself and then have each PDO send IO to that WDFIOTARGET as shown in the following 3 code snippets.

EvtDriverDeviceAdd for the FDO

NTSTATUS EvtDriverDeviceAdd(WDFDRIVER Driver, PWDFDEVICE_INIT DeviceInit) 
{
   WDFDEVICE device;

   WDF_OBJECT_ATTRIBUTES woa;
   WDF_OBJECT_ATTRIBUTES_INIT_CONTEXT_TYPE(&woa, FDO_EXTENSION);

   // ...initialize the DeviceInit...
   status = WdfDeviceCreate(&DeviceInit, &woa, &device);

   if (!NT_SUCCESS(status)) {
      return status;
   }

   PFDO_EXTENSION pFdoExt = GetFdoExt(device);
   status = WdfIoTargetCreate(device, WDF_NO_OBJECT_ATTRIBUTES, &pFdoExt->SelfTarget);
   if (!NT_SUCCESS(status)) {
      return status;
   }

   // open the WDFIOTARGET to point our own PDEVICE_OBJECT
   WDF_IO_TARGET_OPEN_PARAMS openParams;
   WDF_IO_TARGET_OPEN_PARAMS_INIT_EXISTING_DEVICE(openParams, WdfDeviceWdmGetDeviceObject(device));

   status = WdfIoTargetOpen(pFdoExt->SelfTarget, &openParams);
   if (!NT_SUCCESS(status)) {
      return status;
   }
   ...
   return status;
}

EvtChildListCreateDevice for the PDO

NTSTATUS EvtChildListCreateDevice(
    WDFCHILDLIST ChildList, 
    PWDF_CHILD_IDENTIFICATION_DESCRIPTION_HEADER IdentificationDescription,
    PWDFDEVICE_INIT ChildInit
    )
{
   WDFDEVICE pdo;
   NTSTATUS status;

   WDF_OBJECT_ATTRIBUTES woa;
   WDF_OBJECT_ATTRIBUTES_INIT_CONTEXT_TYPE(&woa, PDO_EXTENSION);

   status = WdfDeviceCreate(&ChildInit, &woa, &device);
   if (!NT_SUCCESS(status)) {
      return status;
   }

   PPDO_EXTENSION pPdoExt = GetPdoExt(device);
   PFDO_EXTENSION pFdoExt = GetFdoExt(WdfPdoGetParent(device));

   pPdoExt->ParentTarget = pFdoExt->SelfTarget;
   ...
   return status;
}

EvtIoDefault for the PDO

typedef
VOID EvtIoDefault(WDFQUEUE Queue, WDFREQUEST Request)

{
    // ...  extract Request type ...
    if (RequestShouldBeSentToParent()) {
       WdfRequestFormatRequestUsingCurrentType(Request);
       WdfRequestSend(Request, GetPdoExt(WdfIoQueueGetDevice(Queue))->ParentTarget);
    }
}

As you can see, this is a bit cumbersome! While it works, it is not ideal. The KMDF team addressed this issue in v1.9 by adding 2 new DDIs that must be used together

WdfPdoInitAllowForwardingRequestToParent which tells KMDF that you will be forwarding IO from a PDO to the parent FDO. Internally this sets up some bookkeeping and, more importantly, sets the PDO’s PDEVICE_OBJECT’s StackSize to the FDO’s PDEVICE_OBJECT StackSize+1 so that there will be enough stack locations in the underlying PIRP for both the parent and child stacks
WdfRequestForwardToParentDeviceIoQueue which removes the need for the custom WDFIOTARGET and WdfRequestSend and directly presents the PDO’s WDFREQUEST to the FDO’s WDFQUEUE

Let’s now rewrite the 3 code snippets to make use of these new DDIs, new code in red

NEW EvtDriverDeviceAdd for the FDO

NTSTATUS EvtDriverDeviceAdd(WDFDRIVER Driver, PWDFDEVICE_INIT DeviceInit) 
{
   WDFDEVICE device;

   WDF_OBJECT_ATTRIBUTES woa;
   WDF_OBJECT_ATTRIBUTES_INIT_CONTEXT_TYPE(&woa, FDO_EXTENSION);

   // ...initialize the DeviceInit...
   status = WdfDeviceCreate(&DeviceInit, &woa, &device);

   if (!NT_SUCCESS(status)) {
      return status;
   }

   PFDO_EXTENSION pFdoExt = GetFdoExt(device);

   status = WdfIoTargetCreate(device, WDF_NO_OBJECT_ATTRIBUTES, &pFdoExt->SelfTarget);
   if (!NT_SUCCESS(status)) {
      return status;
   }

   // open the WDFIOTARGET to point our own PDEVICE_OBJECT
   WDF_IO_TARGET_OPEN_PARAMS openParams;
   WDF_IO_TARGET_OPEN_PARAMS_INIT_EXISTING_DEVICE(openParams, WdfDeviceWdmGetDeviceObject(device));

   status = WdfIoTargetOpen(pFdoExt->SelfTarget, &openParams);
   if (!NT_SUCCESS(status)) {
      return status;
   }



   // initialize a WDF_IO_QUEUE_CONFIG to your needs
   status = WdfIoQueueCreate(device, ..., &pFdoExt->>ChildProcessingQueue);
   if (!NT_SUCCESS(status)) {
      return status;
   }

   ...
   return status;
}

NEW EvtChildListCreateDevice for the PDO

NTSTATUS EvtChildListCreateDevice(
    WDFCHILDLIST ChildList, 
    PWDF_CHILD_IDENTIFICATION_DESCRIPTION_HEADER IdentificationDescription,
    PWDFDEVICE_INIT ChildInit
    )
{
   WDFDEVICE pdo;
   NTSTATUS status;

   WDF_OBJECT_ATTRIBUTES woa;
   WDF_OBJECT_ATTRIBUTES_INIT_CONTEXT_TYPE(&woa, PDO_EXTENSION);


   WdfPdoInitAllowForwardingRequestToParent(ChildInit);
   

   status = WdfDeviceCreate(&ChildInit, &woa, &device);
   if (!NT_SUCCESS(status)) {
      return status;
   }

   PPDO_EXTENSION pPdoExt = GetPdoExt(device);
   PFDO_EXTENSION pFdoExt = GetFdoExt(WdfPdoGetParent(device));

   pPdoExt->ParentTarget = pFdoExt->SelfTarget;
   ...
   return status;
}

NEW EvtIoDefault for the PDO

VOID EvtIoDefault(WDFQUEUE Queue, WDFREQUEST Request)

{
    // ...  extract Request type ...
    if (RequestShouldBeSentToParent()) {

       WdfRequestFormatRequestUsingCurrentType(Request);
       WdfRequestSend(Request, GetPdoExt(WdfIoQueueGetDevice(Queue))->ParentTarget);


       WDF_REQUEST_FORWARD_OPTIONS options;
       WDF_REQUEST_FORWARD_OPTIONS_INIT(&options);
       options.Flags = WDF_REQUEST_FORWARD_OPTIONS_FLAGS;


       status = WdfRequestForwardToParentDeviceIoQueue(
           Request, 
           GetFdoExt(WdfPdoGetParent(WdfIoQueueGetDevice(Queue)))->ChildProcessingQueue,
           &options);

    }
}

WDFREQUESTs are not for sharing

doronh — Thu, 05 Mar 2009 03:58:00 GMT

FYI: this is a bit of a long post, but I wanted to be thorough and illustrative and give some insight into how the framework works and potential design that could have been made, but were not for the sake of simplicity and performance

A common misconception a WDFREQUEST handle is the assumption that the WDFREQUEST handle value “follows” (e.g. stays the same) the PIRP around everywhere that the PIRP goes to. Basically, the idea is that everywhere that the PIRP is sent or presented, the same WDFREQUST handle will be used. The reality is that the same WDFREQUEST handle value will only be used while the PIRP is owned by the WDFDEVICE for which it was created. This means that if you send the WDFREQUEST to another driver with a call to WdfRequestSend (e.g. transfer ownership to the driver you are sending it to), the driver which receives the PIRP will have a different handle for the WFDREQUEST.

This means that the WDFREQUEST cannot be used to send extra data or buffers to the driver which will receive the sent request. For instance, this pattern does not work. First, in the sending driver (let's say request's handle value is 0xA) you format a request context, format the request and send it to the WDFIOTARGET

typedef struct _EXTRA_BUFFER {
   PVOID Data;
   ULONG DataLength;
   ...
} EXTRA_BUFFER, *PEXTRA_BUFFER;

PEXTRA_BUFFER pExtra = NULL;
WDF_OBEJCT_ATTRIBUTES woa;
WDF_OBJECT_ATTRIBUTES_INIT_CONTEXT_TYPE(&woa, EXTRA_BUFFER);
NTSTATUS status;

status = WdfObjectAllocateContext(request, &woa, (PVOID*) &pExtra);

// .. initialize pExtra ...
// ... format request as an internal IOCTL ...

if (WdfRequestSend(request, ...) == FALSE) {
   WdfRequestComplete(request, WdfRequestGetStatus(request));
}

And then in the receiving driver's WDFQUEUE dispatch routine would have a new WDFREQUEST handle value (let’s say 0xAA) and not the sender’s WDFREQUEST (0xA).

VOID EvtInternalDeviceIoControl(
    __in WDFQUEUE Queue,
    __in WDFREQUEST Request,
    __in size_t OutputBufferLength,
    __in size_t InputBufferLength,
    __in ULONG IoControlCode
    ) { ... }

Think about how KMDf would have been implemented if the same WDFREQUEST was presented to the lower driver. One of two things would have had to occur:

The first option would have been for the WDFREQUEST handle value to be smuggled somewhere in the PIRP (with the most likely candidate being DriverContext[]), but that would have been unreliable and prone to compatibility issues with non KMDF drivers in the stack.
The second option would have been to maintain a global mapping in KMDF that mapped from PIRP pointer value to WDFREQUEST handle. This would have been very expensive to maintain because we would have had to look up the mapping every time a WDFQUEUE handled PIRP arrived in the framework. It would have grown more even more expensive because this proposed map would have been guarded by a global lock which meant that all KMDF drivers would have been acquiring and releasing this one lock, making it very contentious and a huge performance problem.

Let’s take a step back and look at why there is a WDFREQUEST handle value when you send a request to another device. Here is what happens when a PIRP arrives in a KMDF driver

Call the potentially registered WDM preprocess routine
If it is a PIRP that will be processed by a WDFQUEUE (a read/write/IOCTL/internal IOCTL), allocate a WDFREQUEST for the PIRP. If it is not one of these types, pass it to the appropriate pipeline in the framework
Call the potentially registered in process callback
Pass the WDFREQUEST to the WDFQUEUE handler so that it will be presented on the appropriate WDFQUEUE
The WDFQUEUE calls the right IO event callback that you registered when the WDFQUEUE was created

Step #2 is the key here. The WDFREQUEST is always allocated (from a lookaside if that matters). It means that there is no 1:1 correspondence between a PIRP and WDFREQUEST value. Let’s say we wanted to have a 1:1 correspondence for a singular WDFDEVICE though(and not across multiple WDFDEVICE as in the first example). We would need a WDFDEVICE based mapping of active PIRPs to WDFREQUEST handles to see if the PIRP is an active PIRP in the WDFDEVICE. As in the case with the previously proposed global mapping, such a WDFDEVICE wide mapping would also be prohibitively expensive. While the lock contention would move from a global scope to a WDFDEVICE scope and thus reducing some contention, such a lock would still be a performance hot spot since all PIRPs arriving into the driver would be contending on this lock.

When you look at how KMDF allocates a WDFREQUEST it becomes clear that the WDFRQUEST loosely maps back to the current stack location in the PIRP, not the PIRP itself! Think about the sending the WDFREQUEST to another driver case. The sending driver has its own stack location and its own WDFREQUEST. The receiving driver has its own stack location and WDFREQUEST as well. Just to reinforce this idea, let’s consider a final example. Let’s say

Your driver received a WDFREQUEST formatted for IOCTL ‘A’ in your dispatch routine
In processing the request your driver formatted (the next stack location) it for IOCTL ‘B’
The request is sent to the top of your stack (which means your driver will see the PIRP in its dispatch routine eventually as IOCTL ‘B’).

When PIRP enters your driver as IOCTL 'B', it will have a new WDFREQUEST handle! This is completely by design. The first stack location (IOCTL ‘A’) has one WDFREQUEST, the subsequent stack location (IOCTL ‘B’) has another WDFREQUEST.

In conclusion, WDFREQUEST handles are local to a specific WDFDEVICE. In fact, they are local to a specific stack location. The context off of a WDFREQUEST can only be used by your driver for that particular WDFDEVICE and is not meant to be shared across WDFDEVICEs.

Great WinHEC presentation on device interfaces compared to device clases

doronh — Wed, 04 Feb 2009 04:03:00 GMT

[This is a repeat of a post I made to NTDEV, but I wanted to make sure I reached as many people as possible.]

I just read this deck, http://download.microsoft.com/download/5/E/6/5E66B27B-988B-4F50-AF3A-C2FF1E62180F/CON-T615_WH08.pptx, which was presented at WinHEC this past year. It is by far the best explanation of device interfaces and device classes that I have seen in 10 years. It worthwhile reading for both the new and experienced driver developer.

I was asked to enter a name and password to down the file, I hit cancel and it downloaded just fine. I don’t know if this was an IE8/Win7 quirk or not, so YMMV.

MSDN link on how to set up a user or kernel debugger

doronh — Thu, 29 Jan 2009 22:15:00 GMT

This has got to be one of the top FAQs out there: how do I set up a kernel debugger? I just stumbled across a link on MSDN which gives instructions not only on how to set up a kernel debugger on all transports (serial, 1394, usb2), but also how to set up a user mode debugger or how to attach to a virtual machine. Pretty cool. This used to be an internal web page at Microsoft (and I think a topic in debugger.chm), it is great that it is now public

The MSDN topic is Starting the Debugger. Add it to your bookmarks and maybe we can create room in the FAQ for another question ;)

Dude, what happened to Doron?

doronh — Wed, 28 Jan 2009 01:09:00 GMT

So, I have not written anything in over 6 months and yet I have posted on NTDEV and public newsgroups. What gives? Well, the short answer is that I have been short on time these past few months. I have had only enough extra curricular time to read NTDEV and occassionally the newsgroups. Why? Well, for starters we had a second baby this past June and that took the majority of my time ;). Combine that with parental leave and a project at work taking all of my time for nearly all of last year led to a very poor showing on the blog for 2008. I promise to change that this year.

Have a good new year and hopefully better quality drivers!

How do I cancel an IRP that another thread may be completing at the same time?

doronh — Mon, 30 Jun 2008 09:16:00 GMT

Let's say that you allocated a PIRP and sent it down your device stack. You free the PIRP in the completion routine and then return STATUS_MORE_PROCESSING_REQUIRED. To make life more fun, you decide that you want to be able to cancel the sent IRP after you have sent it so you try to do it simple like this

typedef struct _DEVICE_EXTENSION {
    KSPIN_LOCK SentIrpLock;
    PIRP SentIrp;
} DEVICE_EXTENSION;

Sending thread:

KeAcquireSpinLock(&devext->SentIrpLock, ...);
devext->SentIrp = Irp;
KeReleaseSpinLock(&devext->SentIrpLock, ...);

Canceling thread:

KeAcquireSpinLock(&devext->SentIrpLock, ...);
if (devext->AllocatedIrp != NULL) {
   IoCancelIrp(devext->SentIrp);
}
KeReleaseSpinLock(&devext->SentIrpLock, ...);

Completion routine:

PIRP irp;

KeAcquireSpinLock(&devext->SentIrpLock, ...);
irp = devext->SentIrp;

devext->SentIrp = NULL;
KeReleaseSpinLock(&devext->SentIrpLock, ...);

IoFreeIrp(irp);

return STATUS_MORE_PROCESSING_REQUIRED;

And it then deadlocks ;). If the call to IoCancelIrp causes the IRP to be completed in the calling context (e.g. the one which has acquired the lock), the completion routine will run and try to acquire the lock (SentIrpLock) on the same thread which holds it.

So, life is not that simple and you have to do something more. The basic solution is that you need extra state to track who is touching the PIRP and who can free it. Walter Oney's book has a solution (IIRC, it is in the self initiated I/O section, but I do not have the book handy), but IMHO it is a bit complicated. KMDF has a solution to this problem which I like much more (imagine that ;)).

You need an extra LONG, calling it CompletionCount per PIRP that you want to be able to cancel.

You initialize CompletionCount to 1 before sending it down the stack and storing it in devext.
Whenever there is a thread that wants to cancel the PIRP, it tries to interlocked increment CompletionCount only if and only if the current CompletionCount value is > 0. For this you need to roll your own InterlockedIncrementWithFloor which is fortunately not that hard and I have already shown you how to do that, http://blogs.msdn.com/doronh/archive/2006/12/06/creating-your-own-interlockedxxx-operation.aspx.
After the canceling thread has called InterlockedIncrementWithFloor and IoCancelIrp, it calls InterlockedDecrement.
Whomever wants to complete the PIRP, like the completion routine, interlock decrements CompletionCount.

If the returned value from InterlockedDecrement is zero, the caller can complete the PIRP. If not, somebody else is trying to touch the PIRP and you must leave the PIRP alone. So here is the revised code:

typedef struct _DEVICE_EXTENSION {
    KSPIN_LOCK SentIrpLock;
    PIRP SentIrp;
    ULONG CompletionCount;
} DEVICE_EXTENSION;

Sending thread:

KeAcquireSpinLock(&devext->SentIrpLock, ...);
devext->CompletionCount = 1;
devext->SentIrp = Irp;

KeReleaseSpinLock(&devext->SentIrpLock, ...);

Canceling thread:

PIRP irp = NULL;
KeAcquireSpinLock(&devext->SentIrpLock, ...);
if (devext->AllocatedIrp != NULL && MyInterlockedIcrementeWithFloor(&devext->CompletionCount, 0) > 0) {
   irp = devext->SentIrp;
}
KeReleaseSpinLock(&devext->SentIrpLock, ...);

if (irp != NULL) {
    IoCancelIrp(irp);
    if (InterlockedDecrement(&devext->CompletionCount) == 0) {
        IoFreeIrp(irp);
    }
}

Completion routine:

PIRP irp = NULL;

KeAcquireSpinLock(&devext->SentIrpLock, ...);
irp = devext->SentIrp;devext->SentIrp = NULL;
KeReleaseSpinLock(&devext->SentIrpLock, ...);

if (InterlockedDecrement(&devext->CompletionCount) == 0) {
    IoFreeIrp(irp);
}

return STATUS_MORE_PROCESSING_REQUIRED;

The beauty of this solution is that if you add more actors (let's say a timer for an async timeout, all you have to do is bump the CompletionCount to account for them to asynchronously rundown if you cannot cancel them.

What should you change in a sample before you ship it?

doronh — Wed, 23 Apr 2008 21:41:19 GMT

I was going to write about how to do this, but the awesome folks at WHDC got to it before I did. I did get to review it before it was published, so I did have some influence in what is in the tip ;). So on this one my job is easy, just go read the tip!

The WDF 1.7 cointstallers are now available

doronh — Fri, 18 Apr 2008 02:20:00 GMT

After a long wait (thank you for your patience!), the WDF 1.7 coinstallers are now up on the connect site. To get the bits

go to http://connect.microsoft.com
Log in using your passport account
Navigate to the WDF page (I don't know where it lives in the connection directory, sigh)
Choose Downloads on the left
The package is dated 4/17/2008, you may be able to get it directly from here once you have logged on

Enjoy and let the signing and shipping of v1.7 WDF drivers begin!

Debugger commands (.step_filter) that make my life easier

doronh — Thu, 17 Apr 2008 02:02:19 GMT

This is a pretty cool and somewhat obscure debugger command. It allows you to tell the debugger what functions to skip if you are using the trace command ('t'). I think of the trace command as the 'step into' command though, but that is just me. Let's say we have the following simple application:

#include <stdio.h>

struct Foo {
    Foo() : m_value(0) { }

    int Increment() { return ++m_value; }
    static void Print(int i) { printf("%d\n", i); }

    int m_value;
};

int _cdecl main(int argc, char *argv[])
{
    Foo f;
    Foo::Print(f.Increment());
    return 0;
}

If I were to run the program under the debugger and use the 't' command for each line, it would step into every function. I typically use 't' instead of 'p' because I usually want to step into a function at some point in time and I tend to press 'p' one too many times ;). Here is an example of the debugger session:

0:000> g test!main
>   13: {
0:000> t
>   14:     Foo f;
0:000> t
>    4:     Foo() : m_value(0) { }
0:000> t
>   15:     Foo::Print(f.Increment());
0:000> t
>    6:     int Increment() { return ++m_value; } [1]
0:000> t
>   15:     Foo::Print(f.Increment());
0:000> t
>    7:     static void Print(int i) { printf("%d\n", i); } [2]
0:000> gu
>   16:     return 0;
0:000> t
>   17: }
0:000> t
test!__mainCRTStartup+0x102:

Let's look at the statement Foo::Print(f.Increment()); When using the trace command, it will first step into Foo::Increment ([1]) before stepping into Foo::Print() ([2]). But let's say that I never want to step into Foo::Increment because I know that it is a simple function that I do not want to debug. I can tell the debugger to ignore trace commands into this function with the .step_filter command. The command takes a semi-colon delineated list of fully qualified symbol names (which can include wildcards so you can filter out entire modules) to ignore. Let's see the debugger session again with this command:

0:000> g test!main
>   13: {
0:000> .step_filter "test!Foo::Increment"
Filter out code symbols matching:
  test!Foo::Increment
0:000> t
>   14:     Foo f;
0:000> t
>    4:     Foo() : m_value(0) { }
0:000> t
>   15:     Foo::Print(f.Increment());
0:000> t
>    7:     static void Print(int i) { printf("%d\n", i); }
0:000> gu
>   16:     return 0;
0:000> t
>   17: }
0:000> t
test!__mainCRTStartup+0x102:

You will see now that when I trace into Foo::Print(f.Increment()); that the f.Increment() call is executed but not trace into (ignored is not the right word because it has run, I just didn't see it line by line) and I step directly into Foo::Print(). I think this is a pretty powerful debugger command, it can save you a lot of time if you are always accidentally stepping into the wrong function like I always do ;).

EvtDevicePreprocessWdmIrp is not entirely free

doronh — Thu, 03 Apr 2008 00:14:09 GMT

One of the WDM escapes in KMDF is EvtDeviceWdmIrpPreprocess (or EvtDevicePreprocessWdmIrp in the API in which you register it) which you can register for by calling WdfDeviceInitAssignWdmIrpPreprocessCallback. This function allows you to process a WDM PIRP before KMDF sees it and potentially processes it. From a KMDF adoption point of view, this functionaltiy was a very strong requirement. Without, there would be not be a defined way for a KMDF to support IRPs that KMDF did not natively support. For instance, the KMDF serial example registers a preprocess routine for IRP_MJ_FLUSH_BUFFERS, from [ddkroot]\src\kmdf\serial\pnp.c:

//
// Since framework queues doesn't handle IRP_MJ_FLUSH_BUFFERS,
// IRP_MJ_QUERY_INFORMATION and IRP_MJ_SET_INFORMATION requests,
// we will register a preprocess callback to handle them.
//
status = WdfDeviceInitAssignWdmIrpPreprocessCallback(
    DeviceInit,
    SerialFlush
    IRP_MJ_FLUSH_BUFFERS,
    NULL, // pointer minor function table
    0); // number of entries in the table

You should use this functionality in your driver only if this is the last resort you have; it does not come for free. When you register for any preprocess routine, KMDF will increase the StackSize in the underlying WDM PDEVICE_OBJECT. This is called out in the documention for "Preprocessing and Postprocessing IRPs" in KMDF. By increasing the stack size, all PIRPs that are sent to your device will have an extra IO stack location. This means that the PIRP is a bit larger than it would otherwise be; larger not only for the IRP_MJ code that you registered for, but for each and every PIRP that is sent to your device regardless of IRP_MJ code. Additionally, if your StackSize goes over an internal threshhold in the IO manager, the way the PIRP is allocated could be different (for instance, devices with a StackSize of <= N might have a PIRP allocated out of a lookaside list while a StackSize > N might always be allocated by calling ExAllocatePool each time). Mind you, the internal threshhold is a bit high so if you have a root enumerated device or your initial StackSize is low, this is not something that you will see.

So while registering a preprocess routine is a very useful thing to be able to do, you should consider your options carefully and only register a preprocess routine if you have no other option.

Using KeAcquireSpinLockAtDpcLevel is only a perf gain if you know you are DISPATCH_LEVEL

doronh — Sat, 29 Mar 2008 01:42:00 GMT

Well, that is certainly a long title ;).

First, let us look at an approximate implementation of KeAcquireSpinLock and KeRaiseIrql (and yes I know that KeRaiseIrql is really a #define to KfRaiseIrql, but it is the same thing that happens in the end…)

KIRQL KeAcquireSpinLock(PKSPIN_LOCK SpinLock, PKIRQL PreviousIrql)
{
    KeRaiseIrql(DISPATCH_LEVEL, PreviousIrql);
    [spin on the lock until it has been acquired]
}

VOID KeRaiseIrql(KIRQL NewIrql, PKIRQL, PKIRQL OldIrql)
{
    OldIrql = KeGetCurrentIrql();
    [raise IRQL to NewIrql]
}

What I want to emphasize is that KeAcquireSpinLock will retrieve the current IRQL (to know what to restore the IRQL to when the lock is released) as a part of acquiring the spin lock. Retrieving the current irql is a relatively expensive operation. Enter KeAcquireSpinLockAtDpcLevel. KeAcquireSpinLockAtDpcLevel does away with the IRQL change and just implements [spin on the lock until it has been acquired], but it does this with a large caveat…you must be running at DISPATCH_LEVEL (in reality it requires IRQL >= DISPATCH_LEVEL, but that is another discussion for another day) . It requires DISPATCH_LEVEL or higher so that you do not deadlock. Another caveat to effectively use KeAcquireSpinLockAtLevel you must know 100% that you are at DISPATCH_LEVEL. Naively, one could think that the following code optimizes for both cases

if (KeGetCurrentIrql() < DISPATCH_LEVEL) {
    KeAcquireSpinLock(&lock, &oldIrql);
}
else {
    KeAcquireSpinLockAtDpcLevel(&lock);
}

But the problem here is that in the case of the current IRQL < DISPATCH_LEVEL, the current IRQL is being retrieved twice (once in your code, once in KeAcquireSpinLock), so this relatively expensive operation is being performed twice when it should be performed only once. What all of this boils down to is that you must know with 100% certainty that the current IRQL is DISPATCH_LEVEL before you can use KeAcquireSpinLockAtDpcLevel effectively. Here are a couple of contexts here you can know for certain that the IRQL is DISPATCH_LEVEL.

In a DPC (for ISR, for a timer, your own)
After you have acquired a spin lock. In the case where you have nested locked being acquired (first A, then B), after you have called KeAcquireSpinLock(&A) you can then call KeAcquireSpinLockAtDpcLevel(&B)

Notice that a completion routine is not guaranteed to be called at IRQL == DISPATCH_LEVEL! While you may see that it is called at dispatch, it is not something that you can rely on 100% of the time. For instance, the lower driver could complete the IRP at passive level in an error condition.

Inconceivableable

doronh — Wed, 19 Mar 2008 02:55:00 GMT

I have no idea who created the name for PNP_DEVICE_NOT_DISABLEABLE, but I probably have the same reaction as you ... "seriously? that is what they named?" I mean come on, I think it could have at least been named PNP_DEVICE_CANNOT_BE_DISABLED. I am sure you can think of some better names too. If so, please leave a comment with your suggestions! While we had a chance to rectify this in KMDF in the WDF_DEVICE_STATE structure, we chose to keep the field name (NotDisableable) similar to the WDM name to avoid confusion.

For any readers who have not encountered this bit, it is a part of PNP_DEVICE_STATE. You set this bit in the IRP_MJ_PNP/IRP_MN_QUERY_PNP_DEVICE_STATE IRP after calling IoInvalidateDeviceState.

Once not disableable, forever not disableable

doronh — Tue, 18 Mar 2008 18:21:12 GMT

One interesting quirk about the PNP_DEVICE_NOT_DISABLEABLE state is that once it has been set and the PnP manager has processed it, the state is sticky. By sticky I mean that even if you attempt to clear this bit on a subsequent IRP_MN_QUERY_PNP_DEVICE_STATE IRP, the PnP manager ignores your changes to this state. This state remains stuck until any of the following occur

The machine is rebooted and the device is reenumerated
The device (or any device in its ancestry) is surprise removed
The device (or any device in its ancestry) is ejected

Returning failure from DriverEntry

doronh — Tue, 18 Mar 2008 02:56:00 GMT

One thing that is easily overlooked about implementing DriverEntry is that upon return !NT_SUCCESS, DriverUnload is not called. I mentioned this anecdotally in a previous post, but it is worth expanding on. I was bit by this oversight when I was working on the Bluetooth stack. Driver verifier correctly identified that my driver had leaked pool. The code looked something like this

// Globals
UNICODE_STRING gRegistryPath = { 0 };

NTSTATUS DriverEntry(PDRIVER_OBJECT DriverObject, PUNICODE_STRING RegistryPath)
{
    NTSTATUS status;

    DriverObject->DriverUnload = DriverUnload;

    gRegistryPath.Length = RegistryPath->Length;
    gRegistryPath.MaximumLength = RegistryPath->MaximumLength;
    gRegistryPath.Buffer = (PWCHAR) ExAllocatePoolWithTag(PagedPool, gRegistryPath.MaximumLength, [tag]);
    if (gRegistryPath.Buffer == NULL) { return STATUS_INSUFFICIENT_RESOURCES; }
    RtlCopyMemory(gRegistryPath.Buffer, RegistryPath->Buffer, gRegistryPath.Length);

    status = RegisterWithPortDriver(DriverObject, ...);
    if (!NT_SUCCESS(status)) { return status; } <== leak right here!

    // ... other init ...

    return status;
}

void DriverUnload(PDRIVER_OBJECT DriverObject)
{
    ExFreePool(gRegistryPath.Buffer);
    RtlZeroMemory(&gRegistryPath, sizeof(gRegistryPath));
}

While many WDM drivers do very little outside of initializing the dispatch table and other fields in their DriverObject, a miniport driver or a KMDF driver must register with their port driver (like ScsiPortInitialize) or framework (WdfDriverCreate) and this registration can introduce failure in DriverEntry (just like in my code sample above). What to do?

In a WDM driver you have to be very careful and manage this manually. Either you have a common error exit path out of DriverEntry which performs the cleanup (or manually calls your DriverUnload routine) or cleanup on each possible point of error. This pattern is very easy to get wrong and is not very maintainable, it is quite easy to add a new allocation and forget to cleanup it up later.

In a KMDF driver things are a bit easier to manage if you follow a particular pattern. While EvtDriverUnload has the same problems as the WDM DriverUnload, the EvtObjectCleanup routine registered on the WDFDRIVER is called in both scenarios. To re-emphasize, the EvtObjectCleanup registered on WDFDRIVER will be called when either DriverEntry returns !NT_SUCCESS or if your driver is gracefully unloaded later. This means that if you put all of your cleanup in the cleanup routine your DriverEntry implemention becomes much simpler. The one caveat is that the call to WdfDriverCreate must come before any allocations in your driver or state chaning APIs. WPP_INIT_TRACING is one such state changing API where you must undo its effects by calling WPP_CLEANUP. Quite a few WDK samples show this pattern (although suprisingly to me, not all!), let us look at the nonpnp sample (%wdk%\src\kmdf\nonpnp\sys\nonpnp.c)

NTSTATUS
DriverEntry(
    IN OUT PDRIVER_OBJECT   DriverObject,
    IN PUNICODE_STRING      RegistryPath
    )
{
    NTSTATUS                       status;
    WDF_DRIVER_CONFIG              config;
    WDFDRIVER                      hDriver;
    PWDFDEVICE_INIT                pInit = NULL;
    WDF_OBJECT_ATTRIBUTES          attributes;

    WDF_DRIVER_CONFIG_INIT(&config, WDF_NO_EVENT_CALLBACK);

    // Tell the framework that this is non-pnp driver so that it doesn't set the default AddDevice routine.
    config.DriverInitFlags |= WdfDriverInitNonPnpDriver;

    // NonPnp driver must explicitly register an unload routine for the driver to be unloaded.
    config.EvtDriverUnload = NonPnpEvtDriverUnload;

    // Register a cleanup callback so that we can call WPP_CLEANUP when
    // the framework driver object is deleted during driver unload.
    WDF_OBJECT_ATTRIBUTES_INIT(&attributes);
    attributes.EvtCleanupCallback = NonPnpEvtDriverContextCleanup;

    status = WdfDriverCreate(DriverObject,
                            RegistryPath,
                            &attributes,
                            &config,
                            &hDriver);
    if (!NT_SUCCESS(status)) {
        KdPrint (("NonPnp: WdfDriverCreate failed with status 0x%x\n", status));
        return status;
    }

    // Since we are calling WPP_CLEANUP in the DriverContextCleanup
    // callback we should initialize WPP Tracing after WDFDRIVER
    // object is created to ensure that we cleanup WPP properly
    // if we return failure status from DriverEntry. This
    // eliminates the need to call WPP_CLEANUP in every path
    // of DriverEntry.
    WPP_INIT_TRACING( DriverObject, RegistryPath );

    ...

    return status;
}

VOID NonPnpEvtDriverContextCleanup(WDFDRIVER Driver)
{
    WPP_CLEANUP(WdfDriverWdmGetDriverObject(Driver));
}

The red comments show what is going on. Hopefully the code is self explanatory.

Incidentally, another pattern you can use for global memory allocations is to allocate the memory with WdfMemoryCreate without specifying a parent object. The WDFDRIVER will be the parent object by default and since all child objects are destroyed when the parent is destroyed, all of your allocations will be destroyed after EvtDriverUnload has been called when the WDFDRIVER is destroyed in the unload path.

one of the books that started it all...

doronh — Wed, 12 Mar 2008 18:38:00 GMT

During my sophomore year at Cal Poly, I decided that I wanted to learn about threads, synchronization techniques and other topics associated modern operating systems. Windows 95 had made its debut (yes, it is not a modern OS, but I didn't know that at the time!) and I had heard about Windows NT, but had never seen or used it. Since I had Win95 on my machine at home, I decided that I would go to the bookstore and buy a book on threading.

Well, needless to say there was not a book that was dedicated to threading, but I found a couple of books on Windows programming that included threading as topics. It came down to 2 books, Jeffrey Richter's "Advanced Windows for Win95 and NT" (with Napoleon on the cover) and some other book. I obviously picked Richter's book ;), although now it is called "Windows via C/C++."

This book was, and still is, awesome. I have both the original and new editions. Great detail, very easy to read and met it my needs perfectly (unlike "Inside OLE" by Kraig Brockschmidt which was full of information, I just could not stay awake reading it :) ). IMHO, it successfully started me down the road to becoming a well rounded developer and I think it helped in getting my foot in the door at Microsoft. I have even had the pleasure of letting Jeff know personally what a great book it is and how it shaped me for years to com.