My travels with WDF : WDF

And here you thought I had a cloaking device

patman — Tue, 01 Dec 2009 23:48:00 GMT

Got a good one to share with the group;

Q: Can I make a kernel mode driver that opens a handle and talks to a UMDF based driver in another device stack?

A: Why yes you can! Quite simply done provide you follow all the rules. This is a variant of the UMDF initiated cross stack communication we talked about a while ago, but KMDF, being a DF (driver framework) supports essentially the same IoTarget model that UMDF does.

Now Playing – Rush Far Cry

The feline, it has escaped the fabric based containment pouch!

patman — Tue, 04 Aug 2009 23:09:00 GMT

One of the things I’ve been wanting to do for a while is put Isochronous transfer support back in to the 1394 Hybrid sample. Now that Windows 7 has shipped and we’re picking up projects we want to do during this period of product transitions, I felt the time was right to start this up. All happened to dove tail nicely with the implementation of Direct I/O support which was added to UMDF 1.9 in the end, allowing me to essentially kill two birds with one stone. What we’ll end up with is a demonstration for various methods of I/O flow through this hybrid driver stack as well as having another demonstration of Isochronous transfers for 1394 devices again. Hooray us!

So, I’m happy to report that work has begun on said project, and as a result I’ll have some tidbits, details and follies to present to you along this path of coding fun in the coming weeks.

Now Playing – Geddy Lee My Favorite Headache

Intellisense, the Musical part 3

patman — Thu, 04 Jun 2009 08:47:00 GMT

Like all good stories, sometimes the 2nd act is the hardest to follow. In this case it was simply a matter of other things coming up rather than writers block. :)

What kicked me in the rump to get this next part up was a few more questions about driver writing, WDK integration into Visual Studio and Intellisense. One of the very cool features of Visual Studio 2008 is the ability to make a project from existing code.

The end result can look something like this (warning, this is a 1920x1200 resolution PNG *g*)

I’ll walk you through this using the “a picture is worth a thousand words” mentality;

Select “Next”

Then point it to the root of the WDK installation. Pointing it to the root rather than right at the src folder will save you some intellsense headaches later on, sure you’ll get inclusion of all the WDK folders, but it’s actually not a bad thing in the end;

Select “Next”

Then;

Select “Finish” from here. We’re still not going to use Visual Studio to do any building for now, partly because I refuse to give up all command windows in my life and also because I’m still tinkering with all the settings required to use the WDK build environments under Visual Studio.

Now your Solution Explorer will probably look like a mess of files listed alphabetically. Simply hover over the “Show All Files” button at the top and toggle it until you get an Explorer like layout of the samples sources structure.

Now you have pointer to the first picture in the blog post. :)

As a side note to any Microsoft Employees who may actually read my blog – Yes, you can do the same thing with our Source Control Enlistments (I have a VS project for each branch I work out of). And if you have one of the Source Control plugs in for Visual Studio, you can also have all the check in / check out / history that our source control provides.

Genuine UMDF filtered drivers

patman — Thu, 16 Apr 2009 03:30:00 GMT

Filter drivers have come up in conversations recently (both internal and external), so I wanted to take some time here to address some of the issues that were brought up with regards to UMDF filter drivers and how to make them. Note: I’m not going to cover all the configuration dynamics available for UMDF filter drivers in this one post simply to avoid the risk information overload and clutter, so expect a couple of posts on this subject.

Let’s start with some basics, we’ll simply demonstrate how to configure and load a UMDF Filter as an upper filter driver to a kernel mode driver (KMDF or WDM) in the same INF-

As some of you may remember from my earlier posts, a UMDF filter driver should use the SetFilter(); property in the DeviceInitialization routine. This tells the Framework to do a couple of things for us automatically:

Send I/O for which the Filter has not registered a callback for to the next logical driver in the stack. Example: Your filter driver registers a DeviceIoctl callback method, but not read or write. As a result your filter driver will only see IOCTLs
Automatically forward file create cleanup and close so there is no need to invoke the AutoForwardCreateCleanupClose method.

That covers the really basic code required at driver / device initialization time. Now we need to add the pertinent sections to the INF file. As we’re going to focus on being an upper filter for this example, you need to make the following modifications to your INF (in addition to the normal UMDF specific sections);

[<mydriver>.NT.Wdf]
UmdfService = "<mydriver>", <mydriver>_Install
UmdfServiceOrder = <mydriver>

[<mydriver>_Device_AddReg]
; Load the redirector as an upperfilter on this specific device.
; 0x00010008 - FLG_ADDREG_TYPE_MULTI_SZ | FLG_ADDREG_APPEND
HKR,,"UpperFilters",0x00010008,"WUDFRd"

So a couple of things to note here -

One is the UmdfServiceOrder directive, this is load order specific. I’ll address a more UMDF centric stack in the next post, but for those of you who like to work ahead, this is a left to right reading list to determine stack order.

Second thing is the registry key addition section. As WUDFRd is the kernel mode transport service for UMDF drivers, WUDFRd is the upper level filter driver to a kernel mode driver.

Now that your driver is loaded as an upper filter, what should your driver do in order to be a good citizen in the stack? Short answer is, as little as possible. You perform the same steps for handling PnP and Power as you would for any basic UMDF driver (one that lets the framework handle the heavy lifting for those two). You register your I/O callback routines and build your queues just like any normal UMDF driver, the only additional requirement here is to forward I/O requests as required.

For this example, let’s just demonstrate using a fairly simple pass through driver. How you actually implement this is up to you, but the basic requirements here are - as your driver receives the request on the queue, it needs to pas pass it on to the next driver in the stack. So first you need to get the default I/O target (next driver in the stack). Next, format the request using the same type and then finally, send the request. A condensed version would be something along these lines;

    IWDFIoTarget * kmdfIoTarget = NULL;
    
    this->GetFxDevice()->GetDefaultIoTarget (&kmdfIoTarget);

    Request->FormatUsingCurrentType();

    hr = Request->Send (
        kmdfIoTarget, 
        0,  // 0 Submits Asynchronous else use WDF_REQUEST_SEND_OPTION_SYNCHRONOUS
        0);

I also have some more basic code here for you to peruse.

So now we have a basic UMDF upper filter driver on top of a kernel mode driver.

Now Playing Capitals v. Rangers

Think globally, lock at the smallest scope possible.

patman — Sat, 24 Jan 2009 02:26:00 GMT

Yes, I know that's a rip off of a ThinkGeek shirt (which I own).

Well, this is going to be my second post on this topic (okay, third), but as it managed to rear its head again recently, what better time to blog. And I'm going to also blog it simply to prove to Doron that I can provide something other than comic relief.

You'll notice a few of the existing UMDF samples have their locking constraint set to WdfDeviceLevel in their Device Initialization routines and some others don't set it at all, which defaults to WdfDeviceLevel.

So, what is the problem with doing it the way some of those samples are? Well, it does put the widest locking scope on all your objects that the device holds, meaning the device framework acquires a device presentation lock before passing some I/O to their driver callbacks. This results in serialized calls to that callback for your file and queue operations which is really unnecessary. There's a really good table in the Windows Driver Foundation book (pg 390) that shows what callbacks are serialized when using the Device Scope level. And to save some of you looking it up, I'll tell you now that yes, OnDeviceIoControl, OnRead, and OnWrite will get serialized if you set the locking constraint to Device Level.

So unless your driver really requires a synchronization scope that wide, initialize that constraint to None;

HRESULT
CVDevDevice::Initialize ( 
                         __in IWDFDriver           * FxDriver,
                         __in IWDFDeviceInitialize * FxDeviceInit)
{
    IWDFDevice * fxDevice = NULL;

    HRESULT hr = S_OK;

    FxDeviceInit->SetLockingConstraint (None);

    {
        IUnknown * unknown = this->QueryIUnknown ();

        hr = FxDriver->CreateDevice (FxDeviceInit, unknown, &fxDevice);

        unknown->Release ();
    }

    if (SUCCEEDED (hr)) 
    {
        m_FxDevice = fxDevice;
        fxDevice->Release();
    }

    return hr;
}

The primary differences here between KMDF and UMDF are UMDF doesn't set an execution level, so the likely hood of you bug checking by screwing this up is greatly reduced. :)

Edit: For those on RSS, sorry about the embedded style sheet nonsense, I keep forgetting to clear that tick box

*Currently playing - King's X Silent Wind

When it leaks, it pours

patman — Sat, 17 Jan 2009 00:09:00 GMT

Greetings from the Pacific Northwet. After our recent Snow-pocalypse, we also suffered a run of flooding. Amazingly enough, it wasn't directly due to the 13+ inches of snow I had at my house, it was a combination of the snow and the monumental amounts of rain that followed. Quite a few people were literally cutoff from the rest of Washington State as their egress roads were under rivers. The only hassle I had to suffer was one of FedEx being a day late with a new laptop. Oh bother.

The leaking I did have to suffer was of my own doing. It was my usage of IWDFIoRequest::GetCompletionParams in a test driver. In trying to be a good boy and clean up my I/O requests, and especially since I was going to be using asynchronous I/O, I wanted to get the completion parameters with out having to mess around with a completion routine. Funny thing is, that call takes a reference on the pointer returned. All was fine and dandy until I turned on Object leak detection (using WdfVerifier) and reference tracking. Debugger breaks prevent things from running smoothly I've found. :)

Now those of you much smarter than me would have told me I could have avoided that by using the CComPtr class on the CompletionParams interface as shown in the sample, but if you're like me and still think you know better and didn't use the CComPtr class, you had best make a subsequent Release() call.

pWdfRequest->GetCompletionParams (&CompletionParams);

hr = CompletionParams->GetCompletionStatus ();
CompletionParams->Release();

Now to answer a question nobody asked, yes my dog LOVES snow.

I'll spare you all part two of my FiOS TV installation rant because the guy who came out and installed the CableCard in my TiVO made up for the previous incident, ten fold. All I can say is wow. I really enjoyed watching Rush R30 in un-bitstuffed HD via the Palladia channel. Thank you FiOS.

*Currently playing - Porcupine Tree Blackest Eyes

How to find the droids you might be looking for

patman — Tue, 30 Sep 2008 01:36:00 GMT

Most of you at the DDC will notice that I'm not at the DDC. I'm guessing that makes me one of the "juniors" that Bob was referring to. ;)

4.) How do you build debugging techniques into your driver? Ie, DbgBreakPoint, ASSERT, etc - which is best in various situations?

This is one of those personal preference type questions. For the sake of lineal chatter, I'll just use ASSERT in my commentary. But when I say "ASSERT" you can make the determination on which one to use given your own working methods.

So that preamble aside, it's on with the show. I like to have staged text output via a debug trace level as a first tier technique. Something along these lines;

#define FILTERNAME        TEXT("mydriver")
#define DRIVERNAME        FILTERNAME TEXT(".sys")
#define DRIVERNAME_DBGOUT FILTERNAME TEXT(": ")

#if DBG
#define DBG_TRACE_VERBOSE      0
#define DBG_TRACE_MINIMAL      1
#define DBG_WARNING            2
#define DBG_ERROR              3
#define DBG_FATAL              4 

extern unsigned char DebugLevel;

#define DBG_PRINT( l, _x_ )                   \
    if( (l) >= DebugLevel )      \
    {                                         \
      KdPrint((DRIVERNAME_DBGOUT));         \
      KdPrint( _x_ );                       \
    }
#else  // DBG

#define DBG_PRINT( l,_x_ )

That allows me to compartmentalize failures and thusly allows me to build handlers for each. The question of what to do in each of those situations is another personal preference. As you can see from the error levels above, I follow patterns of building in handlers that fall in to - terminal, really annoying, nuisance, does it work and assuage.

I'll ASSERT more frequently for the nuisance issues, but also for the terminal, and some of the really annoying issues. Nuisance issues such as that HANDLE disappeared while I was using it, better ASSERT. This is usually when I find that the other thread using that handle wasn't synch'd properly...laugh all you want, you've done it too. :)

Using those levels as a guide, I craft an appropriate follow up action. But here I let the driver tell me what to do and it does that based on its end usage. For example;

  Status = IoCreateDevice (
      DriverObject, 
      sizeof (DEVICE_EXTENSION), 
      &DriverName,
      FILE_DEVICE_UNKNOWN, 
      DeviceType, 
      FALSE, 
      &DeviceObject);
  if (!NT_SUCCESS (Status)) 
  {
    DBG_PRINT (DBG_FATAL, TEXT(("Failed IoCreateDevice: 0x%x\n", Status)));
    ASSERT (DeviceObject);
    return Status;
  }

Would I use that ASSERT in a test driver? Not likely, all too often our test cases attempt to invoke failures in just those sort of areas and we run chk (debug, no_opt, etc.) builds in our labs frequently. So that ASSERT would be triggered constantly. You could also argue that in a shipping driver you would hit it in a lab under low resource simulation testing, but unless you plan on running low resource simulation 24/7, the nuisance factor is diminished greatly. But having the debug print there is a means for you to see if something happened in a retail / fre build of the driver. You can also wrap Windows logging methods with the same mentality such as WPP and ETW.

All of the above starts with the driver though. The more complex the driver, the more robust the debugging logic, so some of my internal test drivers have one or two ASSERT calls and maybe a couple of lines of debug print statements. So if you peek at the 1394vdev hybrid sample you can get a better idea of my patterning. The lack of ASSERT calls was a design choice based on the pattern established by the previous versions of that sample driver.

So for those of you at the DDC, enjoy the DDC! If you feel like making Peter laugh, ask him why I'm such a dork.

*Currently playing - Gravity Eyelids Porcupine Tree

It's all in the wrists

patman — Wed, 03 Sep 2008 20:46:00 GMT

I got a few interesting questions recently, which was helpful because I was having a tough time finding good material for you guys and gals of late. That's the problem with working on the next Operating System, I can't tell you about all the cool stuff yet. *g*

So here's today's question (in two parts);

1.) Do you use a single system to code, compile, verify and test a kernel driver?

Nope, pretty much, kinda sorta and nope. We're actually pretty wide open as far as coding environments around here. Across this team alone you'll find Visual Studio, gVIM, Slick Edit and probably 2 or 3 other IDEs in use (I've actually even seen notepad used once). The only coding requirements are really based around formatting, naming convention(s) and the like. Which is really fun when you get somebody who crafts a build script that goes out and checks for tabs rather than spaces in source code. ;)

Our compiler is a different story, pretty much everything we build is done so in very much the same environment shipped in the WDK. Yes there are some caveats, but that's only because I can't speak for every team and some of our internal process isn't really meant for general consumption. But our compiler is linked to our source control environment which makes wrapping a full one stop shopping coding / compiling / source control environment more of a challenge. There are a lot of great tools internally to get most of it done depending on your IDE of choice. Personally (if you haven't figured it out yet), I use Visual Studio as my source editor, a couple of internal tools to manage source control from within Visual Studio and then build in that external razzle environment. Like I said before, I'm still not prone to giving up some of my past. :)

Verification is another tricky question (somewhat open ended really). If we're talking about compile time code verification, then yes we do have a somewhat unified system in place. We use the same tools we ship in the WDK, PREfast and Static Driver Verifier. Our internal build system actually runs PREfast for us as a post build process.

Finally our testing is a real Pandora's Box type question. We do use the same tools we ship externally, Application Verifier, Driver Verifier and WDF Verifier, but we also develop other tools internally to cover scenarios we just can't reach using those tools. Here again, it's not a one stop shopping system across the Windows teams, but for those of you going to the DDC, Bob, Shyamal and Wei will be giving a shared secrets talk which will cover some more of what we do internally.

2.) How do you use virtualization software in the test and dev process?

This is my favorite one, I've been on a big push around here to get more people using virtual environments. Personally I've been using Virtual PC since 2005, I did some test work for them on that release, so I got in on the ground floor somewhat. Anyway, I use VMs to extend test coverage primarily on down level OSes. I keep "clean" installations of Windows XP SP1, SP2, SP3, Server 2003, Vista and Vista SP1 and some variants of those on my system. A lot more people are starting to use HyperV and Virtual PC around here, but given some of the limitations of those environments (e.g. inability to map USB hardware) and other little nuisances we can't use them for a full end to end test beds, this is a little more intrusive on the kernel side where you may require a piece of hardware. I know there are other virtual environments out there that do allow these features, but we're kind of prone to using our own products. :)

*Currently playing - King's X Alone