My travels with WDF : KMDF

And here you thought I had a cloaking device

patman — Tue, 01 Dec 2009 23:48:00 GMT

Got a good one to share with the group;

Q: Can I make a kernel mode driver that opens a handle and talks to a UMDF based driver in another device stack?

A: Why yes you can! Quite simply done provide you follow all the rules. This is a variant of the UMDF initiated cross stack communication we talked about a while ago, but KMDF, being a DF (driver framework) supports essentially the same IoTarget model that UMDF does.

Now Playing – Rush Far Cry

The feline, it has escaped the fabric based containment pouch!

patman — Tue, 04 Aug 2009 23:09:00 GMT

One of the things I’ve been wanting to do for a while is put Isochronous transfer support back in to the 1394 Hybrid sample. Now that Windows 7 has shipped and we’re picking up projects we want to do during this period of product transitions, I felt the time was right to start this up. All happened to dove tail nicely with the implementation of Direct I/O support which was added to UMDF 1.9 in the end, allowing me to essentially kill two birds with one stone. What we’ll end up with is a demonstration for various methods of I/O flow through this hybrid driver stack as well as having another demonstration of Isochronous transfers for 1394 devices again. Hooray us!

So, I’m happy to report that work has begun on said project, and as a result I’ll have some tidbits, details and follies to present to you along this path of coding fun in the coming weeks.

Now Playing – Geddy Lee My Favorite Headache

Think globally, lock at the smallest scope possible.

patman — Sat, 24 Jan 2009 02:26:00 GMT

Yes, I know that's a rip off of a ThinkGeek shirt (which I own).

Well, this is going to be my second post on this topic (okay, third), but as it managed to rear its head again recently, what better time to blog. And I'm going to also blog it simply to prove to Doron that I can provide something other than comic relief.

You'll notice a few of the existing UMDF samples have their locking constraint set to WdfDeviceLevel in their Device Initialization routines and some others don't set it at all, which defaults to WdfDeviceLevel.

So, what is the problem with doing it the way some of those samples are? Well, it does put the widest locking scope on all your objects that the device holds, meaning the device framework acquires a device presentation lock before passing some I/O to their driver callbacks. This results in serialized calls to that callback for your file and queue operations which is really unnecessary. There's a really good table in the Windows Driver Foundation book (pg 390) that shows what callbacks are serialized when using the Device Scope level. And to save some of you looking it up, I'll tell you now that yes, OnDeviceIoControl, OnRead, and OnWrite will get serialized if you set the locking constraint to Device Level.

So unless your driver really requires a synchronization scope that wide, initialize that constraint to None;

HRESULT
CVDevDevice::Initialize ( 
                         __in IWDFDriver           * FxDriver,
                         __in IWDFDeviceInitialize * FxDeviceInit)
{
    IWDFDevice * fxDevice = NULL;

    HRESULT hr = S_OK;

    FxDeviceInit->SetLockingConstraint (None);

    {
        IUnknown * unknown = this->QueryIUnknown ();

        hr = FxDriver->CreateDevice (FxDeviceInit, unknown, &fxDevice);

        unknown->Release ();
    }

    if (SUCCEEDED (hr)) 
    {
        m_FxDevice = fxDevice;
        fxDevice->Release();
    }

    return hr;
}

The primary differences here between KMDF and UMDF are UMDF doesn't set an execution level, so the likely hood of you bug checking by screwing this up is greatly reduced. :)

Edit: For those on RSS, sorry about the embedded style sheet nonsense, I keep forgetting to clear that tick box

*Currently playing - King's X Silent Wind

How to find the droids you might be looking for

patman — Tue, 30 Sep 2008 01:36:00 GMT

Most of you at the DDC will notice that I'm not at the DDC. I'm guessing that makes me one of the "juniors" that Bob was referring to. ;)

4.) How do you build debugging techniques into your driver? Ie, DbgBreakPoint, ASSERT, etc - which is best in various situations?

This is one of those personal preference type questions. For the sake of lineal chatter, I'll just use ASSERT in my commentary. But when I say "ASSERT" you can make the determination on which one to use given your own working methods.

So that preamble aside, it's on with the show. I like to have staged text output via a debug trace level as a first tier technique. Something along these lines;

#define FILTERNAME        TEXT("mydriver")
#define DRIVERNAME        FILTERNAME TEXT(".sys")
#define DRIVERNAME_DBGOUT FILTERNAME TEXT(": ")

#if DBG
#define DBG_TRACE_VERBOSE      0
#define DBG_TRACE_MINIMAL      1
#define DBG_WARNING            2
#define DBG_ERROR              3
#define DBG_FATAL              4 

extern unsigned char DebugLevel;

#define DBG_PRINT( l, _x_ )                   \
    if( (l) >= DebugLevel )      \
    {                                         \
      KdPrint((DRIVERNAME_DBGOUT));         \
      KdPrint( _x_ );                       \
    }
#else  // DBG

#define DBG_PRINT( l,_x_ )

That allows me to compartmentalize failures and thusly allows me to build handlers for each. The question of what to do in each of those situations is another personal preference. As you can see from the error levels above, I follow patterns of building in handlers that fall in to - terminal, really annoying, nuisance, does it work and assuage.

I'll ASSERT more frequently for the nuisance issues, but also for the terminal, and some of the really annoying issues. Nuisance issues such as that HANDLE disappeared while I was using it, better ASSERT. This is usually when I find that the other thread using that handle wasn't synch'd properly...laugh all you want, you've done it too. :)

Using those levels as a guide, I craft an appropriate follow up action. But here I let the driver tell me what to do and it does that based on its end usage. For example;

  Status = IoCreateDevice (
      DriverObject, 
      sizeof (DEVICE_EXTENSION), 
      &DriverName,
      FILE_DEVICE_UNKNOWN, 
      DeviceType, 
      FALSE, 
      &DeviceObject);
  if (!NT_SUCCESS (Status)) 
  {
    DBG_PRINT (DBG_FATAL, TEXT(("Failed IoCreateDevice: 0x%x\n", Status)));
    ASSERT (DeviceObject);
    return Status;
  }

Would I use that ASSERT in a test driver? Not likely, all too often our test cases attempt to invoke failures in just those sort of areas and we run chk (debug, no_opt, etc.) builds in our labs frequently. So that ASSERT would be triggered constantly. You could also argue that in a shipping driver you would hit it in a lab under low resource simulation testing, but unless you plan on running low resource simulation 24/7, the nuisance factor is diminished greatly. But having the debug print there is a means for you to see if something happened in a retail / fre build of the driver. You can also wrap Windows logging methods with the same mentality such as WPP and ETW.

All of the above starts with the driver though. The more complex the driver, the more robust the debugging logic, so some of my internal test drivers have one or two ASSERT calls and maybe a couple of lines of debug print statements. So if you peek at the 1394vdev hybrid sample you can get a better idea of my patterning. The lack of ASSERT calls was a design choice based on the pattern established by the previous versions of that sample driver.

So for those of you at the DDC, enjoy the DDC! If you feel like making Peter laugh, ask him why I'm such a dork.

*Currently playing - Gravity Eyelids Porcupine Tree

It's flick, then swish

patman — Thu, 11 Sep 2008 23:17:00 GMT

So the next part of the little Q&A series on...well, QA;

3.) What security considerations do you take into account when designing and developing a kernel mode driver?

Validate user mode pointers and check the buffers to and from user mode. ;)

That was a little inside joke for the person who asked the question. Seriously security and kernel mode are almost at odds with one another when you really think about it. Kernel mode is implied to be a trustworthy environment so anybody that's down there with you is expected to behave like a good citizen. And anybody that can talk to you from the nether region (user mode) should be treated as an enemy. As security for kernel mode drivers is something of a loaded question in the end, the best advice I can offer here is; Anything that can be accessed by anybody outside your driver should have some form of defense built in to it. I.e. if you have an external callback, think about how that that could be used as an attack entry point. Could somebody float recursive calls to that callback and cause a stack over flow?

That's about the only weird tip I can give other than using what's published here and here as a base.

And finally, my apologize to Bob, I guess I stole some of his thunder. :)

*Currently playing - Rush Faithless

It's all in the wrists

patman — Wed, 03 Sep 2008 20:46:00 GMT

I got a few interesting questions recently, which was helpful because I was having a tough time finding good material for you guys and gals of late. That's the problem with working on the next Operating System, I can't tell you about all the cool stuff yet. *g*

So here's today's question (in two parts);

1.) Do you use a single system to code, compile, verify and test a kernel driver?

Nope, pretty much, kinda sorta and nope. We're actually pretty wide open as far as coding environments around here. Across this team alone you'll find Visual Studio, gVIM, Slick Edit and probably 2 or 3 other IDEs in use (I've actually even seen notepad used once). The only coding requirements are really based around formatting, naming convention(s) and the like. Which is really fun when you get somebody who crafts a build script that goes out and checks for tabs rather than spaces in source code. ;)

Our compiler is a different story, pretty much everything we build is done so in very much the same environment shipped in the WDK. Yes there are some caveats, but that's only because I can't speak for every team and some of our internal process isn't really meant for general consumption. But our compiler is linked to our source control environment which makes wrapping a full one stop shopping coding / compiling / source control environment more of a challenge. There are a lot of great tools internally to get most of it done depending on your IDE of choice. Personally (if you haven't figured it out yet), I use Visual Studio as my source editor, a couple of internal tools to manage source control from within Visual Studio and then build in that external razzle environment. Like I said before, I'm still not prone to giving up some of my past. :)

Verification is another tricky question (somewhat open ended really). If we're talking about compile time code verification, then yes we do have a somewhat unified system in place. We use the same tools we ship in the WDK, PREfast and Static Driver Verifier. Our internal build system actually runs PREfast for us as a post build process.

Finally our testing is a real Pandora's Box type question. We do use the same tools we ship externally, Application Verifier, Driver Verifier and WDF Verifier, but we also develop other tools internally to cover scenarios we just can't reach using those tools. Here again, it's not a one stop shopping system across the Windows teams, but for those of you going to the DDC, Bob, Shyamal and Wei will be giving a shared secrets talk which will cover some more of what we do internally.

2.) How do you use virtualization software in the test and dev process?

This is my favorite one, I've been on a big push around here to get more people using virtual environments. Personally I've been using Virtual PC since 2005, I did some test work for them on that release, so I got in on the ground floor somewhat. Anyway, I use VMs to extend test coverage primarily on down level OSes. I keep "clean" installations of Windows XP SP1, SP2, SP3, Server 2003, Vista and Vista SP1 and some variants of those on my system. A lot more people are starting to use HyperV and Virtual PC around here, but given some of the limitations of those environments (e.g. inability to map USB hardware) and other little nuisances we can't use them for a full end to end test beds, this is a little more intrusive on the kernel side where you may require a piece of hardware. I know there are other virtual environments out there that do allow these features, but we're kind of prone to using our own products. :)

*Currently playing - King's X Alone

Blue Moon, you saw me standing alone.....

patman — Fri, 04 Apr 2008 21:28:35 GMT

I love sharing simple mistakes for some reason. I think it's because we're all prone to searching for how to do something simple, like handle METHOD_NEITHER requests in our KMDF driver, and only read the documentation on how to initialize that ability, but not remembering all the little rules that govern the surrounding APIs. Oft leaving us with a series of bug checks, driver load failures or head scratching "huh" scenarios.

This not so good -

    status = WdfDeviceCreate(&DeviceInit, &deviceAttributes, &device);
    if (!NT_SUCCESS(status)) 
    {
        KdPrint( ("WdfDeviceCreate failed with status code 0x%x\n", status));
        return status;
    }
    <...some other code went here...>
    WdfDeviceInitSetIoInCallerContextCallback (
        DeviceInit,
        SomeEvtInContextCallBack);

This good -

    WdfDeviceInitSetIoInCallerContextCallback (
        DeviceInit,
        SomeEvtInContextCallBack);

    status = WdfDeviceCreate(&DeviceInit, &deviceAttributes, &device);
    if (!NT_SUCCESS(status)) 
    {
        KdPrint( ("WdfDeviceCreate failed with status code 0x%x\n", status));
        return status;
    }

See, at some point in history, I had paged out of my grey matter memory module that tidbit in the MSDN documentation for WdfDeviceCreate which says; After the driver calls WdfDeviceCreate, it can no longer access the WDFDEVICE_INIT structure.

Boy oh boy you sure can't access it afterwards, that bad boy is a glorious, awe inspiring, blue screening NULL after WdfDeviceCreate is through!

How to jiggle the handle(s) and get your 1.7 to work in Vista

patman — Wed, 05 Mar 2008 23:24:00 GMT

As people who write drivers are something of a rare breed, I know we probably share a lot of common readers between our Driver blogs and via the OSR boards, but just in case the three or four of you who read my blog haven't seen this, I wanted to make sure it got out there.

We had a bit of a stumble with the 1.7 coinstaller package that went out with the WDK recently.

We're pretty close to being done with a fix as Bob said, so fret not. I'd like to thank those who pestered me about the issue for helping narrow it down, and also those who pestered the external boards and email alias.

Now, if we could all join hands around the camp fire and sing "row, row, row your boat" to complete this little moment of community unity.

Still working on the Visual Studio stuff, but those of you with free time are more than welcome to skip ahead.

*Currently playing - Not a single thing...

d'OH-nuts

patman — Mon, 10 Dec 2007 23:21:41 GMT

As promised here is the correction to the hybrid INF file you'll need to make.

[SourceDisksFiles]
kmdf1394vdev.sys=1
umdf1394vdev.dll=1
WudfUpdate_01007.dll=1
~~WdfCoInstaller01005.dll=1~~ WdfCoInstaller01007.dll=1

....

[CoInstaller_CopyFiles]
~~wdfcoinstaller01007.dll~~ wdfcoinstaller01007.dll
WudfUpdate_01007.dll

...

[CoInstaller_AddReg]
HKR,,CoInstallers32,0x00010000,"~~wdfcoinstaller01005.dll~~,WdfCoInstaller","WUDFUpdate_01007.dll" wdfcoinstaller01007.dll

...

[KMDF1394VDEV_wdfsect]
KmdfLibraryVersion = ~~1.5~~ 1.7

Apologies for letting that get out in the wild. :)

it's full of stars....

patman — Fri, 21 Sep 2007 02:31:00 GMT

First thing is first, NHL '08 is miserable on Superstar mode. You actually have to play hockey defense. Not just jump up and slam somebody in to the boards. And whatever you do, don't make any mistakes passing in your defensive zone. 99.99999999% of them end up in the back of your net.

Second thing would be, well, second. Astronomy is brutal to your pocket book. I recently acquired a Meade® 8" LX200®R telescope and after talking to a few friends who are further down that slippery slope, it's become apparent that I may need to sell almost everything else to continue this habit. And here I thought photography was bad. Try Astrophotography, which I think is an acronym for Astronomically expensive Photography, it was very cool seeing some of the larger bands on Jupiter and the 4 primary Moons on that first night, but I'm not sure I'm ready to give up my house, yet.

On with our normal show.

I wanted to answer part of a question that has been thrown around by a few people; Why can't you do the 1394 virtual device or a 1394 function driver as a full UMDF driver?

Well the biggy right now is really the 1394 I/O Request Block (IRB) prevents us from doing so. Since the individual 1394 Bus I/O requests are tied in the union within the IRB, we are bound to use that structure. Not such a big deal for UMDF drivers really, until you dig deeper in to the nested elements of the specific requests. Let's take REQUEST_ALLOCATE_ADDRESS_RANGE. That first element in the structure is gonna kill any hope for UMDF right now. MDLs. It is something we plan on addressing, but for right now, we're kind of at an impasse for using IRBs.

So what you'll see in the sample (and in the current 1394 vdev samples as well) are those I/O request structures replicated with 33% more User Mode compliant objects at no additional cost to you!

The same is sort of true for USB which uses nested elements within an URB.

Now isn't this fun? IRB, URB, IRP, MDL. I'm sure Peter has been hearing some of terms these from his newborn, but probably just before a giggle or a burp happens. ;)

And in closing, your "ha-ha" moment to live by today. Apparently comments have formed an alliance w/ the ASCII character "Space". A simple script copied a file to a location in a post build operation. Except it prepended the file name with a space. The post-post build script kept generating errors which caused much consternation in my office, yet again. LUCKILY being a devoted command window user, when I attempted to open the file in notepad with the quick insert command, I noticed the file name was in quotes with that leading space.

It's now a full two pronged attack against me from comments and the Space key. If they manage to ally themselves with the semi-colon, our world may be at risk.

That boy's not right in the head

patman — Tue, 11 Sep 2007 22:18:04 GMT

Holy jumpin'! Some of you might be thinking, "Another blog post this month?" Yessir, I've got another one, and yes, it's another rambler.

See I was pondering last night while watching The Universe: Beyond the Big Bang on my DVR and much like last week, I had something of an epiphany. I started this blog with the intent of publishing entries about the gotchas, snake pits and falling rocks of a WDF newbie writing a WDF hybrid sample driver for the Windows Driver Kit.

I guess I had expected there to be a few more of these along the way, and thankfully, or sadly depending on how you look at it, I really didn't find that many which were not my own fault (curse you and your fuzzy green code, commenting!). So in something of a pre-post mortem, I wanted to determine why that might be; Some of it is down to the driver not exploring the really dark recesses of the Frameworks space as yet, and some of it is down to the Frameworks being really easy to use in the context I was using them in.

In the end, I kind of feel bad that to this point that my blog wasn't filled more with entries detailing those little bumps and bruises we all experience while programming something beyond "Hello World". But in a weird way, it also makes me happy there wasn't more I had to write. And as some of you have determined by now, yes, I am crazy in that manner. So let's think of this little blog as something of a Voyager (or V'ger if you're so inclined. Sorry, is my geek showing through these glasses?).

So going forward, I'll be taking those questions you all toss at me and sharing the answers with the group, and sharing those tidbits of deeper WDF exploration as we all dive further in to what the WDF really is capable of providing. There will be no shortage of driver "stuff" for me to blog on going forward, that's for sure.

Maybe we can also pool our resources and send Bob some real out there issues to debug, or pester Doron with more "why did you do this like this" questions. Because as soon as NHL '08 is released, I'm sure you'll not hear from me for a week or two. :)

UPDATE: I'd like to say a "So long and thanks for all the Bulk, Isoch and Interrupt" as the computing world lost a great engineer recently, Brad Hosler.

For those of of you who don't know, Brad was a driving force in the creation of USB 1.0, the original USB Logo Compliance program, as well as a founding father of USB 2.0 and Wireless USB. He was also a part of the team that defined PCI and served as Chair of the PCI-SIG Serial Working Group. He was very much a key figure in enabling such rapid and stable adoption of USB across the industry and his talks at USB events were always fun. :)

When comments attack

patman — Wed, 08 Aug 2007 23:44:00 GMT

First, let's say hello to BOB! Bob's from the KMDF side of the fence, but I still have to yell at him to get off my lawn from time to time. *shakes fist*

Secondly, another 'ha-ha' moment for you live by; I've been testing the hybrid driver as individual elements, just to get some error path coverage because I'm weird like that. That means I was installing the KMDF driver on its own and running unit tests against it, then I was installing the UMDF driver on its own and running unit test against that.

To do that I had commented out some device initialization elements in the UMDF object. Primarily the SetFilter(); property. Not setting that property tells the framework to handle a CreateFile(); request with out sending it down to any subsequent kernel driver...not so good when your KMDF object is a function driver.

This caused a little consternation on my part when I put the two drivers together for the last rounds of testing. All my request submission from the UMDF driver were being bounced back by the framework with an "invalid handle" error message. I spent a fair bit of time digging for answers on this problem; some fun debugging, some R rated tirades and such, only to have somebody point out the obvious in the end. "Did you set the SetFilter(); property?"

So be wary, they may seem like timid, friedly little lines of green code, but comments can also be silent killers...(cue - Theme to Jaws).

*edit because I just realized I used the same joke twice! *hangs head in shame*

An inf in the hand is worth two in the bush.

patman — Wed, 28 Mar 2007 19:53:00 GMT

A laurel and hearty hello to all of you (2 points if you get the reference).

Before I start on the meat of the post, I want to cover something that came up in conversation recently in regards to creating hybrid driver stacks using UMDF; Your kernel mode object does not have to be KMDF. You can use an existing WDM driver just as easily with out any modifications to the driver.

Now, on w/ the show, I wanted to cover some really geeky code stuff in this post, but it's not quite ready for prime time so you'll have to suffer with a post about inf files.

The first hurdle to really cover in getting this sample driver off the ground is figuring out what goes where in our good friend, the INF file. For the 1394 virtual device hybrid driver, I'm going to install the kmdf stub, then attach the umdf stub as an upper filter to that "device". I am breaking the inf file included w/ the sample in to sections to hopefully make it easier to read. And in doing so it should show better where all the installation magic happens. So the first thing to do is the normal stuff for installing the kmdf1394vdev.sys file, your standard inf will serve as a basis. We just need to set Redirector / Reflector / Redefelctor object as an upper filter to that device (yes, I fully expect somebody to talk to me about the additional name *g*).

We'll add the necessary registry and service entries to the non-WDF installation section, so I'm only going to highlight the additions / changes to a normal inf and I'll be using the model of the UMDF being an upper filter driver;

[kmdfvdev.NT.HW]

AddReg=kmdfvdev_AddReg

[kmdfvdev.NT.Services]

AddSerivce=umdfvdev, ,umdfvdev_ServiceInstall

[kmdfvdev_AddReg]

; Load the UMDF Redirector as our upper filter

HKR,,"UpperFilters",0x00010008, "WUDFRd"

[umdfvdev_ServiceInstall]

;Make sure to create a unique display name for the redirector service

DisplayName = %WUDFRd_SvcDesc%

ServiceType = 1

StartType = 3

ErrorControl = 1

ServiceBinary = %12%\WUDFRd.sys

Pretty straight forward so far, next are the additions for the WDF installer section;

[kmdfvdev.NT.Wdf]

UmdfService=umdfvdev,umdfvdev_Install

UmdfServiceOrder=umdfvdev

[umdfvdev_Install]

UmdfLibraryVersion=$UMDFVERSION$

DriverCLSID= ;generate a class GUID

ServiceBinary=%12%\UMDF\umdfvdev.dll

*Currently Playing - Spock's Beard As Far as the Mind Can See: Pt. 3 They Know We Know

And the winner is...

patman — Fri, 10 Nov 2006 20:26:00 GMT

Well fellow travellers, it looks as though I'm going to port the wonderful 1394 Virtual Device Driver (1394vdev.sys) in to a hybrid driver. I was pushing for this as the project for a number of reasons. I'm familiar w/ it having supported it while working for the 1394 team. It's been in the WDK / DDK / LDK in WDM form, it was migrated to a KMDF driver recently and I think it would be a great show of progression to finish that link of WDF usage.

I'm opening the comment flood gates to a degree so any question you have fire them off!

Synchratic Method (yes, I know it's not a word).

patman — Mon, 06 Nov 2006 23:14:00 GMT

As promised here is an update. I'm still trying to nail down a sample hybrid driver for you all to play with. We've narrowed the contestants to two, and after the talent and judges interview portions of the show we should have a winner.

So rather than let a blog post go to waste by prattling on about my dog or my signed copies of Parallel Worlds and Hyperspace, I thought it would be best served to share a "well, fancy that" moment I discovered while creating a KMDF test driver.

Synchronization scope in KMDF, the key word...scope.

See what I did (and we learn best by laughing at other's mistakes) is to set my WDF_OBJECT_ATTRIBUTES for the device object with the synchronization scope level of WdfSynchronizationScopeDevice. I had also initialized the ExecutionLevel to WdfExecutionLevelPassive thinking it would establish the execution of the object's event callback functions to passive. I then intended to establish the WDF_OBJECT_ATTRIBUTES for my queue object with the scope level of WdfSynchronizationScopeNone to ensure that I had better control over the IRQ levels in my callbacks. Sounded like a great plan to me!

A few BSODs later, I found out it doesn't quite work as I thought it would. When using the WdfSynchronizationScopeDevice as the scope level on the device object, even if I attempted to specify my queue object at the WdfSynchronizationScopeNone, the queue was inheriting the device object's synchronization scope. So everything in my queue was running at DISPATCH_LEVEL. And we all have probably found out at one time or another, some things in the kernel world don't like being called a DISPATCH_LEVEL.

So after some debugging, and then running around talking to some KMDF guys, I found the cure to my woes was to establish my device object scope level attributes to WdfSynchronizationScopeNone. I'm still also setting the execution level in the device object attributes to WdfExecutionLevelPassive, but I'm not establishing any scope on the queue object at creation, so I'm letting the natural order of synchronization objects within the callbacks manage the IRQ levels.

Thus ends my, "Why in the name of Jacques Lemaire is this thing always coming in at DISPATCH_LEVEL!?" adventure for the week. If you want some more info on this, go bug Doron. He loves when you pester him about this, really he does. >:)