Trouble Ahead- Trouble Behind : C++

The Case of the Unexpected Overload

BobKjelgaard — Mon, 03 Aug 2009 17:31:44 GMT

So are you using C++ in a KMDF driver and scratching your head over some unexpectedly unresolved references?

I recently engaged in a somewhat prolonged conversation with Doron and a dev I won’t name [since I didn’t ask for permission] about C++ in KMDF driver. This dev (a very good dev, by the way) had hit a stumbling point converting a driver from C to C++ and was wondering if we had ever tried this in QA.

Yes, I know that means he/she doesn’t read this blog, since I’ve mentioned doing just that on more than one occasion. But that’s OK, not many people do- and since I’ve got a tendency to write about whatever strikes my fancy at any moment, and I’m not the most dazzlingly brilliant human on the face of the planet, that’s not much of a surprise to me. You are of course free to believe what you like about that statement.

But in an attempt to be useful today- the problem was one of those that I guess I’ve hit so many times I don’t give it much thought. But it gave our dev a bit of trouble, and not much gives this person that kind of trouble. So perhaps a word to the wise will give the three or four of you still reading this a leg up the next time you use C++ in a KMDF driver.

The Lesson for Today Is…

It was just a simple conversion of a driver from C to C++ (to gauge my familiarity with the process, I did that with a half dozen or more drivers last week, and it was far from the only thing I did that week- in fact the big job was actually that I also made them W4 and PFD clean along with quite a few more drivers I’d already converted to C++- almost two dozen drivers total), but it ran afoul of this little bit of goodness:

typedef
__drv_functionClass(EVT_WDF_OBJECT_CONTEXT_CLEANUP)
__drv_sameIRQL
__drv_maxIRQL(DISPATCH_LEVEL)
VOID
EVT_WDF_OBJECT_CONTEXT_CLEANUP(
    __in
    WDFOBJECT Object
    );

typedef EVT_WDF_OBJECT_CONTEXT_CLEANUP *PFN_WDF_OBJECT_CONTEXT_CLEANUP;

That’s the “role typedef” for any WDF object’s context cleanup routine. The SDV tool can use role typedefs to do a better job of statically verifying your driver code, so some special cleanup typedefs are defined to make it a better tool, and you can see them in this (by the way, everything I’m posting is from the WDK- no deep dark secrets here).

typedef EVT_WDF_OBJECT_CONTEXT_CLEANUP EVT_WDF_DEVICE_CONTEXT_CLEANUP;
typedef EVT_WDF_OBJECT_CONTEXT_DESTROY EVT_WDF_DEVICE_CONTEXT_DESTROY;
typedef EVT_WDF_OBJECT_CONTEXT_CLEANUP EVT_WDF_IO_QUEUE_CONTEXT_CLEANUP_CALLBACK;
typedef EVT_WDF_OBJECT_CONTEXT_DESTROY EVT_WDF_IO_QUEUE_CONTEXT_DESTROY_CALLBACK;
typedef EVT_WDF_OBJECT_CONTEXT_CLEANUP EVT_WDF_FILE_CONTEXT_CLEANUP_CALLBACK;
typedef EVT_WDF_OBJECT_CONTEXT_DESTROY EVT_WDF_FILE_CONTEXT_DESTROY_CALLBACK;

So, what the original driver, being a good sample [because SDV suitability also matters], had was something like this (not exactly like this- nobody with my twisted sense of humor writes sample code, because humor never works globally- someone always takes offense- I’m just not inclined to care about that anymore, because I’m a callous old bastard with less than a dozen years to retirement age and not much “career” left to lose):

// In the header file:

EVT_WDF_DEVICE_CONTEXT_CLEANUP HideTheBodies;

// In the implementation
VOID
HideTheBodies(
    __in WDFDEVICE MortTheMortuaryTechnicianIsOnTheJob
    )
{
...

Of course, in C, that sort of works- but in C++, that says you have two overloaded versions of HideTheBodies- one takes a WDFOBJECT as input, and isn’t defined, because nobody wrote it that way. The one that is defined takes a WDFDEVICE as input. More annoying, of course, is that from the typedef name, you’d expect it would take a WDFDEVICE as input, so it isn’t an error that’s glaringly obvious.

As is- the code will compile fine, and PFD will be happy, too [they’re not the same function, remember]. But when you link it, you get an unresolved reference for HideTheBodies- decorated to indicate it has a WDFOBJECT parameter. Now, unless you really look closely at such decorations (something my fine history of mistyping the simplest of code has taught me to do), you’re looking at a mystery. It took my colleague digging through preprocessor output and object files and such to finally hit upon the error- a reasonably annoying circumstance!

Another thing you can do (and this is what I usually do) is tie the callbacks to the context itself (with the context a class) as static member functions, as I described earlier (effectively you do the cleanup by calling the context’s “delete” operator). PFD currently doesn’t match the roletype if you do this, so I suppress those warnings (and there is no SDV for C++)- I think we’re trying to get that fixed… But the compiler will catch the parameter type mismatch- a bit earlier than the linker.

But I’ve hosed this both ways- so, until something on the subject gets posted by a recognized authority, this post can at least be found by the diligent searcher…

He’s Gone

BobKjelgaard — Fri, 26 Jun 2009 05:15:16 GMT

It has been a tough week, and a busy one. I’m getting ready to leave on a vacation, and trying to get all those last-minute things done.

While my last post discussed the genesis of what I call the Configuration Agent and the WDF QA Universal Setup Job, this week’s focus has been entirely on another of my oddball notions- it started as a desire to build a more precise fault injection engine for testing the KMDF loader. With that California background, the project name was easy- Fault? Why, San Andreas, of course!

At a high level it was a programming model built around IAT filtering of calls from a driver into the kernel. I wanted precision, so I could specify behavior by thread and IRQL, program each DDI independently, allow a driver to do its own filtering [i.e. get a callback with context for a DDI call under given conditions]- and with enough flexibility and locking behavior that the behavior could be reprogrammed within such a callback. For instance, in one working application of mine a filter on calls to IoCallDriver from the KMDF runtime were routed on specific threads to a callback in the test driver which then turned off all call logging, as it meant the call was leaving it’s stack location- I was using the logs from calls to various resource allocation DDI to make sure the framework did not allocate resources under conditions in which we contractually would not fail- so I wanted to stop logging before I called IoCallDriver and to restart it after it returned [I knew the driver handling it was synchronous- otherwise I would have to hook the completion side of things, of course].

So the concept went well beyond fault injection, but it’s always been a side-line. My initial tests wound up being very white box, and a refactoring of the loader code meant a similar rewrite of the test was necessary, but there was no time to do that… But with Windows 7 winding up [since we’ve announced dates, that can’t qualify as a secret anymore], I have time to go back and try to at least do a few new things beyond mere maintenance. Such time is rare and fleeting, so I’ve been busily making the most of it.

So today was busy- I worked a bit of overtime on a COM automation server (in process) exposing a scriptable interface to my engine for some of the other team members to use, and got it at least working well enough to demo. Still have my packing to do, and dinner to make and so forth, so home I headed.

Then I turned on the television, and immediately heard the news about Michael Jackson. Whoa…

On those old tapes I’ve been scraping for my archives I’ve got many covers of his material from the time I was playing in bar bands: Billie Jean, Thriller, Beat It- and more. I’ve not posted them because frankly I never thought I did any of them justice- a lot of my material from those days has rough edges [anyone who listens to what I have posted can hear that], but these were just not good enough. But add me to the billions of fans, then and since, sure…

The other thought that came early on was that in my previous post I mentioned having learned to keep some of my more controversial opinions to myself- one of those was intense skepticism about the molestation accusations. I knew I wasn’t likely to sway anyone there- so why bother?

He’s gone, and he was younger than I am- scary stuff. Life is fleeting, and it was an unexpected reminder of that.

But, it’s time to pack my CDs and games and controllers, write those last few checks for bills coming due during the vacation. eat that dinner, and get ready for a night’s rest prior to that final day of work, so another slapdash post, and back to work…

There was some offsetting good news today- besides finding out I really do still remember enough about COM to roll out an in-process automation server on the QT- without using ATL of course, where would the challenge be in that [answer: in learning something new for a change- you old dog!]? But of course it falls into that semi-confidential area where I’ll just have to sit on it for a while [not that it’s likely to shatter the world if revealed anyway, but circumspection is worthy of practice even for an old blowhard like myself].

I’ll try to do something more substantial someday- these days I’ve just not got the time to really write anything too terribly technical. My apologies to those diehards who still try to read this awful stuff I’m churning out of late…

Exceptions rule!

BobKjelgaard — Sat, 18 Apr 2009 19:24:55 GMT

I’ve decided to revert to storytelling mode today. So sit right back and you’ll hear a tale, a tale of my fateful trip (or click something useful- you’re the one deciding, and I’m the one typing to satisfy whatever inner demon has driven me to do this today)…

It began when someone found a bug in the static version of KMDF- the one almost nobody gets to use and which we have been trying to discontinue any support. But people were still using it, and we had no clear migration path, and there was a bug. So we decided to begin testing it. Poor Neslihan got that task (well at least it wasn’t me, this time). Soon we had added a static KMDF driver to our list of test drivers used in our daily automated testing.

Then, of course, we got the bugcheck. As often happens, I played “Johnny-on-the-spot” and (virtually speaking) leapt onto the remote debug session accompanying the report. Buffer overflow? In FxDriverEntry? But only on Itanium (I checked the other architectures that ran the same test, of course- no problems)? What in blazes is this about?

Well, the method is well documented, and not all that hard to understand- a “cookie" gets initialized, it gets stored to the local stack frame in such a way that code overflowing buffers in the stack frame will overwrite the stored copy. On exit, code gets called that checks that value to see if that happened. It’s a little more complicated than that, and I’m not going to be precise, because part of that complication is related to securing the method against the sort of people who like to exploit buffer overflows [so I’m not going to make the method clear for them- they can do their own research without my assistance]. The bugcheck analysis will even tell you the two values- expected and actual, making it easy for people who really think overflows are a bad idea to make a quick guess as to where the overflow came from.

But the circumstances seemed suspicious, so I went and read the source code- H’mm- there’s a default value the cookie is initialized to… Interesting- that’s the value we have- but it doesn’t match the expected value.

At which time, something I’ve known for years hit me like the proverbial sledgehammer- our entry code calls GsDriverEntry (which supports the stack probes inserted by the GS compiler switch, hence its name) when it finishes. GsDriverEntry initializes the cookie, which means that that was why the values didn’t match. Like the sword of Damocles, this had been hanging over our heads for years- the first time the compiler decided to do stack probes in our entry code, everything would break. Ouch…

I’ll leave a bunch of story out here- things got “interesting” at that point, but the problem eventually got solved, both for us and for other people facing similar issues, as well. Short answer is that everyone now initializes the cookie right at the start, just as should happen…

But a while back I had occasion to tweak the build process for the WdfVerifier WDK applet, and afterwards I ran it briefly to make sure I didn’t break anything in the process. Oh, joy of joys- NONE of the 1.9 client drivers are being properly identified. They are all identified as “Inbox”, which is what I do when the client identification method I described last year fails on me.

Already beginning to panic, I think- did someone change FxDriverEntry, and I didn’t even notice it? So I go to our source control system, and look at the change history. The most recent change is the fix for the problem with the stack probes (yes it occurred on static, but what scared me then was it could have happened to any version, because were weren't doing things properly). But that just calls library routines to initialize said cookie… Oh, blazes, those routines must be calling an import I wasn’t accounting for!!! Why??? Because I NULL the IAT to force access violations, and I handle the exception by giving up on getting the true version, and fall back to calling it “Inbox”- which is, after all, exactly what I am seeing. Oh, well- so much for my having thoroughly considered all the test and product consequences of that change when it was made…

Well, easy enough to find out what that import might be. One nice trick Ilias showed me is that you can open any binary in WinDbg as a crash dump, and happily resolve symbols and disassemble code. So I pick a random driver on my dev box, and do so.

What, no imports? But, but--- hmm- it uses a fixed address (load of a 64-bit immediate value into RAX, since my dev box is an X64, thank you- you x86 dinosaurs can keep your wimpy processors). What’s with this?

The answer lies in wdm.h, of course- KeQueryTickCount turns out to always be a macro- on x86 and IA64, it accesses KeTickCount [which is an import, and if you followed my earlier tale, it’s clear adding an import to my existing hack-o-matic mechanism is a trivial task]- but on X64, it accesses an essentially hardcoded address in a “SharedUserData” area. This snippet is from wdm.h, so you can see for yourself…

#define KI_USER_SHARED_DATA 0xFFFFF78000000000UI64

#define SharedUserData ((KUSER_SHARED_DATA * const)KI_USER_SHARED_DATA)

#define SharedInterruptTime (KI_USER_SHARED_DATA + 0x8)
#define SharedSystemTime (KI_USER_SHARED_DATA + 0x14)
#define SharedTickCount (KI_USER_SHARED_DATA + 0x320)

#define KeQueryInterruptTime() *((volatile ULONG64 *)(SharedInterruptTime))

#define KeQuerySystemTime(CurrentCount)                                     \
    *((PULONG64)(CurrentCount)) = *((volatile ULONG64 *)(SharedSystemTime))

#define KeQueryTickCount(CurrentCount)                                      \
    *((PULONG64)(CurrentCount)) = *((volatile ULONG64 *)(SharedTickCount))

My. my. my- this is a problem- no import I can just tweak to point into my code. For a plus, at least the value can’t change (if it did, existing drivers would, after all, cease to work)- well, at least not easily, so I’ll save any remaining paranoia about that for another time. But I’m going to access violate trying to access that address in user mode (not to mention the reams of experimental evidence I had just accumulated by noticing the problem in the first place)…

So in reading about exception handling, a phrase happens to catch my eye- putting it in my words, it says that when continuing an exception, you can alter the context record supplied to you with the exception (and I know this process well, but that’s perhaps a story for a different time). Whoa- that must mean I can change the contents of the registers programmatically, and then tell it to repeat the failed instruction! Now, I can’t find that illustrated in any of the samples the material I’m reading points me to [nor could I find it an an internet search, although the latter was by no means exhaustive]. But that HAS to be what it means, right?

So I start coding- first the half dozen or so lines needed to handle the KeTickCount import. Then an exception filter [only for AMD64, of course]. The logic is my usual bit of precise work: If it is an access violation, and if it is a read, and it is a read of that exact address, and one of the integer registers in the context record has that same address, then change that register’s entry in the context record to the address of the proxy I’ve decided to keep in WdfVerifier, and tell the OS to repeat the failed instruction. I began with RAX, because after all, that’s what I had seen in my investigation, and it seemed the most likely place for a while, but I added the whole set since under the circumstances, it seemed unlikely to do any harm. Anything that didn’t match that exact pattern, and I gave up- just execute the handler, which does nothing. The attempt to run the driver entry code to extract the client version will fail, but WdfVerifier keeps running, and the machine itself is still quite safe from my hackery.

Worked the first time, it did (not counting my usual compiles to get rid of typos). Problem solved. Total work time from start to finish- something like 5 or 6 hours- good enough- after all, I did have to do some research… Of course, there’s always somebody who can do it better, faster, and quicker- and they usually jump out of the woodwork if I start talking about things in that fashion, but it’s my story, so I’ll brag now and regret it later…

Exceptions do indeed rule! I love it when a plan comes together…

A glimpse now at handling the new import…

    //  The descriptors are an array of entries per module, each terminated with an all-0 entry.
    //  Each entry has an RVA for the module image name, and a second for a 0-terminated list of RVAs to the structures used for
    //  resolving names of exported entries from the module (in loading these get resolved to real addresses and are plugged into
    //  the loaded image's import address table in the same order).  So we now know how to write an image loader if we need to...

    //  First, find the descriptor for the KMDF loader, and quit if it isn't there.
    DWORD   StringCopy = (DWORD) -1, WdfBind = (DWORD) -1, InitEvent = (DWORD) -1, TickCount = (DWORD) -1; 

    //  Then look for the Bind and Unbind entry points.  Case counts!

    enum {OutdatedTechnology, HasBind, HasUnbind, OneCoolDriver};
    
    BYTE        TargetIdentification = OutdatedTechnology;

....

            else if  (IsKernel)
            {
                if  (0 == strcmp("RtlCopyUnicodeString", (PSTR) NameDescriptor.Name))
                    StringCopy = ImportsIndex;
                else if  (0 == strcmp("KeInitializeEvent", (PSTR) NameDescriptor.Name))
                    InitEvent = ImportsIndex;
                else if  (0 == strcmp("KeTickCount", (PSTR) NameDescriptor.Name))
                    TickCount = ImportsIndex;
                
                ImportsIndex++;
            }
.....


        DWORD       IATSize, RelocationsSize;
        LONGLONG    OurFakeTickCount = 0x8badf00ddeadbeefI64;   //  ersatz Tick count         
        PVOID*  ImportAddresses = (PVOID*) 
            ImageDirectoryEntryToDataEx(DriverImage, TRUE, IMAGE_DIRECTORY_ENTRY_IAT, &IATSize, &Unused);
        
        PIMAGE_BASE_RELOCATION  Relocations = (PIMAGE_BASE_RELOCATION)
            ImageDirectoryEntryToDataEx(DriverImage, TRUE, IMAGE_DIRECTORY_ENTRY_BASERELOC, &RelocationsSize, &Unused);

        if  (NULL == ImportAddresses)
        {
            FreeLibrary((HMODULE) DriverImage);
            return  true;
        }

        //  Plug them in here, and we are good to go!

        DWORD   OldProtect;

        if  (!VirtualProtect(ImportAddresses, IATSize, PAGE_READWRITE, &OldProtect))
        {
            FreeLibrary((HMODULE) DriverImage);
            return  true;
        }

        memset(ImportAddresses, 0, IATSize);

        ImportAddresses[StringCopy] = FakeOutStringCopy;
        ImportAddresses[WdfBind] = CollectVersion;
        if  (InitEvent != (DWORD) -1)
            ImportAddresses[InitEvent] = FakeEventInit;
        if  (TickCount != (DWORD) -1)
            ImportAddresses[TickCount] = &OurFakeTickCount;

        VirtualProtect(ImportAddresses, IATSize, OldProtect, &OldProtect);

So for some other glimpses, this is one part of the other hackery (complete with my elementary-level annotations):

class AllKMDFDrivers
{
    MachineKey&             Owner;
    KMDFDriverList&         InstalledKMDFDrivers;
    LoadedDrivers&          CurrentlyLoadedDrivers;
    LoaderDiagnosticsFlag   LoaderFlag;
    String                  RuntimeVersion;
    DWORD                   Major, Minor, Build;

    bool            ServiceUsesKMDFLoader(__in PCWSTR ServiceName, __in RegistryKey& Service, __out MyBindInfoAlias& Binding);
    static void     FakeOutStringCopy(__in PVOID, __in PVOID);  //  Fake RtlCopyUnicodeString entry
    static void     FakeEventInit(__in PVOID, __in ULONG, __in ULONG);  //  Fake KeInitializeEvent entry
    static ULONG    CollectVersion(__out MyBindInfoAlias& BindingOut, __in PVOID, __in MyBindInfoAlias& BindingIn, __in PVOID);
#if defined(_AMD64_)
    static LONG     Filterx64Exception(__in EXCEPTION_POINTERS* ExceptionInfo, __in LONGLONG& FakeTickCount);
#endif

While the rest of it looks like this (and again, this is partial)- yes, feel free to hate my stylistic indifference to the herd- perhaps I’ll get fired for this, and all can breathe a vast sigh of relief…

First, the code that calls the filter

        MangledDriverEntry  CrashAndBurn = (MangledDriverEntry) ((PBYTE) DriverImage + EntryRva);

        __try
        {
            CrashAndBurn(Binding, &Binding);
        }
#if !defined(_AMD64_)
        __except(EXCEPTION_EXECUTE_HANDLER)
#else     
        __except(Filterx64Exception(GetExceptionInformation(), OurFakeTickCount))
#endif
        {
        }

and now our filter:

#if defined(_AMD64_)
/**********************************************************************************************************************************

LONG    AllKMDFDrivers::Filterx64Exception(__in EXCEPTION_POINTERS* ExceptionInfo, __in LONGLONG& FakeTickCount)

Ahh, the joys of low-level intervention by the truly incorrigible!  On AMD64, we will get an exception when the code tries to get
the tick count.  This value resides at a known address (which cannot change because if it did, all existing drivers would then fail
to work- although I suppose someone could resort to what I am about to do- bring it on, I'll see if I can keep up with the absurd
arms race).

So, first, verify from the exception record that we are getting an access violation of some sort reading that known address.  Then
see if one of the integer registers has that address (start with RAX, since that's where it currently is, and I bet it doesn't
change much).  If it does, change it to point to the value given to this function (which resides in WdfVerifier, since I made it
a reference, and will let the compiler play enforcer), and then continue execution.

In all other cases, execute the handler, which will do nothing (meaning the version will continue to be unknown).

**********************************************************************************************************************************/

LONG    AllKMDFDrivers::Filterx64Exception(__in EXCEPTION_POINTERS* ExceptionInfo, __in LONGLONG& FakeTickCount)
{

    //  cf definition of SharedTickCount in wdm.h
    static const DWORD64    KnownTickCountAddress = 0xFFFFF78000000320UI64;

    if  (NULL == ExceptionInfo || NULL == ExceptionInfo->ExceptionRecord || NULL == ExceptionInfo->ContextRecord)
        return  EXCEPTION_EXECUTE_HANDLER;

    switch  (ExceptionInfo->ExceptionRecord->ExceptionCode)
    {
    case    EXCEPTION_ACCESS_VIOLATION:
    case    EXCEPTION_IN_PAGE_ERROR:    //  Unlikely, but what the heck...
        if  (2 > ExceptionInfo->ExceptionRecord->NumberParameters)
            return  EXCEPTION_EXECUTE_HANDLER;
        break;

    default:
        return  EXCEPTION_EXECUTE_HANDLER;
    }

    if  (EXCEPTION_NONCONTINUABLE == ExceptionInfo->ExceptionRecord->ExceptionFlags)
        return  EXCEPTION_EXECUTE_HANDLER;

    if  (EXCEPTION_READ_FAULT != ExceptionInfo->ExceptionRecord->ExceptionInformation[0] ||
        KnownTickCountAddress != ExceptionInfo->ExceptionRecord->ExceptionInformation[1])
        return  EXCEPTION_EXECUTE_HANDLER;

    if  (KnownTickCountAddress == ExceptionInfo->ContextRecord->Rax)
    {
        ExceptionInfo->ContextRecord->Rax = (DWORD64) &FakeTickCount;
            return  EXCEPTION_CONTINUE_EXECUTION;
    }
    else if  (KnownTickCountAddress == ExceptionInfo->ContextRecord->Rbx)

The rest I leave as an exercise to the reader, having already put all 0.378 of you through the treadmill today…

Using C++ in a KMDF driver part 1- a pattern for using contexts as objects

BobKjelgaard — Thu, 12 Mar 2009 16:36:59 GMT

This is an article I’ve started probably close to a dozen times since I started this blog, but never published. In part because of all the heat the topic of using C++ in the kernel generates, and the rest perhaps because of my reaction to that heat.

So I’ll get one thing off my chest at the start and perhaps that will be enough to let me proceed.

I’ve been writing drivers with C++ in the kernel (or the Win 3.1/9x/ME VMM) pretty much since I began using the language (in the very early 90’s). I routinely use paged code (paged data not so much- I’ve never had designs where there seemed to be any benefit to it). That spans more than a decade- in fact, its close to two decades now.

I’ve NEVER had a problem with the issues raised in the paper. I’m not even going to link to it. If you want to find it- it isn’t hard to find. On top of that, in a determined effort to find people who had, I didn’t find many- and the one clear case I did find was cured with a #pragma that would have made sense if you were a C programmer and had a basic understanding of where VTABLES and such get emitted to. So I personally have a sense of mismatch between my own experience and the strongly worded severity there.

I’ve been told that if I persist, I’ll deserve all the paging problems I get [and that nobody will help me with them]. Well, I do get them (always have), and I’ve been developing long enough in this environment that I don’t really need anybody’s help to find the root cause of a bug like that. The causes are always the ones I remember- acquiring a spin lock in pagable code- making a routine pagable that can be called at elevated IRQL- all the usual ways to screw up in that other language that is the hallmark of a true first-class kernel developer. But none of them had anything to do with my choice of programming language. Not once. Not saying that it never happened across all those years, because in the early ones, the ability to catch a bug like that was severely limited- there was no Driver Verifier, no static analysis tools. So it may not have been noticed early on. But now that I have them, it’s still not happening…

Now there can be plenty of reasons for that- but I look at what people take away from the paper, and I do a lot of the things they think aren’t safe. I use polymorphism and inheritance freely. I don’t use multiple inheritance a lot, but I have used it and have not observed problems arising from its use. On the other hand, there are common features that I don’t use as a matter of personal preference or style that may bear on this. I don’t expose the implementation of functions in header files (meaning the compiler is not going to start out by inlining them and then give up later, dumping them in some unintended segment) with the exception of trivial accessor functions. I don’t use templates (I’m not sure they were a language feature when I started, but at any rate, during my learning curve I never needed them, so while I can handle code with templates, I don’t use them myself). I almost always code my own constructors, destructors, copy constructors and assignment operators (I usually have to- I prefer references to pointers and if you have reference members, the compiler can’t generate default code for most of those routines).

So now that its clear I’m not going to walk the company line on that topic [the opinions I expressed ought obviously to be clearly my own], I’ll proceed to something more useful…

Leveraging the KMDF Object Model

KMDF provides a nice object model, with managed lifetimes and one of the most delightful observations I had in my first KMDF driver was that it was easy for me to blend this with my usual coding patterns.

As an aside, I’ll note that I never use sample code to learn anything. I take the reference materials and code to them. If there are samples, I treat them as the last resort- and I will deliberately change as much as I can of them [in part to see what knowledge of things that can go wrong wasn’t explicitly represented in the sample]. I judge the quality of the reference material by how rarely I have to look at a sample to figure something out [yes, I don’t find much reference material i would call “good”].

So my first KMDF driver was a software bus driver, and I didn’t go near toaster in writing it. In fact it was one of those “ActiveX” (OLE automation) test drivers I mentioned in our DDC presentation. Now we had some samples for them, too- and not surprisingly, virtually no reference material. The samples were fastidious about one thing- all the COM parts were in C++ (not a lot of choice there)- but all the parts using KMDF were in C- precisely because of said paper. I might add that this was even though no attempt was made to mark any of the code or data pagable in those samples. Well, I’m a stubborn <expletive of your choice here>, so I decided then and there I was going to write the whole driver in C++ in spite of the objections I received. I was still in my “trial period” so they could always fire me if they wanted to, but the job market was good enough at that time that I was pretty sure I could find something…

Back to the proper topic- one fine thing is that most of the KMDF macros are agnostic enough they can handle at least straightforward C++. That was one of the happiest discoveries of that time.

So the basic pattern as I use it:

Declare a static function in your class that takes a WDFOBJECT as input and returns a pointer to an object of your class.
Declare a class-specific placement form new operator that takes as its additional input the type of handle you expect your object to live in the context of (that is, it can be more precise than the preceding function can and you can benefit from stronger typing in C++)
Declare a class-specific delete operator that basically does nothing if you need to have your destructor invoked.
If you have such a delete operator, also declare a static member with a void return that takes a WDFOBJECT as input.
Use the WDF_DECLARE_CONTEXT_TYPE_WITH_NAME macro to get the compiler to write that first function for you.
Have your new operator implementation use the first function to return the address of the underlying context (you basically ignore or validate the size parameter) from the passed-in handle.
When creating the context, and you need your destructor called, use a WDF_OBJECT_ATTRIBUTES structure with the EvtCleanupCallback set to the routine in item 4.
Code that routine to use the routine in item 1 to get the context address out of the object handle- and delete that pointer. This causes your destructor to be called at cleanup time (which is much more sensible than destroy time) and your do-nothing delete operator will also be invoked (or inlined out of existence if your compiler is any good).

There- you “create” your object when the KMDF object is created (or via a WdfObjectAllocateContext call if you add your object later), and “delete” it when the object dies. But KMDF manages the memory and most of the object lifetime for you. Sure works for me (a lot).

The following snippet is from that first driver (I’ve since dropped the usage of “C” on class definitions in my general drive toward anarchic style). This is slightly convoluted because I have logic allowing only one instance of a device with this driver- so I’ve deliberately intermingled driver-level and device-level usages (always pushing those boundaries- but I think that’s a good way for an SDET to think). I’ll admit this is slightly doctored (I removed some things related to the COM technology as that I can’t disclose, plus I tried to include support for the standard bus interface and that just complicates things without illustrating this method), but it should show I practiced what I am preaching…

class CTargetTestBus
{
    static WDFDEVICE                Owner;              //  WDF device that "owns" this bus object

    static void*    operator new(size_t size, WDFDEVICE OwningDevice);
    CTargetTestBus(WDFDEVICE OwningDevice);
    ~CTargetTestBus() {}

    //  Private callbacks (ie, accessed from within this class' code)
    static EVT_WDF_IO_QUEUE_IO_DEFAULT      OnIoDispatchDefault;

    static void operator delete(void*) {}
public:

    NTSTATUS    NewChild(LPCWSTR Name, int InstanceID);
    NTSTATUS    RemoveChild(LPCWSTR Name, int InstanceID);
    void        DoneWithBus();

    static CTargetTestBus*  GetThisTargetTestBus(__in WDFOBJECT Object);    //  WDF macro writes this code
    static CTargetTestBus&  GetTheBus(bool& BusPresent);
    
    //  Driver callbacks (public, because used in DriverEntry)
    static EVT_WDF_DRIVER_DEVICE_ADD        OnDriverDeviceAdd;
    static void             OnDriverUnload(IN WDFDRIVER Driver);

    //  This one is accessed from a member in CChildInfo
    static EVT_WDF_CHILD_LIST_CREATE_DEVICE OnAddNewChild;
};


// Macros to get the context
WDF_DECLARE_CONTEXT_TYPE_WITH_NAME(CTargetTestBus, CTargetTestBus::GetThisTargetTestBus);

Now playing: Me (that old recording of Johnny B Goode again)!

Why You'll Never See a Sample From Me...

BobKjelgaard — Fri, 08 Feb 2008 23:05:00 GMT

I noticed yesterday that Peter and Patrick are trading comments and thoughts on coding styles. So I'll come out of my cage and admit I am the consummate iconoclast on most issues of style. When I review code, I make comments about logic, out-and-out errors, and (generally adding that I don't regard it as important) typos in comments. Beyond that, I don't bother. When I read the occasional forum thread on coding, I just sort of sit and shake my head as I read how much people hate the way I like to think and loathe the things I find useful to do.

But that's nothing new- I learned a long time ago that my ideas were different. But I've spent my years in maintenance and other activities and still have them.

Most standards are fueled by someone's personal tastes being enforced by fiat on a team, usually backed up by anecdotal assertions about how useful they are in maintenance. You take liberal amounts of those add a dash of geopolitical correctness or "professional presentation" reviews and a nice heaping of endless semantic analysis of names and what I once found to be enjoyable (writing code to make a computer solve a problem) becomes one of the most tedious tasks on earth.

I loathe standards. I deliberately flout them at times just to get the pent-up frustrations off my chest. I usually revert to using them [pragmatism again], but occasionally get caught when I write some toy that someone later decides should be kept but they don't want to rewrite it [hard to understand, because for the life of me, most devs secretly want to rewrite every line of code they see so it fits their personal style- usually they have to wait unitl they have a senior position, at which point they have learned to call it "refactoring", which makes it sound more worthwhile than "making it pretty because currently it iz teh suk".

Of late, I like this set of rules:

132 columns: 80 columns- that was punched cards. I used them, they sucked. Character mode displays- same limit because they had to display what was on punched cards. They were better, but I like my hi-res displays much better, thank you. Printers- I wrote and tested printer drivers for over a decade. I've personally destroyed enough trees I should rot in hell for several eternities. I only print when I must, and then I use both sides, N-up and whatever else I can do to keep my repentant attitude. And if some line gets longer than that, I'll fix it if I feel like it.
Mnemonic names are good. CamelCasingRules. Elaborate and contrived jokes are best, as long as you don't waste too much time on them.
Putting each parameter in a function call on a separate line is a waste of good screen real estate. I learned how to read more than one word at a time years ago. If I care what the parameters are, I'll look closer- I can even pop up a nice API description from MSDN or wherever if I need to (or use a class browser, etc). If I don't care what they are, I like being able to ignore one or two lines of code instead of a dozen.
C++ overloading is useful. That goes for functions, operators and even casts. If a line of code looks confusing, I use a browser to tell me which if any of those are in play. I then use intuition to tell me where the problems are likely to be.
My sole old-fashioned item is I adhere to Djikstra's seminal laws. Gotos are unnecessary. I could care if they look pretty- I still see more screwups today from their use than anything else. But I also am pragmatist and observer of the human condition enough to know a pointless battle when I see it. Each generation of programmers is smart enough to never do it wrong, and knows better than some dead Dutch guy how to write good code. Just ask them, they'll tell you. Global variables is the other one [at least, that's how I remember it].
C is a good language for dinosaurs. Plod along, ye denizens of the past!

Next week, maybe I'll adopt a different set of attitudes, just to be ornery. But I can only observe hours of discussion about a name or a standard like "do it this way because then when I use a proportional font I like the way it looks" or "all function names will begin with the camel-cased path to the source file from the base of the build tree [that's not a joke, BTW- I really worked where that was the standard- geez, use a PDB]".

SDETs aren't totally immune from such standards, but things are a bit more lax, and people maybe don't watch me as much as they should to keep me from misbehaving. I think my most recent misdeed was creating a quick off-the cuff C++ class to wrap a buffer pointer so it wouldn't leak (similar to the RAII pattern). Being in a hurry, I named the class the first thing that came to mind: leak free buffer = Pampers. Ordinarily I would have gone back and changed it, but I got in a hurry and checked it in anyway. I'm sure I'll get a diaperful about doing that at some point...

So just for fun, here are concrete examples of why you'll not be seeing samples from me:

From WdfVerifier:

/**********************************************************************************************************************************

void    ExplainHowTheCommandLineWorksInMindNumbingDetail()

Does as it says, says what it does.

**********************************************************************************************************************************/

void    ExplainHowTheCommandLineWorksInMindNumbingDetail()
{
    //  Like a good localizable application, the text for the message box is now in the resources.
    String  Explanation(IDS_Explanation);

    MessageBox(NULL, Explanation, ApplicationName, MB_ICONINFORMATION | MB_OK);
}

That happens to get called from here:

/**********************************************************************************************************************************

extern "C" int __cdecl main(int, __in char**)

Yes, I cheated- even though this is a Windows program, it still uses main- it uses the Windows API to get a unicode command line.
Seriously, this works fine, and it isn't any danger at all.  Used to do it all the time in the early days of NT (those halcyon days
when you had to start Windows from the command line after the system booted).

**********************************************************************************************************************************/

extern "C" int __cdecl main(int, __in char**)
{
    //  Check the command line- it will tell us if we need to display a UI or just function as a console app.

    int     ParameterCount = 0, ServiceCount = 0;
    bool    ExplainYourself = false;
    bool    RemoteAccessRequired;
    PWSTR   MachineName = NULL;
    PWSTR*  Parameters = CommandLineToArgvW(GetCommandLineW(), &ParameterCount);
    
    //  Parameter 0 is the program name, and it isn't important here...

    switch  (ParameterCount)
    {
    case    1:
        break;

    default:
        ExplainYourself = true;
        break;

    case    3:
        if  (CompareString(MAKELCID(MAKELANGID(LANG_ENGLISH, SUBLANG_ENGLISH_US), SORT_DEFAULT), NORM_IGNORECASE, 
            Parameters[1], -1, L"-machine", -1) == CSTR_EQUAL)
            MachineName = Parameters[2];
        else
            ExplainYourself = true;
    }

    if  (ExplainYourself)
    {
        ExplainHowTheCommandLineWorksInMindNumbingDetail();
        return  98052;
    }

The comment about starting as UI or console is an artifact I missed (and will thus shortly disappear). You can look up 98052 (hint- it's a US postal code, aka ZIP code).

This will upset all those who loathe overloads- it is from a stress test I wrote early on for thrashing KMDF request processing, pnp and power in a random fashion- this snippet does all the up front configuration stuff (there are built-in defaults which you can override from a file specified on the command line- you can optionally display a huge property sheet-based configuration UI and save it all back to a file- settings are also saved to registry and those settings override defaults (but not from file or UI, of course):

/**********************************************************************************************************************************

main

Establishes the stress run settings, possibly with user interaction, then starts the initial mix and loops forever interfering with
the setup until it gets stopped, the machine reboots, the debugger breaks, or you bugcheck.

**********************************************************************************************************************************/

extern "C" int __cdecl main(int, __in char**)
{
    //  Record the global instance handle, so all are copacetic with Win32.
    PropertyPage::InstanceIs((HINSTANCE)GetModuleHandle(NULL));

    //  Seed the PRNG
    FILETIME    Now;
    GetSystemTimeAsFileTime(&Now);
    srand(Now.dwHighDateTime + Now.dwLowDateTime);

    try
    {
        UsingCom                ButThenWhoDoesNot;

        //  We need to get access to the global configuration.  We must pay toll to the gatekeper to do so.

        ControlHolder           WhatMakesUsTick;

        //  Define objects that make up the stress configuration control (and UI)
        ThreadConfiguration&    ThreadControl = WhatMakesUsTick;
        TargetDelays&           TargetDelayControl = WhatMakesUsTick;
        HunterTimeouts&         HunterTimeoutControl = WhatMakesUsTick;
        HunterSectoring&        HunterSectorControl = WhatMakesUsTick;
        TargetLifetimes&        TargetLifetimeControl = WhatMakesUsTick;
        TargetStatus&           TargetReportingControl = WhatMakesUsTick;
        TargetToggler&          TargetToggleCentral = WhatMakesUsTick;
        HunterIdleTimes&        HunterIdling = WhatMakesUsTick;
        HunterPowerUpDelays&    HunterPowerUp = WhatMakesUsTick;
        HunterPowerDownDelays&  HunterPowerDown = WhatMakesUsTick;
        TargetIdleTimes&        TargetIdling = WhatMakesUsTick;
        TargetPowerUpDelays&    TargetPowerUp = WhatMakesUsTick;
        TargetPowerDownDelays&  TargetPowerDown = WhatMakesUsTick;

        //  Now, define the object that represents our operating situation
        MemoryStorage           Configuration;

        //  Size the parameter store, then populate it.
        SizeCalculator  Sizer;

        Sizer << ThreadControl << TargetDelayControl << TargetReportingControl << HunterTimeoutControl <<
            HunterSectorControl << TargetLifetimeControl << TargetToggleCentral << HunterIdling << HunterPowerUp <<
            HunterPowerDown << TargetIdling << TargetPowerUp << TargetPowerDown;

        Configuration << Sizer;

        //  First, set it to the defaults defined by each participating object.
        Configuration << ThreadControl << TargetDelayControl << TargetReportingControl << HunterTimeoutControl <<
            HunterSectorControl << TargetLifetimeControl << TargetToggleCentral << HunterIdling << HunterPowerUp <<
            HunterPowerDown << TargetIdling << TargetPowerUp << TargetPowerDown;

        //  Then, initialize from the registry if there's anything there (settings from last run)

        Registry    LastRun(HKEY_LOCAL_MACHINE, L"Software\\Microsoft\\KMDF QA\\IoTargetStress");

        Configuration << LastRun;

        //  Now check the command line- it will tell us if we need to display a UI or initialize from some file.

        int ParameterCount = 0;
        bool    DisplayUI = false;
        PWSTR   FileName = NULL;
        PWSTR*  Parameters = CommandLineToArgvW(GetCommandLineW(), &ParameterCount);

        if  (1 < ParameterCount)
        {
            //  Unfortunately LOCALE_INVARIANT is an XP invention, and this has to run on Win2K
            if  (CompareString(MAKELCID(MAKELANGID(LANG_ENGLISH, SUBLANG_ENGLISH_US), SORT_DEFAULT), NORM_IGNORECASE, Parameters[1],
                -1, L"-configure", -1) == CSTR_EQUAL)
            {
                DisplayUI = true;
                if  (2 < ParameterCount )
                    FileName = Parameters[2];
            }
            else
                FileName = Parameters[1];
        }

        //  If we've got a file to initialize from, so be it.

        if  (NULL != FileName)
        {
            FileStorage TreasuredValues(FileName);

            Configuration << TreasuredValues;
        }

        LocalFree(Parameters);

        //  Populate everything with our cool new settings (or perhaps the defaults they gave us in the first place)...
        Configuration >> ThreadControl >> TargetDelayControl >> TargetReportingControl >> HunterTimeoutControl >>
            HunterSectorControl >> TargetLifetimeControl >> TargetToggleCentral >> HunterIdling >> HunterPowerUp >>
            HunterPowerDown >> TargetIdling >> TargetPowerUp >> TargetPowerDown;

        //  If the command line says to display the Configuration UI, then do it.
        if  (DisplayUI)
        {
            Properties          ConfigurationManager;

            //  Populate the property sheet with its pages.
            ConfigurationManager << ThreadControl << TargetDelayControl << TargetReportingControl << HunterTimeoutControl <<
                HunterSectorControl << TargetLifetimeControl << TargetToggleCentral << HunterIdling << HunterPowerUp <<
                HunterPowerDown << TargetIdling << TargetPowerUp << TargetPowerDown;

            if  (ConfigurationManager.Interact() == IDOK)
            {
                //  Pull the updated configuration out of the objects
                Configuration << ThreadControl << TargetDelayControl << TargetReportingControl << HunterTimeoutControl <<
                    HunterSectorControl << TargetLifetimeControl << TargetToggleCentral << HunterIdling << HunterPowerUp <<
                    HunterPowerDown << TargetIdling << TargetPowerUp << TargetPowerDown;

                //  Now, store it off to a file.
                FileStorage Payload;

                Configuration >> Payload;
            }
        }

        //  Save this run's settings off to the registry
        Configuration >> LastRun;

        //  Time to actually start doing the REAL work

Yup, those are cast operators for object references [BTW, they are polymorphic, to boot], overloaded (multiple ways, depending upon the target object) shift operators (iostream-style) and a host of sins that will leave me burning in many folks' idea of programmer Hades for many lifetimes. Yes, there are "hunters" that fire requests at "targets"- they can even acquire them at will. I'd show you THAT code, but this is long enough [a static KMDF device context accessor named Hunter::HidingInBushes and some variables named "Nimrod" might give you a general idea, though- and yes, I did write all those KMDF Drivers in C++].

And like all true sinners, I am pretty much unrepentant. But I do promise never to author a WDK or SDK sample.