Static Driver Tools

PFD/SDV Clean - SDV Roletypes in Samples and Shipping Drivers

In parallel to the Annotation effort (detailed in this post), other members on our team were focused on enabling better checking of Inbox and Sample drivers by adding role types for use with the drivers we were scanning with Static Driver Verifier (SDV).  As mentioned in the previous blog post from Andrew (his entry on Role Types is here), role types “act as documentation by describing the intended role of a driver entry point routine (or callback function) and the routine’s return value and parameter types.  Best of all, role types enable Static Driver Verifier which can discover complex inter-procedural bugs causing blue screens and system hangs.”  

Unlike with the annotation work items, the tactic was different here:  our team reviewed the drivers and added updated the appropriate header files with the role types, then submitted those changes to the developers for review.  Since the role types apply to the IRP handling routines, it’s not necessary for the developer adding the role types to know exactly what those functions do.  The beauty of SDV is that once this map is in place, it does the work of figuring out what a function can and can’t do. 

Sample drivers (WDM and KMDF) in the Windows Driver Kit has been updated to use these role types.  The “featured1” version of Toaster is an excellent example of this, found at WDKRoot\ src\general\toaster\wdm\func\featured1 (clip of func\shared\toaster.h):

…

DRIVER_INITIALIZE DriverEntry;

DRIVER_ADD_DEVICE ToasterAddDevice;

__drv_dispatchType(IRP_MJ_PNP)

DRIVER_DISPATCH ToasterDispatchPnp;

__drv_dispatchType(IRP_MJ_POWER)

DRIVER_DISPATCH ToasterDispatchPower;

__drv_dispatchType(IRP_MJ_DEVICE_CONTROL)

__drv_dispatchType(IRP_MJ_READ)

__drv_dispatchType(IRP_MJ_WRITE)

DRIVER_DISPATCH ToasterDispatchIO; 

…

Including the role types as part of the Toaster sample is a significant “win” for our tools, since Toaster serves as an introduction to many new driver developers, and the template for a number of new drivers.  Each will allow developers to use SDV without significant additional work, enabling them to produce high quality drivers earlier in their development cycle.

Example of how the Annotations in WDM.H and OACR catch a bug on the developer desktop (PFD/SDV "Clean" 2 of 6)

Suppose you have code that acquires a spinlock and then attempts to wait before releasing that spinlock without specifying a timeout value:

NTSTATUS

MyDPCRoutineRunningAtDispatchLevel()

{

  // A whole bunch of code

  KeAcquireSpinLock(&myLock, &oldIrql);

  // More code

  KeWaitForSingleObject(&myTimer, Executive, KernelMode, FALSE, NULL);

  // Even more code

}

Because NULL is an acceptable value for the last parameter to KeWaitForSingleObject, the compiler won’t complain about it.  Shortly after the code is done and compiled, however, OACR pops up to warn me of potential issues, including an instance where I’m calling  KeWaitForSingleObject at an illegal IRQL.  Once I click on the OACR pop-up and bring up the Warning viewer, I see this:

Reading the warning shows that I created a demo driver that is a extremely simplified version of a mistake I’ve seen more than once.  Most driver developers know that waiting indefinitely at DISPATCH_LEVEL is a potentially fatal mistake, so it is unlikely that these two calls would appear so close to one another.  However, if “//more code” on line 59 was actually a long series of conditionals that resulted in the lock not being freed before the wait call, it may be difficult to see in a visual code review.  Another important point:  in my example code, I added no annotations, only making those calls to KeAcquireSpinLock and KeWaitForSingleObject at those various points, and OACR/PFD alerted me to this  extremely painful bug well before the code left my desktop.

The reason that PFD knew I was violating a basic rule of KeWaitForSingleObject is because of the annotations that are now present on those two functions in wdm.h:

__drv_maxIRQL(DISPATCH_LEVEL)

__drv_savesIRQL

__drv_setsIRQL(DISPATCH_LEVEL)

_DECL_HAL_KE_IMPORT

KIRQL

FASTCALL

KfAcquireSpinLock (

    __inout __deref __drv_acquiresExclusiveResource(KeSpinLockType)

    PKSPIN_LOCK SpinLock

    );

__drv_minIRQL(PASSIVE_LEVEL)

__drv_when((Timeout==NULL || *Timeout!=0), __drv_maxIRQL(APC_LEVEL))

__drv_when((Timeout!=NULL && *Timeout==0), __drv_maxIRQL(DISPATCH_LEVEL))

NTKERNELAPI

NTSTATUS

KeWaitForSingleObject (

    __in __deref __drv_notPointer PVOID Object,

    __in __drv_strictTypeMatch(__drv_typeCond) KWAIT_REASON WaitReason,

    __in __drv_strictType(KPROCESSOR_MODE/enum _MODE,__drv_typeConst)

    KPROCESSOR_MODE WaitMode,

    __in BOOLEAN Alertable,

    __in_opt PLARGE_INTEGER Timeout

    );

The bolded portions of each API declaration are important to the example above: 

1.       __drv_setsIRQL tells PFD that when this KeAcquireSpinLock (actually Kf…, due to #defines in wdm.h) is called, it sets the IRQL value to DISPATCH_LEVEL.

2.       When KeWaitForSingleObject is called and the timeout is either null or valid and not equal to zero (zero is return immediately), then KeWaitForSingleObject shouldn’t be called at dispatch level.   

The annotations in WDM enabled the developer to see this coding error in very short order, at least as compared to a manual review that would require him or her to a) walk through whatever logic “//more code” represented and b) remember that calling KeWait… at raised IRQL is bad, which also assumes he or she remembered that the spinlock acquisition also raised the IRQL in the first place.  Even if the developer is completely knowledgeable about all points in b), the savings on walking the code to find this issue cannot be ignored.

PFD and SDV Clean: Improving the Quality of Headers and Samples in the WDK and Windows 7 (Part 1 of 6)

Overview:

With the Windows 7 development process drawing to a close and the product slated to hit the shelves on October 22nd, we’re starting to look back and review the work our team has done.   One of our work items – a major one – was a quality-driven initiative we called “PFD/SDV Clean for Drivers”. 

Introduction:

This article from 2006 talks about our team using Static Driver Verifier (SDV) to find bugs in WDK Samples before the kit was published and get them addressed by the teams that owned the code.  That effort translates into savings for any vendor using this code for their own devices, as they don’t have a potentially very difficult bug to track down before (or after!) the release of their product.  That further benefits the end consumer in terms of stability of the OS and devices that using that sample code.  This thought process serves as the mission statement for the PFD/SDV Clean initiative.  Our goal was to establish a “quality bar” for headers and samples in the Win7 WDK, and further extend this bar to the Microsoft authored drivers that ship inbox with Windows 7.  The remainder of this and next few blog posts details the work our team did to make this effort successful.

PFD Clean (Annotating Headers):

Those of you familiar with PFD know that it is (basically) a function-level code checker that relies upon annotations for accuracy.  It was first included as a tool in the 2003 SP1 DDK, and I am of the opinion that it remained relatively anonymous to most of the developers using those kits.  The addition of OACR to the WDK build environments changes all of that.  OACR, which stands for “Microsoft Auto Code Review”, is a tool that resides in the background once a build environment is opened.  Once code is compiled, it uses idle time to scan that compiled code using the various code review plug-ins, one of which happens to be PREfast for Drivers (PFD).  Once OACR finishes scanning, it throws a pop-up box in the corner to show you any issues it considers warnings (bad) or errors (worse) with your recent compile.  This occurs with any build command in the standard WDK environments, no special commands or switches required:

This addition meant scanning “for free” for everyone that uses the Win7 WDK.  It also meant that if we wanted developers to consider those warnings and errors as anything more than a nuisance, we needed to make sure the code checker was as accurate as possible.  Thus began phase 1 of the “PFD Clean” initiative, annotation of public driver APIs. 

The RC version of the WDK includes 254 files in the inc\ddk folder, and 48 in the inc\wdf\* folders (Windows Driver Framework).  A number of these header files are compilations of smaller headers located in different places of the Windows source, because they are a group of API that are used for a single purpose – wdm.h, for example.  This means that multiple developers create the content that ultimately ends up in wdm.h, and each individual one gets a “work item” to add annotation to the APIs that they own.  In all, to cover the files in the kit that were thought to relate in some way to a driver (which includes some in inc\api as well), nearly 600 work items were filed asking for review and changes to the header files that are published into the WDK – one for each header file. 

Quick sidebar:  It became evident very quickly that asking developers to add annotation to their code would yield far better results than our local team reviewing and annotating their code:   the ramp-up time on the annotation language was far less than any ramp-up time needed to become familiar and comfortable with the code to be annotated.  The additional benefit of the developers gaining knowledge of this valuable technology was not ignored in making this decision, either.

The overall response to this task was fantastic – every header file that even remotely relates to drivers was reviewed and annotated appropriately (where/when needed).  Combined with OACR in the build environment, this creates immediate impact for any vendor that will use the Win7 WDK to build their driver.    If their code is calling one of these API in a manner that it shouldn’t be called in, the OACR pop-up says so.  In the next post (part 2 of the “Clean” series), I’ll provide an example of how these annotations and OACR combine to catch a potentially painful bug in the next post.

Static Analysis for Drivers Best Practices: Declare Entry Points of Your Driver Through Role Types – This Will Prepare It for Static Driver Verifier

Overview

The Windows Driver Kit gives you access to static analysis tools: PREfast for Drivers (PFD) and Static Driver Verifier (SDV).  The purpose of this article is to explain how to use PFD to validate that your driver correctly declares entry points through role types. This is necessary to make SDV effective in finding bugs.

Role types are function typedefs for driver entry points.  They are a way of standardizing function prototypes, which is required by the WDK and MSDN.  Role types act as documentation by describing the intended role of a driver entry point routine (or callback function) and the routine’s return value and parameter types.  Best of all, role types enable Static Driver Verifier which can discover complex inter-procedural bugs causing blue screens and system hangs.

As part of your regular driver development process, you should use both PFD and SDV.  We recommend that you run PFD first and address warnings about missing role types, and then run SDV.

PREfast for Drivers (PFD)

PREfast for Drivers will generate warnings regarding any missing role types, among other issues.  PFD runs quickly, and adding role types is fast and easy.  At the 2008 Microsoft Windows Driver Developer Conference, I worked with a customer who was brand new to the concept of SDV role types but wanted to run SDV on his driver.  We added all of the necessary role types and got a complete run of SDV on his driver in under an hour.

PFD will generate a list of warnings, including which role types are missing.  The warnings regarding missing role types for a WDM driver include:

·         Warning 28101: This warning specifies that PFD has detected the driver’s DriverEntry routine.  Declare this routine with the DRIVER_INITIALIZE role type.

·         Warning 28155: This warning specifies that a driver routine was not declared with the correct role type.

o   Declare driver unload routines with the DRIVER_UNLOAD role type

o   Declare I/O completion routines with IO_COMPLETION_ROUTINE role type

o   Declare add device routines with the DRIVER_ADD_DEVICE role type

o   Declare dispatch routines with the DRIVER_DISPATCH role type in conjunction with the __drv_dispatchType annotation

o   Declare interrupt service routines with the KSERVICE_ROUTINE role type

o   Declare DPC for ISR routines with the IO_DPC_ROUTINE role type

o   For other applicable role types, visit the WDK or MSDN role type documentation

·         Warning 28169:  This warning specifies that the driver is missing a __drv_dispatchType annotation on a dispatch routine.

o   Add the __drv_dispatchType annotation to the specified dispatch routine.  This annotation must be combined with the DRIVER_DISPATCH role type

For example, if the driver header file contains a declaration for the dispatch routine which handles PnP IRPs like this:

NTSTATUS
DriverDispatchPnP(
    IN PDEVICE_OBJECT  DeviceObject,
    IN PIRP  Irp
    );

After adding the appropriate annotations, the dispatch routine declaration will look like this:

__drv_dispatchType(IRP_MJ_PNP)
DRIVER_DISPATCH DriverDispatchPnP;

The process for adding role types for KMDF and NDIS drivers is similar.  In addition to running PFD to discover missing role types, use the WDK or MSDN documentation regarding role types to make sure the driver contains all of the required role types.

Static Driver Verifier (SDV)

SDV calls the entry points in the driver and attempts to find a valid code path which results in a system hang or bug check.  In order for SDV to recognize your driver’s entry points, you must declare them using role types as described above.

SDV assumes that no errors of the NULL pointer dereference type are generated when your driver is scanned with PFD.  In other words, it is recommended that you run SDV on your driver after the driver is PFD clean with respect to NULL pointer dereferences.

To run SDV, open a WDK build environment window and go to the directory which contains the sources file for your driver.  Then run these two commands:

staticdv /clean

staticdv /rule=*

The latter command will run all SDV checks on the driver.  SDV can also run with a single rule or a list of rules as specified in a configuration file which you create (config.sdv).  For a list of possible rules, run:

staticdv /showrules

When SDV has completed, view the results by running:

staticdv /view

For more information about using PFD to add role types in your driver, visit Make Static Driver Verifier More Efficient: Add a Preset Filter to PFD/OACR Defect Viewer.

For questions regarding SDV and PFD, please send email to sdvpfdex@microsoft.com.

Make Static Driver Verifier More Efficient: Add a Preset Filter to PFD/OACR Defect Viewer

Annotating for __success()

__success() is an annotation that hs been around for "a little while", but isn't widely known about.  Admittedly, finding content about it on MSDN is difficult - that should change as we approach the release of Windows 7.   Upon release, exposure to the __success() annotation is going to become very common with the release of the Windows 7 WDK: any function that returns NTSTATUS (read: a large majority of Kernel functions we publish in the WDK) will have the __success() annotation appended to it:

typedef __success(return >= 0) long NTSTATUS;

The annotation carries a dual meaning:  to the developer, it is an easy reminder of the return value (or set of return values) that indicate the function was successful.  Most NTSTATUS functions return 0 to indicate success, the greater-than or equal sign is present because a select few will return some positive number besides zero to indicate success.  The scanning engine expands on that feeling by assuming that  functions annotated with the __success() annotation will only fulfill their contract when returning a value that is considered to be “success”.  The following declaration will help demonstrate this meaning:

typedef

__success(return >= 0)

__drv_acquiresGlobalResource(SomeResource)

long

AcqResource (__out int *x);

When the scanner is processing a function that calls AcqResource, it will assume that SomeResource is acquired only when the function returns a success status.  This follows the a typical coding convention of only doing as the function says when the function is successful; the annotation is a strict reinforcement of that principle.  When the __success() annotation is not included, the scanner will always assume that the resource is acquired.  There is one caveat to keep in mind:  if x was not initialized before this call, the __out annotation stipulates that it will be initialized after this call, successful or not.  When dealing with the fail case, it is common that a fail status from the function means the failure to acquire the resource.    Whether or not the parameter is initialized post-call is a scenario of great debate, so the scanner assumes it was somehow initialized.   The best way to deal with this confusion?  Initialize the variable when you declare it, always a good idea in practice anyway.

Suppose, however, you have another function where you wish to express that an __out parameter would be returned to the caller as NULL in the case where the function failed.  The annotation __on_failure() exists for this reason, to be explained following this example:

typedef

__success (return >= 0)

__drv_acquiresGlobalResource(SomeResource)

long

AcqResourceAndX( __out __on_failure(__null) int* x)

                The __on_failure(__null) annotation states that on function exit, when the return value is less than 0 (__on_failure), assume that the value of x (the address at which the value accessed by *x is found) is null.    You may find that within your own coding style, __out parameters are always null or otherwise invalid in a fail case, and wonder why the scanner cannot go figure that out for itself.  Remember that the PFD scanner only deals with a function at a time, and has to rely solely on the annotations for information concerning a call within that function.  So while you may have code that obviously would set x to null on the failure instance, PFD will not know that unless you tell it specifically.

PFD Annotations - They're just comments, really.

As someone that has done development work, I know the joy experiecned when asked to better comment my own code after I’ve completed it.  It works, I know what it does, and I would rather work on a different feature than go back and change something I’ve already completed.  It doesn’t feel like progress; it feels like regress until I can clear all of that other work out.  I have these feelings knowing full well that comments are incredibly beneficial, and knowing that for as much as I have thought my own code to be clear and concise without the comments, revisiting the code even just a few months later requires time to become reacquainted with it when those comments aren’t there.  The case for commenting has been drilled into every aspiring Dev’s head somewhere along the way, so restating it beyond what I’ve already said isn’t necessary.

Both PFD and SDV make use of SAL (standard source code annotation language) annotations in the code to assist with their checking engines.  Both will use these to determine the correct usage of a function, and to warn the developer against things that will cause potential troubleshooting pain later on.  SAL annotations have been in place in some form for while – the easily recognizable “IN”, “OUT”, and “OPTIONAL” annotations are found in abundance in the current Windows 2008 WDK.   When code compiles, these resolve out to nothing, they’re simply in-code comments that help the developer get  a quick idea of what a particular parameter is used for.  The annotations for PFD and SDV take this to another level in both code readability and usability of these tools.

Take an example of a function, defined in wdm.h in the Windows 2008 WDK, that seems reasonably easy to determine what it does, based on it’s name:

NTKERNELAPI

NTSTATUS

NTAPI

KeSaveFloatingPointState (

    OUT PKFLOATING_SAVE     FloatSave

    );

This function is to be used whenever floating point arithmetic of any sort is to happen in Kernel code.  Calling this before doing any sort of operation relieves you of the possibility of nasty little issues that can occur with the floating point  registers if you’re switched out while executing your codepath.  Based on what is seen above, we know it’s a Kernel API, it returns NTSTATUS and a structre we will use when we need to call KeRestoreFloatingPointState to restore it.   Neither wdm.h or fltsafe.h (where the FLOATSAFE struct is defined) provides any other real hint as to concerns we should have for this function call.  A trip to the MSDN page for KeSaveFloatingPointState is necessary to tell me that it’s okay to call this at <=DISPATCH_LEVEL.

Consider the information provided when appropriate PFD annotations are added:

__checkReturn

__drv_maxIRQL(DISPATCH_LEVEL)

__drv_valueIs(<0;==0)

__drv_when(return==0, __drv_floatSaved)

__forceinline

NTSTATUS

KeSaveFloatingPointState (

    __out __deref __drv_neverHold(FloatState)

    __drv_when(return==0, __deref __drv_acquiresResource(FloatState))

    PVOID FloatingState

    )

Based on this information, I now know, without a trip to my browser and MSDN:

-          It is not safe to assume this function is always successful.  While the fact that it returns a value of any sort should cement this belief anyway, the annotation “__checkReturn” acts as the concrete. 

-          The maximum IRQL level that this function can be called at is DISPATCH_LEVEL, so it’s safe to call from a DPC, but not my ISR; and should relieve me of the worry of paged code when calling it.

-          The value returned is either less than 0 or 0.

-          When the value returned is zero, the floating point state is saved.

The parameters themselves also hold annotations, which admittedly take a bit more getting used to than looking past simple “IN” or “OUT” annotations.  The annotations applied to FloatingState translate out to:

-          When this function returns STATUS_SUCCESS, it acquires the resource that we’ll call ‘FloatState’.  With that in mind, the ‘FloatState’ resource should never be held when calling this function.

-          The __deref is part of properly pointing to the resource that the FloatingState pointer is pointing to. 

-          The __out annotation is self-explainatory:  we’re expecting this function to provide us with something useful to point to with that parameter.

For informational purposes only, those annotations realized the same effect as a simple comment placed before the function defination would have.  Review of the SAL annotations leaves a developer with a far better idea of what an API is capable of, regardless of any documentation they may have found on the subject.   For example, the annotation makes it apparent that holding the FloatState and attempting to save it again is not a good idea; the MSDN write-up doesn’t mention this. 

Annotations can range from very complex and situational down to “duh” for any programmer.  Checking the return value of a function is generally a good idea, but you can not do so and nothing in the compiler will stop you.  You can also try to drink coffee in large gulps the instant the Barista hands it to you, too, but that doesn’t mean it’s a good idea.  It’s why coffee cups have “CAUTION: HOT” written on them these days, and why an annotation like “__checkReturn” exists.  They’re safety measures, and help you keep yourself from getting hurt if you follow them, no matter how trivial they may be.  The __checkReturn annotation creates a warning for anyone that decides to ignore the status, and much like “CAUTION:HOT”, they can continue to do so possibly at their own peril - I’ve seen a number of crashes simply because a status was ignored. 

Whether they’re to be considered fancy comments or warning signs, the benefits of these annotations especially when paired with PFD result in APIs used in their intended manner and less crashes on the user desktop.  Problems that can create crashes are brought to the developer’s attention during development, not when users of the beta are clamoring for a fix – or even worse, after release.  Application of these annotations for a developer familiar with their DDI is minimal, the savings potentially incredible.  As a developer, if applying these means finding a bug in my code before releasing it, the time invested in application has paid for itself already – for basically nothing more than providing information that I should be putting in comments anyway.

PREfast for Drivers and IRQL Levels

IRQL levels can sometimes be a tricky thing to get right, especially if you’re new to drivers and coming from user mode.  Playing with IRQL levels, memory, structures… Mix them the wrong way and you bugcheck the box.   What can really be painful is the process of inheriting code and having to figure out what’s going on.  Did the developer want this to come in at DISPATCH_LEVEL?   With all the complexities of writing drivers using driver annotations is almost a necessity. 

Let’s take this example:

Off the top of your head without looking at the documentation you may or may not be able to determine what the IRQL level should be when you call this.   Now can you be sure from all the different points in your code where this is called that you are at the correct IRQL?   

Now let’s add one line of driver annotation to the same prototype:

You can see the obvious benefit in the code of the documentation.   I could give this to my 9 yr old nephew and ask him what IRQL level this DDI requires.   Here’s the other benefit: You can actually use PFD to check if this is valid in different areas of your code.   Any place this is called PFD can detect it from one of two ways.  

1.       Annotation inference from another DDI call that either requires a certain IRQL level or raises/lowers the IRQL level.  Looking at another DDI it can infer that the logic between the two DDI’s doesn’t make sense.  (DDI-1 requires PASSIVE_LEVEL while DDI-2 four lines down requires APC_LEVEL)

2.       It allows someone anyone who may have inherited the code to immediately understand the contract requirements and constraints on the DDI.  Obviously, the code above has been shrunk down for simplicity, but you can see how this would be beneficial for a more complicated DDI.

This is something very easy that you can add to your code that will help with code portability and maintenance.    If you’re all excited to jump into the next set of annotations you can feel free to read up more here:  http://msdn2.microsoft.com/en-us/library/cc264089.aspx

Welcome to the Static Driver Tools Blog!

Welcome to the Static Driver Tools blog!  This is a blog that was created by our team to inform the driver development community of upcoming features and changes to Static Driver Verifier (SDV) and PREfast for Drivers (PFD). 

If you’re not familiar with the tools you can read more about each tool here:

PREfast for Drivers:

http://www.microsoft.com/whdc/devtools/tools/PREfast.mspx

Static Driver Verifier:

http://www.microsoft.com/whdc/devtools/tools/sdv.mspx

We’re looking forward to helping out with the tools, investigating bug traces, or just talking about developing, debugging, and testing drivers.