Pointless Blathering : Device Drivers General

New KMDF white-paper out on writing a bus driver

PeterWie — Tue, 21 Oct 2008 20:48:59 GMT

Penny Orwick has been working on a white paper that talks abut using KMDF to write a bus driver. It’s been stalled waiting for us lazy developers to review it, but it finally came out last night:

http://www.microsoft.com/whdc/driver/wdf/KMDFBusDrv.mspx

Measuring DPC time

PeterWie — Tue, 07 Oct 2008 03:59:00 GMT

At the DDC this year we had some very useful “community forums”. I generally come out of these feeling like I've been beaten with a sack of oranges, and this year was no exception. But one question in particular struck my eye – someone was saying that MS didn’t provide any useful tools for figuring out why the DPC latency they were seeing on notebooks was so poor. When he asked about this my first thought was “I bet XPerf can do that”.

XPerf, for those who haven’t seen it, is part of a performance toolkit that the Windows performance team released. It collects ETW events reported by various system components and then provides an … interesting … UI to pull all of those events together into graphs and visualize what in the world is going on.

The down-side of XPerf is that it can be very hard to use. Every time I approach this tool I think “wow that’s a lot of data to make sense of”. But I would think that we can use it to collect information about what DPCs are running and how long they run for.

Once I installed XPerf I went into the installation directory and looked at the tools that it provides. There are two interesting tools – XPerf.exe and xperfview.exe. The first allows you to easily start and stop trace sessions, merge trace logs and do some data processing on the resulting logs. The second is the GUI visualization tool and lets you see your trace data. Let’s start with XPerf:

C:\xperf>xperf

    Microsoft (R) Windows (R) Performance Analyzer Version 4.1.6512

    Performance Analyzer Command Line

    Copyright (c) Microsoft Corporation. All rights reserved.

    Usage: xperf options ...

        xperf -help view             for xperfview, the graphical user interface

        xperf -help start            for logger start options

        xperf -help providers        for known tracing flags

        xperf -help stackwalk        for stack walking options

        xperf -help stop             for logger stop options

        xperf -help merge            for merge multiple trace files

        xperf -help processing       for trace processing options

        xperf -help symbols          for symbol decoding configuration

        xperf -help query            for query options

        xperf -help mark             for mark and mark-flush

        xperf -help format           for time and timespan formats on the command line

Well we want to use this to start a trace session, so let’s start with xperf –help start:

C:\xperf>xperf -help start

    Trace start options:

    xperf [-start|-update [LoggerName]] -on Flags|Groups options ...

            or -flush [LoggerName] ...

            or -SetProfInt [<n>] [cached]

    User provider format for -on: (GUID|KnownProviderName)[:Flags[:Level]]

        -start      LoggerName       Start a logging session for LoggerName

        -update     LoggerName       Update a logging session for LoggerName

        -flush      LoggerName       Flush a logging session for LoggerName

        -on         Flags|Groups     For kernel logging sessions, the sequence of

                                     kernel flags and groups to be enabled,

                                     separated by '+'.  For user logging

                                     sessions, the sequence of providers to be

                                     enabled, separated by '+'.  The accepted

                                     provider format is:

                                     (GUID|KnownProviderName)[:Flags[:Level]].

                                     Use "xperf -help providers" for a list of

                                     valid flags

        -f          filename         Log events to specified file; default:

                                     \kernel.etl for kernel traces and \user.etl

                                     for user traces

        -BufferSize Size             Set trace buffer size to Size KB (4 KB -

                                     1024 KB); default: 64 KB

        -MinBuffers n                Set minimum number of trace buffers to n;

                                     default: 64

        -MaxBuffers n                Set maximum number of trace buffers to n;

                                     default: 320

        -MaxFile    Size             Set maximum file size to Size MB

        -FlushTimer n                Set the flush timer to n seconds

        -BootTrace  Flags|Groups|off Config the Event Tracing For Windows Logger

                                     to trace boot.  Set flags as "off" to turn

                                     off boot tracing.  All loggering control can

                                     be used in conjunction with this.  Use in

                                     conjunction with -f to log to a file other

                                     than \Perf.etl.

        -RealTime                    Enable real time tracing

        -Buffering                   Enable buffering mode tracing

        -FileMode   Mode             Set the File Mode; default: Sequential.

                                     Mode: Sequential, Circular, Append, NewFile

        -ClockType  ClockType        Set the clock type; default: PerfCounter.

                                     Type: Cycle, PerfCounter, SystemTime.

        -StackWalk  flags|@file      Enable stack walking for the events

                                     specified as Flag+Flag+..., or parse 'file'

                                     file for flags. Run "xperf -help stackwalk"

                                     for more information.

        -Pids       pid [...]        Apply flags to processes pid [...].  Used in

                                     conjunction with private loggers.

        -PidNewProcess <command-line>   Apply flags to a new process xperf will

                                     start with <command-line>.  Used in

                                     conjunction with private loggers.

        -heap                        Enable heap tracing in processes specified

                                     by Pids and PidNewProcess.

        -critsec                     Enable critical section tracing in processes

                                     specified by Pids and PidNewProcess.

        -SetProfInt [<n>] [cached]   Set sampled profile interval to <n> [1221 ..

                                     10000000].  If "cached" is specified,

                                     intervals are cached in ETW and reapplied

                                     whenever new ETW kernel loggers with sampled

                                     profile are started. [Default: <n> = 10000;

                                     not cached]

    Multiple loggers can be started using multiple -start options, each followed by

    the options to be applied to that logger.  If LoggerName, or -start LoggerName,

    is omitted, the Kernel Logger is implied.  Only a single instance of Kernel

    Logger can exist at any time.  If one of the loggers fails to start, all the

    already started loggers are stopped.

The only required argument to –start is a set of flags. The definition of Flags says to look at xperf providers to see what the possible flags are. The help for that command says:

C:\xperf>xperf -help providers

    Providers Query options:

    xperf -providers [Installed|I] [Registered|R] [KernelFlags|KF] [KernelGroups|KG] [Kernel|K]

    Query all installed/known and registered providers, as well as all known kernel flags and groups.

    The following options are supported:

        I,  Installed    : include Installed/known providers

        R,  Registered   : include Registered providers

        KF, KernelFlags  : include Kernel Flags

        KG, KernelGroups : include Kernel Groups

        K,  Kernel       : include Kernel flags and groups; shortcut for "KF KG".

    If no options are specified, all providers are included in the output.

We want to look at DPC’s, so let’s dump just the kernel providers (if only because dumping all of them would probably exceed what I can post in one blog entry ).

C:\xperf>xperf -providers K

Kernel Flags:

       PROC_THREAD    : Process and Thread create/delete

       LOADER         : Kernel and user mode Image Load/Unload events

       PROFILE        : CPU Sample profile

       CSWITCH        : Context Switch

       COMPACT_CSWITCH: Compact Context Switch

       DISPATCHER     : CPU Scheduler

       DPC            : DPC Events

       INTERRUPT      : Interrupt events

       SYSCALL        : System calls

       PRIORITY       : Priority change events

       ALPC           : Advanced Local Procedure Call

       PERF_COUNTER   : Process Perf Counters

       DISK_IO        : Disk I/O

       DISK_IO_INIT   : Disk I/O Initiation

       FILE_IO        : File system operation end times and results

       FILE_IO_INIT   : File system operation (create/open/close/read/write)

       HARD_FAULTS    : Hard Page Faults

       FILENAME       : FileName (e.g., FileName create/delete/rundown)

       SPLIT_IO       : Split I/O

       REGISTRY       : Registry tracing

       DRIVERS        : Driver events

       POWER          : Power management events

       NETWORKTRACE   : Network events (e.g., tcp/udp send/receive)

       VIRT_ALLOC     : Virtual allocation reserve and release

       MEMINFO        : Memory List Info

       ALL_FAULTS     : All page faults including hard, Copy on write, demand zero faults, etc.

Kernel Groups:

       Base           : PROC_THREAD+LOADER+DISK_IO+HARD_FAULTS+PROFILE+MEMINFO

       Diag           : PROC_THREAD+LOADER+DISK_IO+HARD_FAULTS+DPC+INTERRUPT+CSWITCH+PERF_COUNTER+COMPACT_CSWITCH

       DiagEasy       : PROC_THREAD+LOADER+DISK_IO+HARD_FAULTS+DPC+INTERRUPT+CSWITCH+PERF_COUNTER

       Latency        : PROC_THREAD+LOADER+DISK_IO+HARD_FAULTS+DPC+INTERRUPT+CSWITCH+PROFILE

       FileIO         : PROC_THREAD+LOADER+DISK_IO+HARD_FAULTS+FILE_IO+FILE_IO_INIT

       IOTrace        : PROC_THREAD+LOADER+DISK_IO+HARD_FAULTS+CSWITCH

       ResumeTrace    : PROC_THREAD+LOADER+DISK_IO+HARD_FAULTS+PROFILE+POWER

       SysProf        : PROC_THREAD+LOADER+PROFILE

       Network        : PROC_THREAD+LOADER+NETWORKTRACE

That “Latency” group sounds interesting – it includes DPCs and interrupts along with process/thread create/delete and loader events (both of which it turns out are needed if you want to see what process anything is happening in). So let’s start up a trace session for “Latency”.

C:\xperf>xperf -on Latency

Now to create some “load” I'll use WMP to watch the stock footage of a bear in the river. Then I exit WMP and stop the trace:

C:\xperf>xperf -stop

C:\xperf>dir \kernel.etl

 Volume in drive C is Safe

 Volume Serial Number is 042D-D2B2

 Directory of C:\

10/06/2008  04:52 PM        16,384,000 kernel.etl

               1 File(s)     16,384,000 bytes

               0 Dir(s)  57,208,664,064 bytes free

By default the kernel logger dumped all 16 MB of output into c:\kernel.etl. Now if we want to see what’s in that file we can load it up with xperfview:

C:\xperf>xperfview \kernel.etl

These are two of 15 different graphs that XPerf makes available from the .ETL file. Here we can see the CPU time for both of my CPUs as well as a count of disk I/O operations in flight over the collection period. I have pulled out the menu of “frames” that you can enable (clicking on the chevron in the middle will make it disappear or reappear).

Let’s say that I want to drill in on this a little bit, and look at CPU time just for a single process – say WMPLAYER.EXE since I was watching a video. I can select the “CPU Sampling by Process” option from the frame list:

And then click the down arrow next to “Processes” and select just WMPLAYER.EXE (The short cut to do this is to click All, then click it again to unselect everything, then scroll and find just the one or two processes that you need"). If we do this with Disk Utilization as well, and remove the intervening frames we can get a side-by-side view of CPU activity and disk I/O for one process that looks like this:

We can also overlay the DPC activity on top of the CPU graph if we want. Right click in the CPU Sampling by processor frame and choose “Overlay Graph” then “DPC CPU Usage” and “All” processors:

The red and blue lines are CPU 0 and CPU 1 respectively, with the yellow line still showing normal CPU utilization for WMPLAYER.EXE.

XPerfView also has the ability to show you your data in pivot-table form. Let’s say we want to look at DPC CPU Utilization. Right click on any frame showing DPC data (either the merged graph we just setup above, or re-enable the DPC frame from the frame list) and choose “Summary Table” and you’ll get something like this:

This is showing us every DPC event in the .ETL file. The columns on the left of the yellow line control how the data is grouped and the columns to the right of the line show us the aggregate results for that grouping. Here we’re grouped by module name and we can see the total count and average and max duration of all of the DPCs for that module. NDIS stands out with a 2.6 ms DPC. If you want to see every DPC in a particular module (sorted by “Enter Time”) you can click the + box next to the name and the view will expand.

Let’s say we want to see all the entries grouped not just by module but by the actual DPC routine within that module. The function column shows us the address of the DPC routine that was executed. If we drag that to the right of the yellow line we get a new grouping:

For each module we can see which DPC function is being called the most and which one is taking up so much time. You can use this to find out other things, like which threads in WMP are taking up time and what thread routine they are running. Go back and enable the CPU Utilization by processor frame then get a summary table for that:

This would all be more impressive if symbol loading were working. In theory XPerf can load symbols and use them to determine the name of the function rather than just showing an address or “Unknown”. However I’ve rarely gotten symbol loading to work, and when it does work it still seems pretty fragile. If you use xperf –help symbols it provides some help on getting symbols working.

There’s one more handy trick that I found while I was trying to get symbols to work. XPerf.exe has some processing options too. XPerf –help processing shows us a list of the options:

C:\xperf>xperf -help processing

    Trace Post-Processing options:

        xperf -I <trace file> ... [-o output] [-symbols ...] [-target {human|machine}]

                           [-a action ... [-a action ...] ...]

        -I       trace file       The trace file to be processed.

        -o       output file      Optional, the name of the file that output goes to.  If not

                                  given, stdout is used.

        -symbols [options]        Enable and configure symbol decoding support.

 See "xperf

                                  -help symbols" for detailed help

        -target  {human|machine}  Select the target audience of the output. Default is human.

        -quiet                    Do not print progress information.

        -tle                      Process the trace even in the presence of lost events.

        -tti                      Process the trace even in the presence of time inversions.

        -a       action ...       Optional, the actions to take.  Default action is to dump

                                  the event in text form.

    Examples:

        xperf -I trace.etl -o out.csv      : dump the events in trace.etl to file out.csv

        xperf -help registry               : print help on action registry

        xperf -I trace.etl -a registry     : print registry access statistics to stdout

        xperf -I trace32.etl trace64.etl -o out.csv : dump the events in trace32.etl and trace64.etl to file out.csv

    Available Actions:

        boot             Show boot and shutdown statistics

        bootprefetch     Show boot-time prefetching information

        cpudisk          Show CPU/Disk activity report

        cswitch          Show Context Switch data

        diskio           Show Disk IO Statistics

        dpcisr           Show DPC/ISR Statistics

        drvdelay         Show Driver delays

        dumper           Dump events in the trace in text form

        filename         Show File Names in the trace

        focuschange      Show the Windows thread focus change events in the trace

        hardfault        Show hard fault statistics by process and file.

        marks            Show Marks Information

        pagefault        Show page fault information in the trace

        perfctrs         Show process performance counters.

        pnp              Show PnP events in the trace

        prefetch         Show Prefetch Information

        process          Show process, thread, image information

        profile          Show Sampled profiler data

        readyBoot        Show ReadyBoot statistics

        registry         Show Registry Access Statistics

        services         Show service status information

        shutdown         Show shutdown statistics

        stack            Show Stack Information

        suspend          Show Suspend Transition Information

        sysconfig        Show System Configuration

        tracestats       Show Trace Statistics

        winlogon         Show Winlogon events in the trace

    xperf -help <action> [<action> ...] for detailed help

    If no action is present, dumper will be invoked.

Since this all started with DPC latency, let’s try out that dpcisr action, which will “Show DPC/ISR Statistics” and see what it has:

C:\xperf>xperf -I \kernel.etl -symbols verbose -a dpcisr

xperf: Using symbol path: srv*c:\DownstreamStore*http://msdl.microsoft.com/download/symbols

xperf: Using executable path: srv*c:\DownstreamStore*http://msdl.microsoft.com/download/symbols

xperf: Using SymCache path: c:\DownstreamStore

--------------------------

DPC Info

--------------------------

CPU Usage Summing By Module For the Whole Trace

CPU Usage from 0 us to 24682000 us:

     CPU 0 Usage      CPU 1 Usage

     usec      %      usec      % Module

      248   0.00         0   0.00 CLASSPNP.SYS

       31   0.00         0   0.00 HTTP.sys

      562   0.00         0   0.00 NETIO.SYS

       17   0.00         0   0.00 Ntfs.sys

     5121   0.02         0   0.00 USBPORT.SYS

      273   0.00         0   0.00 afd.sys

    16752   0.07       135   0.00 ataport.SYS

      101   0.00        36   0.00 b57nd60x.sys

        1   0.00         0   0.00 fltmgr.sys

        5   0.00         0   0.00 hal.dll

     3851   0.02         0   0.00 i8042prt.sys

       14   0.00         0   0.00 luafv.sys

   981815   3.98         0   0.00 ndis.sys

      239   0.00         0   0.00 netbt.sys

     2364   0.01       227   0.00 ntkrnlpa.exe

    37360   0.15         0   0.00 portcls.sys

      119   0.00       103   0.00 rassstp.sys

      317   0.00         0   0.00 rdbss.sys

       27   0.00        17   0.00 srv.sys

        4   0.00         0   0.00 srvnet.sys

    62926   0.25     49358   0.20 tcpip.sys

     5054   0.02         0   0.00 usbhub.sys

        6   0.00         0   0.00 watchdog.sys

Total = 43599

Elapsed Time, >        0 usecs AND <=        1 usecs,     49, or   0.11%

Elapsed Time, >        1 usecs AND <=        2 usecs,    379, or   0.87%

Elapsed Time, >        2 usecs AND <=        4 usecs,  10388, or  23.83%

Elapsed Time, >        4 usecs AND <=        8 usecs,   9765, or  22.40%

Elapsed Time, >        8 usecs AND <=       16 usecs,   7806, or  17.90%

Elapsed Time, >       16 usecs AND <=       32 usecs,   3885, or   8.91%

Elapsed Time, >       32 usecs AND <=       64 usecs,   5375, or  12.33%

Elapsed Time, >       64 usecs AND <=      128 usecs,   5175, or  11.87%

Elapsed Time, >      128 usecs AND <=      256 usecs,    639, or   1.47%

Elapsed Time, >      256 usecs AND <=      512 usecs,     55, or   0.13%

Elapsed Time, >      512 usecs AND <=     1024 usecs,     55, or   0.13%

Elapsed Time, >     1024 usecs AND <=     2048 usecs,     22, or   0.05%

Elapsed Time, >     2048 usecs AND <=     4096 usecs,      6, or   0.01%

Total,                                                 43599

Total = 10 for module CLASSPNP.SYS

Elapsed Time, >        0 usecs AND <=        1 usecs,      2, or  20.00%

Elapsed Time, >        1 usecs AND <=        2 usecs,      3, or  30.00%

Elapsed Time, >        2 usecs AND <=        4 usecs,      0, or   0.00%

Elapsed Time, >        4 usecs AND <=        8 usecs,      0, or   0.00%

Elapsed Time, >        8 usecs AND <=       16 usecs,      0, or   0.00%

Elapsed Time, >       16 usecs AND <=       32 usecs,      0, or   0.00%

Elapsed Time, >       32 usecs AND <=       64 usecs,      5, or  50.00%

Total,                                                    10

Total = 8 for module HTTP.sys

Elapsed Time, >        0 usecs AND <=        1 usecs,      0, or   0.00%

Elapsed Time, >        1 usecs AND <=        2 usecs,      1, or  12.50%

Elapsed Time, >        2 usecs AND <=        4 usecs,      3, or  37.50%

Elapsed Time, >        4 usecs AND <=        8 usecs,      4, or  50.00%

Total,                                                     8

Total = 124 for module NETIO.SYS

Elapsed Time, >        0 usecs AND <=        1 usecs,      0, or   0.00%

Elapsed Time, >        1 usecs AND <=        2 usecs,      0, or   0.00%

Elapsed Time, >        2 usecs AND <=        4 usecs,     57, or  45.97%

Elapsed Time, >        4 usecs AND <=        8 usecs,     61, or  49.19%

Elapsed Time, >        8 usecs AND <=       16 usecs,      5, or   4.03%

Elapsed Time, >       16 usecs AND <=       32 usecs,      1, or   0.81%

Total,                                                   124

Total = 5 for module Ntfs.sys

Elapsed Time, >        0 usecs AND <=        1 usecs,      0, or   0.00%

Elapsed Time, >        1 usecs AND <=        2 usecs,      0, or   0.00%

Elapsed Time, >        2 usecs AND <=        4 usecs,      4, or  80.00%

Elapsed Time, >        4 usecs AND <=        8 usecs,      1, or  20.00%

Total,                                                     5

Total = 481 for module USBPORT.SYS

Elapsed Time, >        0 usecs AND <=        1 usecs,      0, or   0.00%

Elapsed Time, >        1 usecs AND <=        2 usecs,      0, or   0.00%

Elapsed Time, >        2 usecs AND <=        4 usecs,     81, or  16.84%

Elapsed Time, >        4 usecs AND <=        8 usecs,     64, or  13.31%

Elapsed Time, >        8 usecs AND <=       16 usecs,    257, or  53.43%

Elapsed Time, >       16 usecs AND <=       32 usecs,     76, or  15.80%

Elapsed Time, >       32 usecs AND <=       64 usecs,      3, or   0.62%

Total,                                                   481

Total = 44 for module afd.sys

Elapsed Time, >        0 usecs AND <=        1 usecs,      0, or   0.00%

Elapsed Time, >        1 usecs AND <=        2 usecs,      1, or   2.27%

Elapsed Time, >        2 usecs AND <=        4 usecs,      3, or   6.82%

Elapsed Time, >        4 usecs AND <=        8 usecs,     30, or  68.18%

Elapsed Time, >        8 usecs AND <=       16 usecs,      9, or  20.45%

Elapsed Time, >       16 usecs AND <=       32 usecs,      1, or   2.27%

Total,                                                    44

Total = 736 for module ataport.SYS

Elapsed Time, >        0 usecs AND <=        1 usecs,      0, or   0.00%

Elapsed Time, >        1 usecs AND <=        2 usecs,      0, or   0.00%

Elapsed Time, >        2 usecs AND <=        4 usecs,     15, or   2.04%

Elapsed Time, >        4 usecs AND <=        8 usecs,     28, or   3.80%

Elapsed Time, >        8 usecs AND <=       16 usecs,    207, or  28.13%

Elapsed Time, >       16 usecs AND <=       32 usecs,    335, or  45.52%

Elapsed Time, >       32 usecs AND <=       64 usecs,    139, or  18.89%

Elapsed Time, >       64 usecs AND <=      128 usecs,     12, or   1.63%

Total,                                                   736

Total = 50 for module b57nd60x.sys

Elapsed Time, >        0 usecs AND <=        1 usecs,      0, or   0.00%

Elapsed Time, >        1 usecs AND <=        2 usecs,     23, or  46.00%

Elapsed Time, >        2 usecs AND <=        4 usecs,     19, or  38.00%

Elapsed Time, >        4 usecs AND <=        8 usecs,      6, or  12.00%

Elapsed Time, >        8 usecs AND <=       16 usecs,      1, or   2.00%

Elapsed Time, >       16 usecs AND <=       32 usecs,      1, or   2.00%

Total,                                                    50

Total = 1 for module fltmgr.sys

Elapsed Time, >        0 usecs AND <=        1 usecs,      0, or   0.00%

Elapsed Time, >        1 usecs AND <=        2 usecs,      1, or 100.00%

Total,                                                     1

Total = 1 for module hal.dll

Elapsed Time, >        0 usecs AND <=        1 usecs,      0, or   0.00%

Elapsed Time, >        1 usecs AND <=        2 usecs,      0, or   0.00%

Elapsed Time, >        2 usecs AND <=        4 usecs,      0, or   0.00%

Elapsed Time, >        4 usecs AND <=        8 usecs,      1, or 100.00%

Total,                                                     1

Total = 329 for module i8042prt.sys

Elapsed Time, >        0 usecs AND <=        1 usecs,      0, or   0.00%

Elapsed Time, >        1 usecs AND <=        2 usecs,      0, or   0.00%

Elapsed Time, >        2 usecs AND <=        4 usecs,      0, or   0.00%

Elapsed Time, >        4 usecs AND <=        8 usecs,      9, or   2.74%

Elapsed Time, >        8 usecs AND <=       16 usecs,    298, or  90.58%

Elapsed Time, >       16 usecs AND <=       32 usecs,     22, or   6.69%

Total,                                                   329

Total = 5 for module luafv.sys

Elapsed Time, >        0 usecs AND <=        1 usecs,      0, or   0.00%

Elapsed Time, >        1 usecs AND <=        2 usecs,      1, or  20.00%

Elapsed Time, >        2 usecs AND <=        4 usecs,      3, or  60.00%

Elapsed Time, >        4 usecs AND <=        8 usecs,      1, or  20.00%

Total,                                                     5

Total = 32833 for module ndis.sys

Elapsed Time, >        0 usecs AND <=        1 usecs,      0, or   0.00%

Elapsed Time, >        1 usecs AND <=        2 usecs,    178, or   0.54%

Elapsed Time, >        2 usecs AND <=        4 usecs,   9073, or  27.63%

Elapsed Time, >        4 usecs AND <=        8 usecs,   7653, or  23.31%

Elapsed Time, >        8 usecs AND <=       16 usecs,   4082, or  12.43%

Elapsed Time, >       16 usecs AND <=       32 usecs,   1930, or   5.88%

Elapsed Time, >       32 usecs AND <=       64 usecs,   4425, or  13.48%

Elapsed Time, >       64 usecs AND <=      128 usecs,   4727, or  14.40%

Elapsed Time, >      128 usecs AND <=      256 usecs,    627, or   1.91%

Elapsed Time, >      256 usecs AND <=      512 usecs,     55, or   0.17%

Elapsed Time, >      512 usecs AND <=     1024 usecs,     55, or   0.17%

Elapsed Time, >     1024 usecs AND <=     2048 usecs,     22, or   0.07%

Elapsed Time, >     2048 usecs AND <=     4096 usecs,      6, or   0.02%

Total,                                                 32833

Total = 10 for module netbt.sys

Elapsed Time, >        0 usecs AND <=        1 usecs,      0, or   0.00%

Elapsed Time, >        1 usecs AND <=        2 usecs,      0, or   0.00%

Elapsed Time, >        2 usecs AND <=        4 usecs,      0, or   0.00%

Elapsed Time, >        4 usecs AND <=        8 usecs,      0, or   0.00%

Elapsed Time, >        8 usecs AND <=       16 usecs,      0, or   0.00%

Elapsed Time, >       16 usecs AND <=       32 usecs,     10, or 100.00%

Total,                                                    10

Total = 597 for module ntkrnlpa.exe

Elapsed Time, >        0 usecs AND <=        1 usecs,     43, or   7.20%

Elapsed Time, >        1 usecs AND <=        2 usecs,    129, or  21.61%

Elapsed Time, >        2 usecs AND <=        4 usecs,    344, or  57.62%

Elapsed Time, >        4 usecs AND <=        8 usecs,     50, or   8.38%

Elapsed Time, >        8 usecs AND <=       16 usecs,      4, or   0.67%

Elapsed Time, >       16 usecs AND <=       32 usecs,      3, or   0.50%

Elapsed Time, >       32 usecs AND <=       64 usecs,     23, or   3.85%

Elapsed Time, >       64 usecs AND <=      128 usecs,      1, or   0.17%

Total,                                                   597

Total = 2087 for module portcls.sys

Elapsed Time, >        0 usecs AND <=        1 usecs,      0, or   0.00%

Elapsed Time, >        1 usecs AND <=        2 usecs,      0, or   0.00%

Elapsed Time, >        2 usecs AND <=        4 usecs,      1, or   0.05%

Elapsed Time, >        4 usecs AND <=        8 usecs,    216, or  10.35%

Elapsed Time, >        8 usecs AND <=       16 usecs,    675, or  32.34%

Elapsed Time, >       16 usecs AND <=       32 usecs,   1076, or  51.56%

Elapsed Time, >       32 usecs AND <=       64 usecs,    119, or   5.70%

Total,                                                  2087

Total = 100 for module rassstp.sys

Elapsed Time, >        0 usecs AND <=        1 usecs,      0, or   0.00%

Elapsed Time, >        1 usecs AND <=        2 usecs,     37, or  37.00%

Elapsed Time, >        2 usecs AND <=        4 usecs,     62, or  62.00%

Elapsed Time, >        4 usecs AND <=        8 usecs,      1, or   1.00%

Total,                                                   100

Total = 99 for module rdbss.sys

Elapsed Time, >        0 usecs AND <=        1 usecs,      0, or   0.00%

Elapsed Time, >        1 usecs AND <=        2 usecs,      0, or   0.00%

Elapsed Time, >        2 usecs AND <=        4 usecs,     87, or  87.88%

Elapsed Time, >        4 usecs AND <=        8 usecs,      8, or   8.08%

Elapsed Time, >        8 usecs AND <=       16 usecs,      4, or   4.04%

Total,                                                    99

Total = 13 for module srv.sys

Elapsed Time, >        0 usecs AND <=        1 usecs,      0, or   0.00%

Elapsed Time, >        1 usecs AND <=        2 usecs,      1, or   7.69%

Elapsed Time, >        2 usecs AND <=        4 usecs,      8, or  61.54%

Elapsed Time, >        4 usecs AND <=        8 usecs,      4, or  30.77%

Total,                                                    13

Total = 2 for module srvnet.sys

Elapsed Time, >        0 usecs AND <=        1 usecs,      0, or   0.00%

Elapsed Time, >        1 usecs AND <=        2 usecs,      1, or  50.00%

Elapsed Time, >        2 usecs AND <=        4 usecs,      1, or  50.00%

Total,                                                     2

Total = 4998 for module tcpip.sys

Elapsed Time, >        0 usecs AND <=        1 usecs,      1, or   0.02%

Elapsed Time, >        1 usecs AND <=        2 usecs,      0, or   0.00%

Elapsed Time, >        2 usecs AND <=        4 usecs,     48, or   0.96%

Elapsed Time, >        4 usecs AND <=        8 usecs,   1242, or  24.85%

Elapsed Time, >        8 usecs AND <=       16 usecs,   2179, or  43.60%

Elapsed Time, >       16 usecs AND <=       32 usecs,    421, or   8.42%

Elapsed Time, >       32 usecs AND <=       64 usecs,    660, or  13.21%

Elapsed Time, >       64 usecs AND <=      128 usecs,    435, or   8.70%

Elapsed Time, >      128 usecs AND <=      256 usecs,     12, or   0.24%

Total,                                                  4998

Total = 1060 for module usbhub.sys

Elapsed Time, >        0 usecs AND <=        1 usecs,      0, or   0.00%

Elapsed Time, >        1 usecs AND <=        2 usecs,      0, or   0.00%

Elapsed Time, >        2 usecs AND <=        4 usecs,    579, or  54.62%

Elapsed Time, >        4 usecs AND <=        8 usecs,    386, or  36.42%

Elapsed Time, >        8 usecs AND <=       16 usecs,     85, or   8.02%

Elapsed Time, >       16 usecs AND <=       32 usecs,      9, or   0.85%

Elapsed Time, >       32 usecs AND <=       64 usecs,      1, or   0.09%

Total,                                                  1060

Total = 6 for module watchdog.sys

Elapsed Time, >        0 usecs AND <=        1 usecs,      3, or  50.00%

Elapsed Time, >        1 usecs AND <=        2 usecs,      3, or  50.00%

Total,                                                     6

All Module =  43599,  Total = 43599,   EQUAL

--------------------------

Usage From 0 ms to 24682 ms, Summing In 1 second intervals. Intervals=25

                             CPU 0 Usage       CPU 1 Usage

Start (ms) End (ms)        (usec)      %     (usec)      %

         0-1000      :     33992,   3.40       897,   0.09

      1000-2000      :     24437,   2.44      1332,   0.13

      2000-3000      :     42317,   4.23      4510,   0.45

      3000-4000      :     40179,   4.02      1757,   0.18

      4000-5000      :     34190,   3.42       988,   0.10

      5000-6000      :     19434,   1.94      2066,   0.21

      6000-7000      :     25339,   2.53      2618,   0.26

      7000-8000      :     46758,   4.68      1691,   0.17

      8000-9000      :     81696,   8.17      1998,   0.20

      9000-10000     :     77932,   7.79      1500,   0.15

     10000-11000     :     64336,   6.43      1684,   0.17

     11000-12000     :     45099,   4.51      2311,   0.23

     12000-13000     :     59469,   5.95      1890,   0.19

     13000-14000     :     44035,   4.40      2393,   0.24

     14000-15000     :     50393,   5.04      2455,   0.25

     15000-16000     :     71508,   7.15      1694,   0.17

     16000-17000     :     56853,   5.69      2054,   0.21

     17000-18000     :     63219,   6.32      4046,   0.40

     18000-19000     :     62428,   6.24      4742,   0.47

     19000-20000     :     33793,   3.38      1134,   0.11

     20000-21000     :     28147,   2.81       759,   0.08

     21000-22000     :     30630,   3.06       640,   0.06

     22000-23000     :     33294,   3.33       710,   0.07

     23000-24000     :     26121,   2.61      1819,   0.18

     24000-24682     :     21608,   3.17      2177,   0.32

--------------------------

Interrupt Info

--------------------------

CPU Usage Summing By Module For the Whole Trace

CPU Usage from 0 us to 24682000 us:

     CPU 0 Usage      CPU 1 Usage

     usec      %      usec      % Module

    10420   0.04         0   0.00 USBPORT.SYS

    46437   0.19         0   0.00 VIDEOPRT.SYS

      125   0.00         0   0.00 acpi.sys

     7657   0.03         0   0.00 ataport.SYS

     8722   0.04         0   0.00 i8042prt.sys

    72940   0.30         0   0.00 ndis.sys

    13633   0.06         0   0.00 portcls.sys

Total = 48319

Elapsed Time, >        0 usecs AND <=        1 usecs,      0, or   0.00%

Elapsed Time, >        1 usecs AND <=        2 usecs,  12026, or  24.89%

Elapsed Time, >        2 usecs AND <=        4 usecs,  24954, or  51.64%

Elapsed Time, >        4 usecs AND <=        8 usecs,   9693, or  20.06%

Elapsed Time, >        8 usecs AND <=       16 usecs,   1611, or   3.33%

Elapsed Time, >       16 usecs AND <=       32 usecs,     29, or   0.06%

Elapsed Time, >       32 usecs AND <=       64 usecs,      6, or   0.01%

Total,                                                 48319

Total = 2256 for module USBPORT.SYS

Elapsed Time, >        0 usecs AND <=        1 usecs,      0, or   0.00%

Elapsed Time, >        1 usecs AND <=        2 usecs,      0, or   0.00%

Elapsed Time, >        2 usecs AND <=        4 usecs,    602, or  26.68%

Elapsed Time, >        4 usecs AND <=        8 usecs,   1556, or  68.97%

Elapsed Time, >        8 usecs AND <=       16 usecs,     97, or   4.30%

Elapsed Time, >       16 usecs AND <=       32 usecs,      0, or   0.00%

Elapsed Time, >       32 usecs AND <=       64 usecs,      1, or   0.04%

Total,                                                  2256

Total = 20976 for module VIDEOPRT.SYS

Elapsed Time, >        0 usecs AND <=        1 usecs,      0, or   0.00%

Elapsed Time, >        1 usecs AND <=        2 usecs,  12026, or  57.33%

Elapsed Time, >        2 usecs AND <=        4 usecs,   7821, or  37.29%

Elapsed Time, >        4 usecs AND <=        8 usecs,   1081, or   5.15%

Elapsed Time, >        8 usecs AND <=       16 usecs,     43, or   0.20%

Elapsed Time, >       16 usecs AND <=       32 usecs,      1, or   0.00%

Elapsed Time, >       32 usecs AND <=       64 usecs,      4, or   0.02%

Total,                                                 20976

Total = 11 for module acpi.sys

Elapsed Time, >        0 usecs AND <=        1 usecs,      0, or   0.00%

Elapsed Time, >        1 usecs AND <=        2 usecs,      0, or   0.00%

Elapsed Time, >        2 usecs AND <=        4 usecs,      0, or   0.00%

Elapsed Time, >        4 usecs AND <=        8 usecs,      0, or   0.00%

Elapsed Time, >        8 usecs AND <=       16 usecs,     11, or 100.00%

Total,                                                    11

Total = 811 for module ataport.SYS

Elapsed Time, >        0 usecs AND <=        1 usecs,      0, or   0.00%

Elapsed Time, >        1 usecs AND <=        2 usecs,      0, or   0.00%

Elapsed Time, >        2 usecs AND <=        4 usecs,      0, or   0.00%

Elapsed Time, >        4 usecs AND <=        8 usecs,    225, or  27.74%

Elapsed Time, >        8 usecs AND <=       16 usecs,    574, or  70.78%

Elapsed Time, >       16 usecs AND <=       32 usecs,     12, or   1.48%

Total,                                                   811

Total = 1178 for module i8042prt.sys

Elapsed Time, >        0 usecs AND <=        1 usecs,      0, or   0.00%

Elapsed Time, >        1 usecs AND <=        2 usecs,      0, or   0.00%

Elapsed Time, >        2 usecs AND <=        4 usecs,      0, or   0.00%

Elapsed Time, >        4 usecs AND <=        8 usecs,    905, or  76.83%

Elapsed Time, >        8 usecs AND <=       16 usecs,    260, or  22.07%

Elapsed Time, >       16 usecs AND <=       32 usecs,     12, or   1.02%

Elapsed Time, >       32 usecs AND <=       64 usecs,      1, or   0.08%

Total,                                                  1178

Total = 20976 for module ndis.sys

Elapsed Time, >        0 usecs AND <=        1 usecs,      0, or   0.00%

Elapsed Time, >        1 usecs AND <=        2 usecs,      0, or   0.00%

Elapsed Time, >        2 usecs AND <=        4 usecs,  16181, or  77.14%

Elapsed Time, >        4 usecs AND <=        8 usecs,   4713, or  22.47%

Elapsed Time, >        8 usecs AND <=       16 usecs,     81, or   0.39%

Elapsed Time, >       16 usecs AND <=       32 usecs,      1, or   0.00%

Total,                                                 20976

Total = 2111 for module portcls.sys

Elapsed Time, >        0 usecs AND <=        1 usecs,      0, or   0.00%

Elapsed Time, >        1 usecs AND <=        2 usecs,      0, or   0.00%

Elapsed Time, >        2 usecs AND <=        4 usecs,    350, or  16.58%

Elapsed Time, >        4 usecs AND <=        8 usecs,   1213, or  57.46%

Elapsed Time, >        8 usecs AND <=       16 usecs,    545, or  25.82%

Elapsed Time, >       16 usecs AND <=       32 usecs,      3, or   0.14%

Total,                                                  2111

All Module =  48319,  Total = 48319,   EQUAL

--------------------------

Usage From 0 ms to 24682 ms, Summing In 1 second intervals. Intervals=25

                             CPU 0 Usage       CPU 1 Usage

Start (ms) End (ms)        (usec)      %     (usec)      %

         0-1000      :      5820,   0.58         0,   0.00

      1000-2000      :      3686,   0.37         0,   0.00

      2000-3000      :      6472,   0.65         0,   0.00

      3000-4000      :      5410,   0.54         0,   0.00

      4000-5000      :      4638,   0.46         0,   0.00

      5000-6000      :      2420,   0.24         0,   0.00

      6000-7000      :      2793,   0.28         0,   0.00

      7000-8000      :      5988,   0.60         0,   0.00

      8000-9000      :     12071,   1.21         0,   0.00

      9000-10000     :     12461,   1.25         0,   0.00

     10000-11000     :      9816,   0.98         0,   0.00

     11000-12000     :      8206,   0.82         0,   0.00

     12000-13000     :      8926,   0.89         0,   0.00

     13000-14000     :      6189,   0.62         0,   0.00

     14000-15000     :      7649,   0.76         0,   0.00

     15000-16000     :      9613,   0.96         0,   0.00

     16000-17000     :      8431,   0.84         0,   0.00

     17000-18000     :      7618,   0.76         0,   0.00

     18000-19000     :      7212,   0.72         0,   0.00

     19000-20000     :      4813,   0.48         0,   0.00

     20000-21000     :      3925,   0.39         0,   0.00

     21000-22000     :      4557,   0.46         0,   0.00

     22000-23000     :      5169,   0.52         0,   0.00

     23000-24000     :      3489,   0.35         0,   0.00

     24000-24682     :      2552,   0.37         0,   0.00

Distribution of number of 2000 ms intervals w.r.t. DPC/ISR usage:

                                      CPU 0                      CPU 1

 DPC/ISR Usage %      DPC      ISR Combined      DPC      ISR Combined

>=  0 AND <=   1        0       12        0       13       13       13

>   1 AND <=   5        8        1        7        0        0        0

>   5 AND <=  10        5        0        6        0        0        0

>  10 AND <=  20        0        0        0        0        0        0

>  20 AND <=  40        0        0        0        0        0        0

>  40 AND <=  60        0        0        0        0        0        0

>  60 AND <=  80        0        0        0        0        0        0

>  80 AND <= 100        0        0        0        0        0        0

---

Total:                 13       13       13       13       13       13

Which is a pretty nice summary of all the DPCs and ISRs that ran during the data collection period and how long they ran for.

I like XPerf a lot and it’s something we plan to use in WDF to collect data around specific performance scenarios. In particular its ability to tie system performance data together with custom ETW events logged by components (like, say, UMDF) gives a very nice way to see how the parts of an end-to-end performance scenario break down temporarily and also to see how the scenario either causes system activity or is impacted by system activity.

I just wish I could get the symbols working .

WDK available in other languages

PeterWie — Mon, 24 Sep 2007 21:22:13 GMT

This is pretty cool, though my high-school french isn't sufficient to appreciate it completely. The WDK content on MSDN has been translated into multiple languages.

It's a machine translation and to make up for the inconsistencies they do side-by-side english & other language. It's not as good as native documentation, but hopefully it will help bridge the gap for those who don't speak english as their first language.

The language options are:

· Brazilian Portuguese (pt-br)

Enjoy

-p

My UMDF driver has some data - how do I get it to my app?

PeterWie — Mon, 05 Feb 2007 21:57:00 GMT

This came up as a question on NTDEV today. After I thought about it for a bit I realized it was actually a pretty good question. You've started writing your driver, or at least thinking about your driver. You know you'll get some data back from the device and you need to get it to the app. How do you do this?

For simplicity's sake, let's say there are two models for a driver to operate in. The "push" model and the "pull" model (of course there are more, but this captures the big ones.) When I say "push" and "pull" here I'm talking about both how the driver decides it should talk to the device, and how it returns the data to the driver.

But before I dive in, realize that I'm talking here about how the driver operates in "steady state" - the time between PNP and Power Management operations, between recovering from device errors/resets, the time when it's just servicing I/O requests.

Pull Model

We'll start with the Pull model because it's easier to understand. In steady state the typical pull-model driver does nothing on its own. When the app wants to talk to the device it sends a request to the driver, which triggers a driver callback. In the simplest case the driver formats a command to send to the device and uses the buffers from the application to hold any data the command returns. When the command returns the app's buffers contain the necessary data. The driver sets the information property on the request to indicate how many bytes of data the app is getting back then completes the request. When the app sees the request completion the data your driver provided will be in its buffers.

It's rarely this simple. Often you'll need to do apply some transformation to the data before you can return it. In that case your driver would allocate a buffer to use in the command you send to the device. When the command completes you use its result to write the correct data into the buffers on the app's requests. Then set information, complete the app's request, and free the intermediate buffer. Better yet, you can make the WDF memory object for the intermediate buffer a child of the request so that it's freed automatically when you complete the request.

Push Model

Say your device is going to bubble data up to your driver when something happens. It might be GPS coordinates, printer status, a finger print scan, etc… How do you get this data back to an application? Clearly you don't want the application to poll for data. All you want to do is push the data you're getting from the device up to the application. How do you do this?

The first thing to understand is that you can only give the application data that it asks for*. Unless the app makes a request and gives you a buffer you don't have a way to put data in its address space. So clearly the Push model is just a variant of the pull model.

In the pull model you let the app send you one or more requests asking for data. You queue those requests for later processing. In the background you have some code which is reading data from the device. When those read commands complete you pickup an app request off the queue, copy your command data into the app's request buffers then complete the app request.

You may choose to copy the data into multiple requests, you may choose not to complete the request if you don't have enough data to fill its buffer with just one command. And if you don't have any pending requests you need to decide whether to just throw the data away or whether to hold it in an internal buffer until the app comes down to ask for more.

The key difference between the push and pull models is whether the driver is sending commands to the device on its own, or whether every device command comes because the app sent your driver a request.

As I said before - it's never this simple. Looking at the UMDF Fx2 sample you see that it uses the pull model for read/write operations on the device and for most of the i/o controls that it supports. However it uses the push model to handle requests to read the state of the DIP switches.

-p

* You can use PNP device events if you need to push the fact that something has happened to any app which might be listening. But it's an unreliable mechanism and isn't suitable as a way to stream data.

Do UMDF drivers require signing?

PeterWie — Tue, 23 Jan 2007 22:32:03 GMT

To quote from Rev. Lovejoy:

Shoo, short answer: "Yes" with an "If", long answer "No" - with a "But.

There are two types of driver signing, which complicates the answer. As usual any answers I provide are probably vague and can be overridden by standard documentation. In this case the documentation seems to be available on MSDN.

Driver Package Signing:

First there's the standard signing of the 'driver package' (all the files needed to install the driver) which has been around since Win2k. Signing your driver package allows you to avoid the "unsigned driver" popup, and allows end-users to determine whether your package has been tampered with. Personally I think it's goodness, but having never had to go through the signing process I realize it's easy for me to feel that way.

To sign your driver package your INF should have a CatalogFile directive in the [Version] section. The catalog file contains hashes for all the files in your driver package and is then signed to ensure it can't be tampered with.

Code Integrity Signing:

In Vista there's a new code-signing requirement. Kernel-mode binaries that load in 64-bit Windows Vista (or drivers on 32-bit & 64-bit windows that stream protected content) need to be code signed so that the OS can determine whether they have been tampered with.

From a quick scan of the documentation it sounds like this mostly affects the type of certificate you can use when you do your signing. The Code Integrity system (which checks your driver signatures when the driver is loaded) requires something called a "Software Publisher Certificate (SPC)", while regular driver signing can use either an SPC or a "commercial release certificate". However I believe that if you are installing a kernel-mode boot driver you have to make sure your binaries are self signed (signature embedded in the binary) while if you are installing non-boot kernel-mode drivers that you can use a catalog signature (signature embedded in the .cat file installed with the package) instead.

Regardless your driver package will still require a signature in order to get through installation.

So what does all this mean for UMDF?

UMDF drivers do not require Code Integrity Signing since they aren't kernel-mode code. However your driver package still needs to be signed so that it can't be tampered with & can be traced back to the author (or you continue to have unsigned driver popups). And if your package includes any non-Microsoft kernel-mode drivers in addition to the user-mode driver then the Code Integrity Signing rules apply to that too.

To finish with the quote from Rev. Lovejoy:

Uh, if you need additional solace, by the way, I've got a copy of something or other by Art Linkletter in my office.

-p

Fixing the UMDF USB samples to install both on XP & Vista

PeterWie — Wed, 10 Jan 2007 22:10:00 GMT

UMDF's content in the WDK contains a handful of examples for controlling a USB device. Unfortunately they only show how to do it on Vista. The problem was that WinUSB's co-installer wasn't added to the WDK until very late - too late for us to be able to change our samples to show how to use it.

WINUSB introduces a number of new things into the INF because of the complexity of installing it on XP and making it work with the in-box Vista support. In Vista WinUSB is present on the disk but the driver & service aren't "installed" until some INF copies them into place. The WinUSB folks provided an in-box INF (winusb.inf) that you can reference in your INF with the Includes/Needs directives so that you don't have to setup the service entries yourself.

On XP WinUSB doesn't exist until someone installs it. The preferred installation route is to use the WinUsbCoinstaller. However because WINUSB.INF isn't already on the machine in XP, you can't depend on it to setup the WinUSB service entries. And we couldn't rely on the WinUsbCoinstaller to do it because the device installer requires that your INF have an AddService directive for the driver which will act as the FDO.

The result is that installing WinUSB requires you to add a few things to your INF. Some of these are already demonstrated in the existing sample INFs. Others are new, and there's one "bug fix". I've added the full INF as an attachment to this blog entry.

So starting with the Fx2 driver's INF, here are the necessary changes to build a driver package that will install both on XP and Vista.

Include sections from WINUSB.INF in your INF

Specifically in your [DDInstall.NT] section you add:
```
Include=WINUSB.INF ; Import sections from WINUSB.INF
Needs=WINUSB.NT ; Run the CopyFiles & AddReg directives
```
The first tells Device Installation to locate WINUSB.INF and read in all of its sections. The second tells DI that it should import any directives in the "WINUSB.NT" section into this section as well. WINUSB.NT contains CopyFiles & AddReg directives that tell Vista to copy the WinUSB.Sys binary as part of your driver package & sets up some registry entries.

Now you may recall I said that there was a problem on XP because this INF doesn't exist. However when you ask DI to include an INF file that doesn't exist (or say you Need a section that doesn't exist) the installer just ignores you. Normally that would drive me nuts (I hate hidden errors) but it works in my favor in this case.

On XP these two directives are benign and we depend on the WinUsbCoinstaller to copy the files into place and setup the registry entries. On Vista WinUsbCoinstaller doesn't do anything and we depend on these directives to copy the binaries.

This leads us to step 2 ...
Copy & Invoke the WinUsbCoinstaller in your INF

To invoke the WinUSB coinstaller requires three things in your INF:
1. Add the file to [SourceDisksFiles] so that DI knows where to get the file from
```
[SourceDisksFiles]
...
WinUsbCoinstaller.dll=1
```
2. Add the file to [CoInstallers_CopyFiles] so that DI knows that it needs to copy it
```
CoInstallers_CopyFiles]
...
WinUsbCoinstaller.dll
```
3. Add the name of the coinstaller to the CoInstallers32 registry entry so that DI knows to invoke it.
```
[CoInstallers_AddReg]
HKR,,CoInstallers32,0x00010000,"WudfUpdate_$UMDFCOINSTALLERVERSION$.dll",\
                               "WinUsbCoinstaller.dll", \
                               "WdfCoInstaller01005.dll,WdfCoInstaller"
```
Note that the names of the last two sections need to match what you provide to CopyFiles and AddReg in your INF's [DDInstall.Coinstallers] section.
Fix the bug in the [WinUsb_Install] section

WinUsb requires version 1.5 of KMDF, not version 1.0. We didn't catch this before the samples shipped. Change the section to look like:
```
[WinUsb_Install]
KmdfLibraryVersion=1.5
```
Include WinUsbCoinstaller.dll as part of your signed driver package

Along with WudfUpdate_01005.dll & WdfCoinstaller01005.dll you also now need to copy WinUsbCoinstaller.dll & add it to the catalog for your INF. You can find this file in c:\winddk\6000\redist\winusb\<arch>.

Like the KMDF and UMDF coinstallers, it comes in a _chk version also. If you're installing your driver (whether you built it checked or free) on a checked system you should use the _chk version of the DLL.

Four little changes and you're done. The same changes apply to any of the UMDF USB samples.

-p

Glossary Definition of "WDM"

PeterWie — Mon, 23 Oct 2006 23:52:00 GMT

I got looped into a discussion about the WDK glossary entry for "Windows Driver Model". It got me thinking about how difficult it is to explain things in the driver space to non-driver folks.

When I was working in storage I used to have this problem all the time. Someone, usually a family member, would ask me what I did and I'd have to talk about what a "storage device" was, about what miniports were, etc… For a while I gave up and described my job as "something technical and utterly uninteresting". Finally I settled on "I make the light on your hard-disk blink".

Then I moved to the WDF team and everything became complicated again. Explaining what WDF is to a non-driver person is a bit like explaining what colors are to someone who is color blind.

So here's my attempt to explain WDM to someone sort of technical who has no idea what a device driver does:

Windows Driver Model

An architecture for Windows extensions (device drivers) which allow the operating system core to communicate with a wide variety of devices. WDM defines a Device Driver Interface (DDI) for drivers to interact with each other and the OS, and a set of required behaviors that device drivers must implement in order to work well with the rest of the system. WDM and WDM drivers work together to provide device I/O, Plug and Play, Power Management and WMI support.

WDM is a packet-based I/O architecture. Applications (as well as the OS and other drivers) can make requests of particular devices. WDM represents these requests as I/O Request Packets (IRPs) and sends them to a particular set of device drivers for processing. The drivers typically work in a layered fashion – each performing some portion of the request then handing the request off to the next driver – until the bottom most driver programs the device to perform the requested operation(s).

Not all devices are supported by WDM drivers. Some device classes (such as storage devices or network adapters) provide their own class-specific DDIs and behaviors which represent a subset of WDM. Other drivers may be implemented using the Windows Driver Frameworks, which provides a simpler general-purpose infrastructure for writing drivers.

Does Windows have a "Scheduler"

PeterWie — Fri, 20 Oct 2006 23:28:00 GMT

The question came up on the NTDEV mailing list a while back about how the "Scheduler" works in Windows … specifically about what thread it ran on and how it got control.

The answer is that you should ignore this idea that there's a "scheduler" as an independent entity. It doesn't exist like that in Windows.

In windows there are basically two functions for scheduling in the dispatcher. The first selects the most appropriate thread to run on a processor at any given time based on a number of factors such as thread priority, thread affinity, ideal processor, etc… . The second initiates a context switch between the current thread and the new one.

The Ke subsystem implements these functions and uses them to schedule threads in a distributed manner. Basically any time a Ke function is called which changes the state of a thread (to block it, to wake it up, to change its priority, to mark it as having exceeded a quantum) that function will invoke these two other functions to see if a different thread should be running and, if so, to swap over to it.

So you've got some user-mode code on a processor. That code will continue to run until it:

Calls WaitForSingleObject, which calls the NtWaitForSingleObject system call which transitions to kernel mode and (eventually) calls KeWaitForSingleObject. KeWaitForSingleObject links the current thread onto the wait list for the object (say an event) then selects another thread which could be run and does a context switch

Gets a page fault. This triggers the kernel trap handler, which calls the memory manager which will initiate a page-in then calls KeWaitForSingleObject to block the thread until the I/O completes. KeWaitForSingleObject will select a new thread to run and switch to it.
Is interrupted by a timer interrupt. The kernel's timer interrupt handler will see if the current thread has run longer than its quota and, if so, will attempt to select a new thread to run. If the current thread is the highest priority thread in the system then it will be left on the processor. Otherwise the kernel code will get select a new one and switch to it.

Is interrupted by a device interrupt which completes an I/O request initiated by another process. Often this involves boosting the priority of the other process by calling a Ke function. This function sets the new priority, then calls the selection function to see what process should be running on the processor. If there's a better candidate after the priority boost then Ke will swap to the new thread

There are a number of other conditions which can cause a context switch, and I've dramatically oversimplified some of them, but hopefully this will help you get the idea. The responsibility for scheduling in NT is distributed across all threads in the system, which eventually end up in a Ke routine.

Meet my boss

PeterWie — Mon, 02 Oct 2006 22:20:00 GMT

Nar's talk on the NT I/O manager is now up on channel 9.

Ilias Tsigkogiannis has started a Blog about learning to write Windows Drivers

PeterWie — Thu, 28 Sep 2006 20:55:00 GMT

Ilias is the latest addition to the UMDF development team. I'm really excited that he's decided to start blogging about his initial experience learning about how WDM and UMDF drivers work. Hopefully this can serve as a helpful introduction for anyone who is just starting off with driver development.

COM in UMDF

PeterWie — Fri, 15 Sep 2006 20:53:00 GMT

UMDF defines all of its interfaces using COM interfaces, rather than the flat C-style DDI that the KMDF team used. When kernel developers see this their first question is often "why did you do that?" and it's an understandable question. I had a long discussion at WinHEC about this exact topic.

First not everyone is sold on the flat C DDI that we put together for KMDF. In the polls we've done, the sample group (let's be optimistic and call it the "industry") is split about evenly on whether they want C or C++. However for folks that concentrate more on user-mode stuff there's a greater preference for C++. So we wanted a DDI that would work well with C++ but that wouldn't force you to use it.

Using straight C++ objects also wouldn't work. C++ has its nice points, but sharing objects across DLL boundaries isn't one of them. There's no way to publish only your "public" members, there are concerns about whether our compiler will do the same name mangling from release to release with exports (long story), there's the fragile base class problem, there are all sorts of issues with the new & delete operators, and it can't be used from C without a wrapper library.

We could have used C++ abstract classes – describing the interfaces that our objects exposed without any of the pesky internal details. But we'd then have to add reference counting on top of that, and some method of casting one interface type to another, and when you're all done it looks like every object supports IUnknown.

So the bottom line is that we chose COM because we had to expose something that acted an awful lot like COM. Rather than create our own semantic which everyone would have to learn, we decided to borrow from an existing one that's already understood by many developers.

We've worked very hard to keep the COM runtime goop out of UMDF. UMDF drivers don't register as COM DLLs, they don't get to do any of the weird COM synchronization stuff, and they don't have to worry about IDispatch or monikers or whether they're running in-proc/out-of-proc. We try to use the COM programming model but without the weird magic that the COM runtime introduces. We do allow you to use something like ATL though if you're tolerant of a little magic and want to make the COM boilerplate easier. And I have to admit that as much as I dislike magical templated macros, being able to use CComPtr to do my reference counting for me is a helpful thing.

-p

Building the UMDF 1.0 (beta) samples

PeterWie — Thu, 29 Jun 2006 05:45:00 GMT

I'm sorry it's taken me so long to write about this. The last week or two have been pretty hectic as we try to lockdown for Vista. But we still want people to download the UMDF 1.0 beta, and here are the instructions for that.

The first step, of course, is to install the WDK. You can either use the Beta 2 WDK or the 5456 version which was recently published on the beta site. If you don't have either of these installed yet, you can my earlier blog entry on installing the WDK.

Next you need to download the UMDF beta bits and install those on your development machine as well. Go to the connect.microsoft.com site for the beta & look under downloads. You will see several items but you want to download and run either "UMDFx86" or "UMDF64" depending on the architecture of your development machine (not your test machine). Download and install the MSI and you'll be ready to go.

To actually build one of the samples you'll want to start a "Windows Vista and Windows Server Longhorn" build environment from the WDK. This is burried under "Windows Driver Kits" on your start menu. Be sure you either use the 5483 or 5456 WDK ... if you have Beta 1 or the February CTP kit still installed those may not work.

Once you've started your build window, you need to run a script to tell it to use the headers & all from the MSI you just installed rather than the ones in the kit. Do the following:

    cd \WinDDK\WDF\UMDF10_xp
    set_umdf_env.cmd
    cd src\umdf
    build -cZ

And you've just built the UMDF 1.0 samples.

Next time i'll walk through installing the echo sample driver.

UMDF 1.0 (for XP only) beta is FINALLY avaialble

PeterWie — Thu, 15 Jun 2006 21:16:00 GMT

YIPPY!!!

Sometimes it seems like releasing software is more complicated than developing it. We really wanted to have this at WinHEC, but that just wasn't going to happen. Fortunately I have an amazing team to help push through all the problems and process imposed upon us. So we have a beta release.

If you're already signed up for the Windows Driver Kit beta then you should have email telling you that the UMDF beta is available. The mail helpfully fails to tell you where to get the beta - it's on the download page for the WDK beta.

If you aren't signed up but you've read this far you clearly want to. Go ahead and do it. Unless you're chicken* that is (or don't have the bandwidth to download a 2+ GB kit).

I really want to write some more about using it, but my team is trying to lockdown for the Vista RC and I don't have time today. In the next few days i'll walk through the steps of installing it, compiling the samples and getting the resulting driver installed.

Until then, the readme file is your friend.

Enjoy.

-p

* they say goading can be an effective motivator. Let's try it now.

Windows Driver Kit Beta 2 is finally available

PeterWie — Tue, 30 May 2006 22:41:00 GMT

I got back from vacation today and went to connect.microsoft.com to see if the WDK beta was up yet, and i'm happy to say that it is. It's a fairly large (2.3 GB) ISO image to download and i'm pulling it down now to make sure it works okay.

Here are the steps i had to follow to get into the beta:

Navigate to connect.microsoft.com and sign in using your passport account (the passport link is at the top right of the web page).
Go to the Available Programs page (there's a menu link on the left side of the window) and scroll down to find the Microsoft Windows Driver Kit (WDK) Beta Program entry then click on the Apply/ link.
Decide if you agree with the terms and conditions. If you do then click on the I Agree button at the bottom of the screen.
This should bring you to the Mandatory Surveys page. The Beta 2 Feedback survey is very, very short so you should just take it. If you want to skip it there's a link for that too, but I'm guessing that you'll be forced to take it by 6/15 if you want to keep access to the beta site (based on the web page content).
Either way you'll end up at the web page for the WDK beta. There's nothing really on the page, but you can get the kit from the Downloads link in the menu on the left-hand side of the screen.
On the downloads page, choose the last item (WDK Beta 2). Downloading this requires that you install an ActiveX control, and the download will take a while. Once you're done you should be able to burn it to a DVD using your favorite tool, or use the Virtual CD-ROM tool to mount the image. from Microsoft.
For purposes of this discussion, we'll assume you mount the ISO image (either with a DVD or VirtualCD) on z:.
Open Z: and run Installer.exe. This will let you install the prerequisites and then the build environment and doc set. Install whatever other parts you see fit.

You should now have a kit installed which will let you build UMDF drivers for use on Windows Vista (beta of XP support is on the way.) I'll talk about getting started with that later.

Enjoy.

-p

GetScatterGatherList will coallesce contiguous SG list entries

PeterWie — Fri, 26 May 2006 03:24:00 GMT

GregH asked me an interesting question at WinHEC today - one for which I didn't immediately know the answer. He was wondering whether the new DMA DDIs (Get|PutScatterGatherList) would return a single SG entry for each page, or whether they would coallesce physically contiguous pages into a single entry.

The answer is that the coallescing is "kind of complex and probably won't happen in a way that will make complete sense to someone just looking at the generated [SG] lists". To anyone who is familiar with the DMA DDIs that sort of answer probably isn't surprising :)

On X86 & for non-scatter-gather devices, the bounce buffers are allocated in big contiguous blocks and the DMA DDI will coallesce entries where possible. Of course non-SG devices always get one big contiguous block no matter what, and so they'll only have a single entry.

On AMD64 the bounce buffers for an SG adapter aren't necessarily contiguous so there may be less coallescing.

Still, since you always have to plan for the worst case, this ends up being an optimization but shouldn't change your driver's behavior (unless your device uses single-page SG entries rather than address/length pairs).