Certain devices are exclusive access, or in simpler terms, only one handle can be opened for a particular device. A serial port is an example of an exclusive device; it would make no sense for 2 applications to have the port open because each would expect exclusive state with device plugged into the port and the settings (baud, parity, etc.) on the port itself.

You, as the driver writer, can also choose to make your device exclusive (if you are not writing a driver for a particular device class). Alternatively, you might choose to allow multiple handles to be created against your device. Finally, you might even want to have file system semantics for exclusive access or shared access (which is a whole topic in and of itself).

But what happens if you are trying to create a device which can have multiple handles created against it, but it ends up being exclusive for some unknown reason? How do yo debug this? The biggest aid in debugging this issue is understanding how you can create an exclusive device in the first place. Here is a list of ways of how to create an exclusive access device. (This list is by no means a exhaustive list, it just contains the methods I know of ;) ).

The DDK also has a great section on this topic, you can read it here. Hopefully the link does not move around.

  1. When calling IoCreateDevice(), you specify TRUE for the Exclusive parameter (the sixth one). This method works for legacy (e.g. non PnP) devices and for raw PDOs, but it does not work well for non raw PDOs, FDOs, or filters and here is why:
    • PDO: Typically you don't know what device stack is going to be loaded on the PDO. As such, creating an exclusive access device might be contrary to the needs of the stack that is loaded on the PDO.
    • FDO or Filter DO: These devices typically are unnamed devices. When you create a device interface, it uses the PDO's name as the destination for the symbolic link. When the I/O manager resolves the symbolic link, it uses the settings of the named device object itself (and not the top of stack) to manage exclusivity (and actual rights for access based on the security descritor for the device). You could create a named FDO or Filter device object so that the I/O manager does the check on the correct device object, but that doesn't stop someone from opening the stack using the PDO's name, so this solution does not work all the time.

    As a further point of complexity, the I/O manager only enforces exclusivity on the base name of the device object. If the caller appends a string to the end of the base name, the I/O manager will not count that create as exclusive.

  2. When you implement your dispatch routine for IRP_MJ_CREATE (in WDM) or EvtDeviceFileCreate (in KMDF), you maintain a count of handles which have been opened against your device and fail requests when the count is > 1. Typically this count is maintained on a field in the device extension using the InterlockedIncrement()/Decrement() functions and is decremented in your IRP_MJ_CLOSE dispatch routine (in WDM) or EvtFileClose (in KMDF). Note: in KMDF you don't need to do this at all. During device initialization, if you call WdfDeviceInitSetExclusive(), KMDF will maintain this count for you automatically.

  3. A (class) filter driver above your could be enforcing exclusivity. For example, in the keyboard or mouse device classes, the class driver (kbdclass.sys or mouclass.sys) will enforce exlusivity on the device (by maintaining a count of open handles). To debug this, you really need to consider which device class you are installed under. The device class is controlled by the INF which installs your device.

  4. The INF can enforce exclusivity. By using the INF, you solve the problem where the FDO wants exclusivity while the PDO (which has the openable name) was marked as exclusive when it was created. With the INF directive in place, the PnP manager will apply the exclusivity setting to each device in the device stack after the stack has successfully completed the IRP_MN_START_DEVICE IRP. You can specify exclusivity for the class or only for specific device instances. Under the [ClassInstall32] or [DDInstall.HW] sections, add the following
    HKR,,Exclusive,0x10001,exclusive-device
    
    where exclusive-device is 1 and then the PnP manager will apply the exclusive attribute to all devices in the stack. The same caveat I mentioned earlier about the I/O manager skipping the exclusivity check if there is an additional path at the end of the file name applies here as well.

So, now that you know how to create an exclusive device object, how do you debug it? Well, you know if you are specifying Exclusive in IoCreateDevice(), so there is no need to debug that at runtime. What you can debug at runtime is a filter on top of you failing the create request. First, you run !devstack on your devobj to find the filter driver attached to your device. Then, run !drvobj [driver name] 3 and you will get the IRP_MJ_CREATE dispatch handler. Put a bp on that handler and then see if the request is being completed without being sent down the stack. Obviously you are now debugging code with out source, but at least you have a place to start.