WDK Kernel Programming: Object Reference Tracing with Tags
Kernel objects are primitive data objects that the Windows kernel implements in system memory to represent the various features of the computing environment that are managed by the operating system. Kernel objects represent features such as devices, drivers, files, registry keys, events, semaphores, processes, and threads.
Most kernel objects are not permanent. To prevent a nonpermanent kernel object from being deleted while a kernel-mode driver uses the object, the driver can acquire a counted reference to the object. When the driver no longer needs the object, the driver releases its reference to the object.
If a driver fails to release all of its references to an object, the object's reference count never decrements to zero and the object is never deleted. Thus, any system resources that are used by the object (for example, system memory) are leaked. That is, they are unavailable for use until the next time that the operating system starts up.
Another type of reference error occurs if a driver under-references an object. In this case, the driver releases more references to an object than the driver actually holds. This error can cause the object to be deleted prematurely, while other clients are still trying to access the object.
Leaks and under-references of kernel objects can be difficult bugs to track down. For example, a process object or a device object might have tens of thousands of references. To identify the source of a reference bug in these circumstances can be a tedious and frustrating task.
In Windows 7, object references can be tagged to make these bugs easier to find. The Windows kernel developers have added a new set of routines to associate tags with references to kernel objects. These new routines are ObReferenceObjectWithTag, ObReferenceObjectByHandleWithTag, ObReferenceObjectByPointerWithTag, ObDereferenceObjectWithTag, and ObDereferenceObjectDeferDeleteWithTag.
For example, ObReferenceObjectWithTag and ObDereferenceObjectWithTag are enhanced versions of the familiar ObReferenceObject and ObDereferenceObject routines. These enhanced routines enable you to supply a four-byte, custom tag value as an input parameter. The tag value for each call gets added to an object reference trace that can be accessed by the Windows debugging tools. Although ObReferenceObject and ObDereferenceObject do not enable the caller to specify custom tags, the Windows 7 versions of these routines add default tags (with tag value "Dflt") to the trace. Thus, a call to ObReferenceObject or ObDereferenceObject has the same effect as a call to ObReferenceObjectWithTag or ObDereferenceObjectWithTag that specifies a tag value of "Dflt". (In your program, this tag value is represented as 0x33796b4c or 'tlfD'.)
To track down a potential object leak or under-reference, identify a set of associated ObReferenceObjectXxxWithTag and ObDereferenceObjectXxxWithTag calls in your driver that increment and decrement the reference count of a particular object. Choose a common tag value (for example, "Lky8") to use for all of the calls in this set. After your driver is done using the object, the number of decrements should exactly match the number of increments. If these numbers do not match, your driver has an object reference bug. The debugger can compare the number of increments and decrements for each tag value and tell you if they fail to match. With this capability, you can rapidly pinpoint the source of the reference-count mismatch.
To view an object reference trace in the Windows debugging tools, use the !obtrace kernel-mode debugger extension. In Windows 7, the !obtrace extension is enhanced to display object reference tags, if object reference tracing is enabled. By default, object reference tracing is turned off. Use the Global Flags Editor (Gflags) to enable object reference tracing. For more information about Gflags, see Configuring Object Reference Tracing.
After object reference tracing is turned on, the output that is produced by the !obtrace extension includes a "Tag" column, as the following example shows:
0: kd> !obtrace 0x8a226130
Object: 8a226130
Image: leakyapp.exe
Sequence (+/-) Tag Stack
-------- ----- ---- ---------------------------------------------
36 +1 Dflt nt!ObCreateObject+1c4
nt!NtCreateEvent+93
nt!KiFastCallEntry+12a
37 +1 Dflt nt!ObpCreateHandle+1c1
nt!ObInsertObjectEx+d8
nt!ObInsertObject+1e
nt!NtCreateEvent+ba
nt!KiFastCallEntry+12a
38 -1 Dflt nt!ObfDereferenceObjectWithTag+22
nt!ObInsertObject+1e
nt!NtCreateEvent+ba
nt!KiFastCallEntry+12a
39 +1 Lky8 nt!ObReferenceObjectByHandleWithTag+254
leakydrv!LeakyCtlDeviceControl+6c
nt!IofCallDriver+63
nt!IopSynchronousServiceTail+1f8
nt!IopXxxControlFile+6aa
nt!NtDeviceIoControlFile+2a
nt!KiFastCallEntry+12a
3a -1 Dflt nt!ObfDereferenceObjectWithTag+22
nt!ObpCloseHandle+7f
nt!NtClose+4e
nt!KiFastCallEntry+12a
-------- ----- ---------------------------------------------------
References: 3, Dereferences 2
Tag: Lky8 References: 1 Dereferences: 0 Over reference by: 1
The bottom line in the preceding example indicates that the reference and dereference counts that are associated with the "Lky8" tag do not match and that the result of this mismatch is an over-reference by one (that is, a leak). If the result were instead an under-reference, the bottom line of the !obtrace output might look like this:
Tag: Lky8 References: 1 Dereferences: 2 Under reference by: 1
By default, the operating system conserves memory by deleting the object reference trace for an object when the object is freed. To track down an under-reference requires that the trace remain stored in memory even after the object has been freed. For this purpose, the Gflags tool provides a "Permanent" option, which preserves the trace in memory while the computer shuts down and starts up again.
Object reference tracing—without tags—was introduced in Windows XP. Because the trace did not include tags, developers had to resort to less convenient techniques to identify object reference bugs. The debugger could track the references of groups of objects, which the developer selected by object type. The only way that the developer could identify the various sources of object references and dereferences was to compare their call stacks. Although the preceding !obtrace example contains only five stacks, certain types of object, such as a process (EPROCESS) object, might be referenced and dereferenced many thousands of times. With thousands of stacks to inspect, the task of identifying the source of an object leak or under-reference without the help of tags might be very time consuming.
Special thanks to Salahuddin J. Khan of the Windows kernel development team for providing the information for this blog entry.
– Jerry Van Aken [MSFT], WDK Programming
