Using ETW for SQL Server 2005

queryproc — Mon, 13 Nov 2006 05:32:00 GMT

ETW stands for “Event Tracing for Windows” and it is used by many Windows applications to provide debug trace functionality. This “wide” availability is a key point of using ETW because it can help to track certain activities from end to end. For example, you can literally track a request coming from IIS, passing through protocol layer, and then finally handled by database engine in single trace file. Unfortunately, not many people know that SQL Server 2005 provides a full ETW functionality which can output most of SQL Trace events available for SQL Server Profiler. In this article, I will explain how to use SQL ETW with examples.

First of all, how do I know if SQL ETW is really active on my machine? The following shows output on my server which has SQL Server 2005 Standard Edition as 2^nd instance (named Yukon).

C:\>logman query providers
Provider                                 GUID
-------------------------------------------------------------------------------
YUKON Trace                              {130A3BE1-85CC-4135-8EA7-5A724EE6CE2C}
.NET Common Language Runtime             {e13c0d23-ccbc-4e12-931b-d9cc2eee27e4}
ACPI Driver Trace Provider               {dab01d4d-2d48-477d-b1c3-daad0ce6f06b}
Active Directory: Kerberos               {bba3add2-c229-4cdb-ae2b-57eb6966b0c4}
IIS: Request Monitor                     {3b7b0b4b-4b01-44b4-a95e-3c755719aebf}
Local Security Authority (LSA)           {cc85922f-db41-11d2-9244-006008269001}

My other server has a default instance of SQL Server 2005 and it shows something like this:

MSSQLSERVER Trace {2373A92B-1C1C-4E71-B494-5CA97F96AA19}

Is there a connection between these 2 examples? Yes, the SQL ETW provider name appears to be a combination of “Instance name” and “Trace”, and its GUID is unique across instances. You can use either provider name or GUID to start and stop ETW session.

Please note this ETW functionality is not available for SQL Express, because, well…, you get what you paid for J. If you don’t see SQL ETW provider name, and you are sure you have a regular SQL Server 2005, then you may be able to fix it yourself. Here’s a short list of instructions.

Go to the directory where your SQL server binaries are. On my machine, it is on
“C:\Program Files\Microsoft SQL Server\MSSQL.1\MSSQL\Binn”.
Look for “etwcls.mof”. If you cannot find it, something is definitely wrong with your installation. You probably need to seek help elsewhere.
Run “mofcomp etwcls.mof” in command window.
Try “logman query providers” again. You should see your SQL ETW provider.

Now I will describe how to configure SQL ETW to collect interesting traces. Go to the same directory described in the above instruction and find a file called “etwcnf.xml”. This is your configuration file and you can modify it with any text editor. There should several entries already defined in this file for your examples. See the following example picked out of etwcls.xml file.

This defines a tracing template called “TSQL replay” with template ID = 1. It contains a list of events with ID = 11, 13, and so forth. This event ID is matched to SQL Trace Event Class. By consulting BOL documentation on available SQL Trace events, you can find “RPC:Starting” event has ID of 11, and “SQL:BatchStarting” event has ID of 13. Yes, now you know how to change this template to suite your need!

Now let’s try to get some actual trace events. I’m using “logman.exe” and “tracerpt.exe” which should be available on most Windows platform.

First, create a text file as shown in the following example. You can activate multiple providers by listing them in separate lines. For simplicity, let’s enable only SQL ETW.

C:\etw>type prov.txt
"YUKON Trace" 1 0

As you have figured out already, the first column is SQL ETW provider name (you can use GUID also). The next number (1) is called “enable flag” and should match the template ID in etwcls.xml file. The 3^rd number (0) is called “enable level” and should be kept as zero for SQL ETW. The meaning of these 2 flags depends on providers and you should consult appropriate documentations for other providers.

Now you are ready to start your 1^st SQL ETW session! In my example, I decided to give a very creative/original name: “mytrace”. Here’s a screen dump from my machine.

C:\etw>logman start mytrace -pf prov.txt -ets
Name:                      mytrace
Age Limit:                 15
Buffer Size:               64
Buffers Written:           1
Clock Type:                System
Events Lost:               0
Flush Timer:               0
Buffers Free:              2
Buffers Lost:              0
File Mode:                 Sequential
File Name:                 C:\etw\mytrace.etl
Logger Id:                 3
Logger Thread Id:          1308
Maximum Buffers:           25
Maximum File Size:         0
Minimum Buffers:           3
Number of buffers:         3
Real Time Buffers Lost:    0

Provider                                  Flags                     Level
-------------------------------------------------------------------------------
* "YUKON Trace"                           0x00000001                0x00
{130A3BE1-85CC-4135-8EA7-5A724EE6CE2C} 0x00000001                0x00

To test if this ETW session can actually receive something from SQL Server, let’s issue “select * from sys.dm_exec_requests”. After that, you can stop the tracing as shown in the following example.

C:\etw>logman stop mytrace -ets
The command completed successfully.

Look for newly created file in the working directory (mytrace.etl). Unfortunately, this is a binary trace file and you will need some help to crack it open. “Tracerpt.exe” does some rudimentary job of converting this binary trace file into human-readable form.

C:\etw>tracerpt mytrace.etl
Input
----------------
File(s):
     mytrace.etl
100.00%
Output
---------------
Text (CSV):         dumpfile.csv
Summary:            summary.txt
The command completed successfully.

Note this creates 2 files: dumpfile.csv and summary.txt. Here’s a screen dump.

C:\etw>type dumpfile.csv
Event Name,       Type,        TID,           Clock-Time, Kernel(ms),   User(ms), User Data
EventTrace,     Header, 0x00000BF4,   128075886826813362,          0,         10,    65536, 33620485,     790, 1, 128075887366383792,   100144,        0, 0x00000000,        2,        1,        4,        0,     2785, 0x01402930, 001402940,                0,          3579545, 128075886826813362, 0x00000002,        0, 0, 0
SQL:BatchStarting,          0, 0x00000874,   128075886939625782,          0,         10, "select * from sys.dm_exec_reqests",       1,                0, "xxxx", "REDMOND", "JAYC8V1",     1580, "SQL Query Analyzer", "REDMOND\xxxx",       51, 28075598939600000,       13, "master", 010500000000000515000000A065CF7E784B9B5FE77C8770A9640000",        0,   0,      38,        0, "REDMOND\xxxx", 0, 0
C:\etw>

Yes, this looks pretty cryptic and you will probably need to look at etwcls.mof (and SQL Trace description) to decipher each column. From etwcls.mof file, we know 1^st column is “TextData”, 2^nd column is “DatabaseID”, 3^rd column is “TransactionID”, etc.

       [WmiDataId(1),
       Description("TextData"),
       format("w"),
       StringTermination("Counted"),
       read
       ]
       string TextData;
       [WmiDataId(2),
       Description("DatabaseID"),
       read
       ]
       sint32 DatabaseID;
       [WmiDataId(3),
       Description("TransactionID"),
       read
       ]
       sint64 TransactionID;

Now we know this ETW session received “SQL:BatchStarting” event correctly, and its TextData column contains correct TSQL statement.

OK, you know how to start and stop SQL ETW. But with very little UI support compared to the pretty looking SQL Server Profiler, why even bother to consider ETW if I don’t need end-to-end tracing? Well, one good thing about SQL ETW is that your ETW session survives SQL Server restart, unlike ordinary SQL Trace.

Jay Choe

Tips, Tricks, and Advice from the SQL Server Query Processing Team : ETW

Using ETW for SQL Server 2005