Store Statistics XML in database tables using SQL Traces for further analysis.

queryproc — Fri, 01 Jun 2007 19:51:00 GMT

Since SQL Server 2005, query plan as well as statistics of query execution can be captured in XML format. Also, SQL Server 2005 has XQuery support to directly query XML document. By combining these two new features, users can analyze the query plans using queries.

However, in SQL Server, there is no easy way to capture the statitics XML into a table. Fortunately, there are SQL traces provided by SQL Server to capture the showplan XML and statistics XML information into trace files and loaded into tables.

Note: The only limitation is the 128 level of nesting levels supported by XML data type in SQL 2005. In that case, you have to write client code to parse the query plan, which going to be a very complex query plan.

Here is a small example using SQL traces to store the statistics XML and extract the estimated rows and actual rows.

/*Using Traces to Capture Statistics XML*/

declare @trace_id int

declare @trace_file nvarchar(200)

select @trace_file = 'c:\temp\test_stats_' + cast(newid() as varchar(100))

-- using trace table.

exec sp_trace_create @trace_id output,

@tracefile=@trace_file

-- capture statistics-xml, textdata, on

exec sp_trace_setevent @trace_id, 146, 1, 1

-- start

exec sp_trace_setstatus @trace_id, 1

-- test statement.

select * from sys.objects

-- stop

exec sp_trace_setstatus @trace_id, 0

-- close

exec sp_trace_setstatus @trace_id, 2

-- load trace files into table

if object_id('temp_trc') is not null

drop table temp_trc

select *

into temp_trc

from fn_trace_gettable(@trace_file + '.trc', default)

-- look at the captured stats xml

declare @plan xml

select @plan=cast(textdata as xml)

from temp_trc

where eventclass = 146;

-- collect the actual and also estimate stats.

with XMLNAMESPACES ('http://schemas.microsoft.com/sqlserver/2004/07/showplan' as sql)

select

ro.relop.value('@NodeId', 'int') NodeId,

ro.relop.value('@PhysicalOp', 'nvarchar(200)') PhysicalOp,

ro.relop.value('@LogicalOp', 'nvarchar(200)') LogicalOp,

(ro.relop.value('@EstimateRows', 'float')

* (ro.relop.value('@EstimateRewinds', 'float')

+ ro.relop.value('@EstimateRebinds', 'float')

+ 1.0)) EstimateRows,

case

when root_actual.ActualRows = 0

then null

else root_actual.ActualRows

end ActualRows,

cast(ro.relop.exist('*/sql:RelOp') as bit) IsNotLeaf

from @plan.nodes('//sql:RelOp') as ro(relop)

cross apply (

select sum(rti.info.value('@ActualRows', 'float')) ActualRows

from ro.relop.nodes('sql:RunTimeInformation/sql:RunTimeCountersPerThread') as rti(info)

) root_actual;

The output of the estimate rows and actual rows is given below:

NodeId	PhysicalOp	LogicalOp	EstimateRows	ActualRows	IsNotLeaf
0	Nested Loops	Left Outer Join	52	52	1
1	Nested Loops	Left Outer Join	52	52	1
2	Filter	Filter	52	52	1
3	Compute Scalar	Compute Scalar	52	NULL	1
4	Clustered Index Scan	Clustered Index Scan	52	52	0
13	Clustered Index Seek	Clustered Index Seek	52.000031	NULL	0
14	Clustered Index Seek	Clustered Index Seek	52	52	0

Hash Warning SQL Profiler Event

queryproc — Fri, 02 Feb 2007 06:18:00 GMT

One of the less well-known warning events that is logged to SQL Profiler trace is the Hash Warning event. Hash Warning events are fired when a hash recursion or hash bailout has occurred during a hashing operation. Both of these situations are less than desirable, as they mean that a Hash Join or Hash Aggregate has run out of memory and been forced to spill information to disk during query execution. When a hashing operation spills to disk, this almost always results in slower query performance and can cause space consumption increase in tempdb.

Note that the Hash Warning event needs to be explicitly enabled within SQL Profiler. It is not one of the “default” set of events. More info on SQL Profiler can be found here

What can be done if you see a lot of Hash Warning events? The recommended actions are:

· Make sure that statistics exist on the columns that are involved in the hashing operation. Without statistics, the hashing operation has no way to know how much memory to pre-allocate.

· Even if statistics do exist, try updating them, as this can give more current information to the hashing operation when it decides how much memory to allocate.

· Try using a different type of join (this can be done by hinting OPTION(MERGE JOIN) or OPTION(LOOP JOIN)). Note that forcing a join type does not necessarily guarantee a better execution plan.

· If all of these fail, you can increase the available memory on the computer.

A sample of what you will see in the profiler would look something like the following. Note the batch starting, followed by a number of Hash Warning events prior to batch completion. Also, the SPID that is causing the events will be recorded

EventClass	StartTime	SPID
SQL:BatchStarting	2007-02-01 18:53:34.703	51
Hash Warning	2007-02-01 18:53:48.267	51
Hash Warning	2007-02-01 18:53:48.283	51
Hash Warning	2007-02-01 18:53:50.097	51
Hash Warning	2007-02-01 18:54:05.300	51
SQL:BatchCompleted	2007-02-01 18:54:19.130	51

- Steve Herbert

SQL Server Query Execution

Tips, Tricks, and Advice from the SQL Server Query Processing Team : SQL Trace

Store Statistics XML in database tables using SQL Traces for further analysis.

Hash Warning SQL Profiler Event