Tips, Tricks, and Advice from the SQL Server Query Processing Team : Showplan_XML

How to Check Whether the Final Query Plan is Optimized for Star Join Queries?

queryproc — Mon, 09 Apr 2007 21:52:00 GMT

The star join optimization technique is an index based optimization designed for data warehousing scenarios to make optimal use of non-clustered indexes on the huge fact tables. The general idea is to use the non-clustered indexes on the fact table to limit the number of rows scanned from it. More details of index based star join optimization can be found at http://blogs.msdn.com/bi_systems/pages/164502.aspx.

The following discussion is based on SQL Server 2005 query plans.

In SQL Server 2005, we put "StarJoinInfo" element in Showplan XML to highlight star join optimization. If the query plan contains the “StarJoinInfo” element, then SQL Server has identified this plan as a star join plan and it definitely is one.

However, the query optimizer may not detect all star join plans due to star join detection restrictions. Hence there are some star join plans that won’t have the StarJoinInfo. This post will shed some light on how to manually detect if a given query plan is a star join plan.

These steps can help you identify what’s NOT a star join plan:

First identify your fact table.
If you see clustered index scan (or table scan) on fact table, then it’s not an index-based star join plan (however, this is a valid multi-table join plan, which can benefit from multiple-bitmap filter pushdown).

To identify a star join plan, you should:

Again, first identify your fact table
You should have a single RID lookup or clustered index seek on the fact table
Restrictive dimensions (dimension tables with restrictive filters in the query) should be processed before processing the clustered index seek or RID lookup on the fact table. You should find either:

A Cartesian product between the dimensions joined with a multi-column index on the fact table.
Or a semi-join of the dimensions with some non-clustered indexes on the fact table.
Or a join between multiple dimensions.

Non-restrictive dimensions are joined later with the fact table.

So the rule of thumb is: to detect whether a given plan is using index-based star join optimization, you should always look for a seek on a fact table that is based on some joins of some dimension tables.

Showplan Trace Events

queryproc — Tue, 17 Oct 2006 22:57:00 GMT

Besides SSMS, another great tool available to database developers and DBAs to view query plans and troubleshoot query compilation or execution issues is the SQL Server Profiler. In the Profiler, all the showplan events are listed under the Performance Event category. All the SQL Trace events generate showplan information at query-level granularity, i.e. a single XML document is generated for each query. There are 9 showplan events in SQL Server 2005! In this post, I will describe the commonly-used ones.

Showplan XML – This event is new to SQL Server 2005. It generates the query plan in XML and displays it as text in the TextData column. It also displays the query plan in graphical format in the profiler trace window. When enabled, this trace event is generated every time a T-SQL query is executed. It displays batch information along with the Showplan by grouping together statements that are executed in a batch. This trace event is equivalent to using Showplan XML SET OPTION in SSMS.
Showplan XML For Query Compile – Also introduced in SQL Server 2005, this event is similar to the Showplan XML event except that Showplan output is generated only when the query is compiled (or recompiled). For subsequent executions, if the query plan is retrieved from plan cache, no showplan information is displayed. This event is less expensive from a performance perspective and best for quick view of the generated query plan.
Showplan XML Statistics Profile – This event is similar to the Showplan XML event, it not only contains the compile-time plan information, but also includes the runtime query execution statistics such as actual number of rows and executions per iterator, memory grant and degree of parallelism, etc. The event is generated once per execution. The showplan output is displayed as XML in textual format in the TextData column. The query plan portion for the query below is shown in the attached document:

use nwind
go
select * from employees where hiredate < '1-1-2001'
go

Besides the XML events, the Performance Events category also contains other legacy showplan events. Of these I find the Showplan Statistics Profile event useful in displaying the execution stats in tabular format. The Performance Statistics event provides additional information (such as sql_handle, number of recompiles, etc.) which can be used to debug costing, plan generation, IO bottlenecks and other such related issues. Most of the legacy events generate Showplan in binary format which is displayed in the BinaryData column. The binary xml is converted to text format and is displayed in the expanded trace event viewing pane. We recommend using XML showplan event over the legacy showplan events since they contain additional information such as missing indexes, rows processed on each thread in a parallel query plan, memory grant per iterator, etc.

If using the Profiler to generate trace events is not an option, consider using SQL Trace to collect server trace data. SQL Trace is a mechanism that SQL Server supports to generate and capture server-side trace events using system stored procedures.

Gargi Sur
SQL Server Query Processing

Viewing and Interpreting XML Showplans

queryproc — Sat, 07 Oct 2006 00:03:00 GMT

As mentioned in our previous blog posting, SQL Server 2005 supports Showplan generation in XML format. XML-based Showplans provide greater flexibility in viewing query plans and saving them to files as compared to legacy Showplans. In addition to the usability benefits, XML-based Showplans also contain certain plan-specific information which is not available in Legacy Showplans. For example, Showplan XML contains the cached plan size, memory fractions (how memory grant is to be distributed across operators in the query plan), parameter list with values used during optimization, and missing indexes information which is not available with the legacy Showplan All option. Similarly Statistics XML contains additional information such as degree of parallelism, runtime memory grant, parameter list with actual values for all parameters used, and execution statistics such as count of rows/executes aggregated per thread (in a parallel query) as compared to the legacy Statistics Profile option. Such information is very useful in analyzing query compilation, execution and performance issues, hence using the new XML features for Showplan is highly recommended.

Showplan XML can be generated in two ways:

1. Using T-SQL SET options

2. Using SQL Server Profiler trace events

With the SET options, Showplan XML returns batch-level information, i.e. it produces one XML document per T-SQL batch. If the T-SQL batch contains several statements, it generates one XML node per statement and concatenates them together. However when Showplan XML is generated using trace events, it only generates statement-level information. Let’s analyze the Showplan XML output for the following query (see attached document - showplan.xml):

use nwind

set showplan_xml on

SELECT ContactName, OrderDate

FROM Customers inner join Orders

ON Orders.CustomerID = Customers.CustomerID

WHERE Orders.ShipCountry = 'Canada'

set showplan_xml off

The root element of the document contains the Showplan XML namespace attribute with the location of the Showplan XML schema, and the SQL Server build information. It contains batch and statement sub-elements. Each statement contains a "StatementText" attribute that describes the T-SQL query being executed, a "StatementId" attribute indicating the relative position of the statement in the batch, the relative cost of the statement and the level of optimization used to generate the query plan output. It also contains additional information like SET options in effect when executing the query.

<Batch>

<StmtSimple StatementText="SELECT ContactName, OrderDate FROM Customers inner join Orders ..

...

The QueryPlan node contains plan information such as size of the compiled plan, compilation time, etc. which is useful for debugging purposes. The iterators in the query plan are represented as nested elements, each of type ‘RelOp’. Every ‘RelOp’ element contains two types of information:

Generic information such as logical and physical operator names, optimizer cost estimates, etc. as attributes.
Operator specific information such as a list of output columns, a set of defined values, predicates, database objects on which they operate (tables/indexes/views), etc. as sub-elements.

For example, the topmost ‘RelOp’ in our example is a ‘Nested Loops’ that contains the following generic information:

And the following operator specific information:

</OutputList>

</OuterReferences>

…

</NestedLoops>

The runtime counterpart of Showplan XML is called ‘Statistics XML’. It displays query execution “statistics” by executing the query and aggregating runtime information on a per-iterator basis. To obtain the Statistics XML output, use the following SET option.

set statistics xml on

Now lets analyze the Statistics XML output of the above query. The XML generated is semantically similar to the Showplan XML output, some of the key differences are:

In the QueryPlan node, an additional attribute indicating the degree of parallelism is present:

Each iterator ‘RelOp’ node contains an additional sub-element called RunTimeInformation that contains iterator execution profile such as the number of rows, number of executions, indexes accessed, join order, etc.

<RelOp NodeId="0" PhysicalOp="Nested Loops" LogicalOp="Inner Join" EstimateRows="43" EstimateIO="0"

EstimateCPU="0.00017974" AvgRowSize="34" EstimatedTotalSubtreeCost="0.0308059" Parallel="0" EstimateRebinds="0" EstimateRewinds="0">

</OutputList>

<RunTimeInformation>

<RunTimeCountersPerThread Thread="0" ActualRows="43" ActualEndOfScans="1" ActualExecutions="1" />

</RunTimeInformation>

The runtime information is shown per thread, and not aggregated as in legacy showplan.
For memory consuming iterators such sort or hash join, the memory grant information is printed.

In a later blog post, i will cover some of the Profiler Trace Events for Showplan.

- Gargi Sur

SQL Server Query Processing Development Team

Introduction to Showplan

queryproc — Sat, 30 Sep 2006 01:16:00 GMT

Showplan is a feature in SQL Server to display and read query plans. While some of you may already be very familiar with Showplan, it is one of the most important diagnostic tools that we use in the query processing team to locate and identify problems, and therefore deserves some extra exposure. Being able to collect, read, and understand Showplan data is a critical skill to have, and one which we plan to blog about a fair bit. Consider this an introductory posting to cover some of the basics which we will drill into more in further posts.

We generate two types of Showplans in SQL Server, one at query compilation time (when an optimized query plan is produced) and the second at query runtime (when the optimized query plan is executed). The former allows you to see the compiled query plan in its entirety, as well as a ton of useful information about each operator in the plan. The runtime plan, often referred to as query execution “statistics”, enables you to collect actual metrics about the query during its execution, such as its execution time and actual cost.

Showplan and statistics information can be extracted in two ways from the server, 1) using query SET options, 2) using Profiler Trace events. The various SET options available are:

· Legacy Showplan

o Compile Time: SET SHOWPLAN_ALL ON or SET SHOWPLAN_TEXT ON

o Runtime: SET STATISTICS PROFILE ON

· Showplan XML (Preferred)

o Compile Time: SET SHOWPLAN_XML ON

o Runtime: SET STATISTICS XML ON

SQL Server Management Studio (SSMS) uses Showplan to display a graphical representation of the query plan. A graphical view of the query plan for the following query on the ‘nwind’ database is shown in the picture below.

-- Query: select customer name and order date for orders placed by customers in Canada

use nwind

select ContactName, OrderDate

from Customers inner join Orders

on Orders.CustomerID = Customers.CustomerID

where Orders.ShipCountry = 'Canada'

(See the attached .bmp file for a picture of this showplan)

In SSMS, to view the graphical plan, click on the Execution Plan tab in the Results Pane. The graphical plan is read from top-to-bottom, right-to-left. When you select any operator in the query tree or simply hover the mouse on it, you will see a tooltip that describes the operator. It displays the Query Optimizer cost estimates (operator and subtree costs, number of rows, row size, etc.) and additional information like output columns, predicates, etc. The detailed operator information is shown in the Properties window (View --> Properties Window), which is usually displayed on the extreme right frame in SSMS.

To save the graphical showplan to a file you can right click on the Execution Plan and select 'Save Execution Plan As'. The query plan is saved with extension '.sqlplan' and can be reloaded into SSMS anytime, sent via email, etc.

Showplan output can be generated in text, grid or XML formats. The XML format was introduced in SQL Server 2005 which we refer to as “Showplan XML” and the other formats as “Legacy Showplan”. In the next posting we will describe how to generate and analyze Showplan XML.

- Steve and Gargi