Bart Duncan's SQL Weblog : SQL Performance

Sometimes the Simplest Solution Isn't the Best Solution (The All-In-One Search Query)

bartd — Sun, 03 May 2009 22:13:00 GMT

Programmers should naturally gravitate toward the simplest, most elegant solution. This is because the simplest coding solution is so often the best solution: simple solutions are cheaper to implement; easier for others to understand, maintain, and extend; and less prone to bugs. Simplicity can also be associated with efficient and consistent execution, but performance is one area where Occam's Razor doesn't apply quite as consistently. Performance requirements, more often than many other types of requirement, may demand introducing complexity into what would otherwise be a nice, tidy implementation. I’m going to discuss one such case here.

The All-In-One Search Query

Sometimes you need a query or a stored procedure to expose several search parameters but actually do the search only for the subset of the parameters that the caller cares about. For example, consider a [find_customer] stored procedure that can search for a @name value, a customer @id, an @address, or any combination of these. The most obvious solution to this problem looks something like this:

SELECT [id], [name], [address], ...

FROM [customers]

WHERE (@name IS NULL OR [name] = @name)

AND (@id IS NULL OR [id] = @id)

AND (@address IS NULL OR [address] = @address)

This allows the caller to pass non-NULL values for the search criteria they care about, and to pass NULLs for any criteria that they want to be ignored. For lack of a better term, I have always referred to this as the “All-In-One Search Query”. Here’s a sample stored proc that shows a slightly simplified example with just two available search parameters:

CREATE PROC find_customer

@name nvarchar(128) = NULL,

@id int = NULL

BEGIN

SELECT [id], [name], [address], [creation_date]

FROM [customers]

WHERE (@name IS NULL OR [name] = @name)

AND (@id IS NULL OR [id] = @id);

END;

This idiom is simple enough that it is relatively easy to understand. Despite its simplicity, it allows a single query to meet many different needs (in this case: a search on name, or a search on address, or a search on both name and address). It is understandable that queries like this are so popular; it seems to be an elegant solution, and developers like elegance. Unfortunately, it’s usually a bad idea for practical performance reasons. The problems are significant enough that I regard this as an anti-pattern in my own code.

What's Wrong With the Obvious Solution?

The first problem is that the OR operators will prevent an efficient index seek-based query plan. If the search parameters are mutually exclusive (the caller is only supposed to provide one parameter value), you can avoid this problem by changing the query to:

SELECT [id], [name], [address], [creation_date]

FROM [customers]

WHERE (@name IS NOT NULL AND [name] = @name)

OR (@id IS NOT NULL AND [id] = @id);

That may allow an index seek, but you should expect that it will still perform poorly in some cases because it suffers from other problems that I’ll discuss next. Also note that the query semantics aren't quite the same as the first query.

The second problem that affects both the original problem query and the attempted rewrite is that SQL does a poor job with plan costing for queries like this. In general, you should avoid using filter predicates where both operands are constants and one or both are variables or parameters (e.g. “where @var = 1”, or “where @parameter is null”). These have a tendency to screw up the optimizer’s estimates because at plan compile time the optimizer has no idea whether the predicate will qualify all rows, or disqualify all rows. You are essentially guaranteed that its blind guess will be exactly wrong for some parameters. The bad estimate is likely to cause inaccurate costing of plan alternatives, which in turn is likely to cause selection of a suboptimal plan.

The third problem is that there is no single plan that will be appropriate for all of the different combinations of parameters, even if the optimizer was smart enough to predict the outcome of “variable = constant”-type predicates at compile time. For example, suppose you had a nonclustered index on the [name] column, and another nonclustered index on [id]. If SQL chose a plan that first did a lookup using the index on the [name] column, this would be a very poor choice whenever the caller passed NULL for @name. Conversely, a plan that first scanned an index on the [id] column would be inefficient when @id was null and @name was non-null.

The only situations where you should use queries like either of those shown above is when (a) you don't care about the performance of the query, or (b) you can guarantee that the table will only contain a handful of rows so that a table scan-based plan will never be noticeably slower than a seek-based plan.

Possible Solutions

One possible solution would be to have a different query for each search parameter (or combination of search parameters, if that is legal input for the procedure). This is shown below. It may be a practical solution if you only have two or three possible search parameters.

CREATE PROC find_customer

@name nvarchar(128) = NULL,

@id int = NULL

BEGIN

IF (@name IS NULL)

BEGIN

SELECT [id], [name], [address], [creation_date]

FROM [customers]

WHERE [id] = @id;

END

ELSE IF (@id IS NULL)

BEGIN

SELECT [id], [name], [address], [creation_date]

FROM [customers]

WHERE [name] = @name;

END

ELSE BEGIN

SELECT [id], [name], [address], [creation_date]

FROM [customers]

WHERE [name] = @name AND [id] = @id;

END;

Note that parameter sniffing may cause bad plan selection even in this case if each search query is more complex than the queries shown here (if the queries include joins against large tables, for example). You can protect against this problem by assigning the parameters to local variables and using the variables in the queries. You can also avoid the problem by moving the search queries to a different compilation scope (child stored procedures, dynamic SQL, or "OPTION(RECOMPILE)" query hints).

The second possible solution uses dynamic SQL to construct the WHERE clause dynamically.

CREATE PROC find_customer

@name nvarchar(128) = NULL,

@id int = NULL

WITH EXECUTE AS OWNER

BEGIN

DECLARE @search_query nvarchar(max);

SET @search_query = '

SELECT [id], [name], [address], [creation_date]

FROM [customers]

WHERE 1 = 1 ';

IF (@name IS NOT NULL)

BEGIN

SET @search_query = @search_query + 'AND [name] = @name';

END;

IF (@id IS NOT NULL)

BEGIN

SET @search_query = @search_query + 'AND [id] = @id';

END;

EXEC sp_executesql @search_query, N' @name nvarchar(128), @id int',

@name = @name, @id = @id;

END;

You should explicitly parameterize the query, as shown here, to avoid the risk of SQL injection attacks. Note the EXECUTE AS OWNER used for this procedure, which may be appropriate to avoid the need to grant the end user SELECT permission on the base table(s). Also note that the pattern is simplified by always passing all parameters to sp_executesql, even though the query may only make use of a subset of the parameters. I generally don't like to use dynamic SQL where I can avoid it, but I make an exception for this case; the dynamic WHERE clause is actually my preferred solution when I'm faced with this problem.

I’ve seen the All-In-One search query cause perf problems in code written both inside and outside of Microsoft, and I’ve been burned by the problem in my own code. It’s unfortunate that the better alternatives all require more complex code, but this is a case where the additional complexity is justified.

UPDATE: Erland Sommarskog has a page on this very topic here: http://www.sommarskog.se/dyn-search-2005.html. He calls it the problem of "dynamic search conditions" (a less awkward phrase than "All-In-One Search Query"). The page provides a very detailed discussion of the topic, and would be a great read if you really want to drill into the details.

UPDATE #2: Another good read on a variation of this problem: http://sqlinthewild.co.za/index.php/2009/03/19/catch-all-queries/ (and, to a lesser degree, http://sqlinthewild.co.za/index.php/2009/09/15/multiple-execution-paths/), both from Gail Shaw. Gail calls this class of query "Catch-All Queries".

Defining Complex Server "Health" Policies in SQL 2008

bartd — Fri, 12 Sep 2008 01:17:00 GMT

Policy Based Management (PBM) is a new feature in SQL Server 2008 that allows you to define a set of policies that capture the intended state for a group of servers. For example, you could define a policy that says that your user databases should all have the auto update statistics database option enabled. (If you’re not yet familiar with PBM, you can read more about it in Books Online or in the PBM blog.)

In SQL 2008, the focus of PBM is primarily on static aspects of server management – policies covering things like schema requirements, or server and database configuration settings. However, there are certain more dynamic aspects of server state that are equally important, but much harder to monitor. Server “health” monitoring (e.g. uptime, responsiveness) is one example. I’m going to show you how you can use the ExecuteSql function to extend the normal capabilities of PBM by defining policies that ensure your servers are servicing queries effectively. You can use live Dynamic Management View (DMV) queries, or even query historical data that you are capturing in a Management Data Warehouse.

IMPORTANT: Before going on, read through this blog post for an overview of the PBM ExecuteSql function.

Suppose you wanted to define a policy like this one:

The average disk response time for all data and log files that have a non-trivial number of I/Os should not exceed 100ms.

You can use a query like this one to find files that violate this policy:

SELECT

CASE

WHEN MAX (avg_ms_per_io) > 100 THEN

'Excessive disk response time ('

+ CONVERT (varchar, MAX (avg_ms_per_io))

+ 'ms) for file ID ' + CONVERT (varchar, MAX (file_id))

+ ', database ' + MAX (database_name)

ELSE ''

END AS policy_violation_message

FROM

(

SELECT TOP 1

io_stall / (num_of_reads + num_of_writes) AS avg_ms_per_io,

io_stall AS io_stall_ms,

(num_of_reads + num_of_writes) AS num_io,

DB_NAME (database_id) AS database_name,

file_id

-- Only check I/O stats for the current database

FROM sys.dm_io_virtual_file_stats (DB_ID(), null)

WHERE

-- Ignore idle databases and files; we do not care about disk

-- wait time if the file has barely seen any I/O.

(num_of_reads + num_of_writes) > 500

-- PBM will use the first column of the first row to evaluate policy

-- compliance, so return the file with the longest avg I/O time.

ORDER BY 1 DESC

) AS slowest_file;

We’re going to wrap this query in PBM’s ExecuteSql() function, which will allow us to reference it within a policy condition. To permit PBM to bubble up more actionable info to the user (other than the simple fact that a database is out of policy), I’ve written the query to provide a string-type policy compliance indicator rather than a simple numeric 0/1 value. If the query detects that a database is in policy, it returns an empty string for the [policy_violation_message] column. But if disk response time is found to be out of policy, the query returns a message identifying the offending file and its average disk response time. This message will be logged with the policy results, which can help during postmortem diagnosis.

1. Create a new policy (in Management Studio under Management\Policies, right-click the Policies folder and select New Policy). Name the policy “Disk Health”.

2. In the Check Condition drop-down listbox, select New condition…

3. Name the condition “Disk Response Time is Healthy”, and set the Facet drop-down listbox to Database.

4. Click the “…” button next to the Field textbox. In the Advanced Edit dialog, enter the text shown below, then click OK (this is the same query as above, except that single quotes have been escaped by doubling them).

ExecuteSql('String', '

DECLARE @max_allowed_ms_per_io int;

SET @max_allowed_ms_per_io = 100;

SELECT

CASE

WHEN MAX (avg_ms_per_io) > @max_allowed_ms_per_io THEN

''Excessive disk response time (''

+ CONVERT (varchar, MAX (avg_ms_per_io))

+ ''ms) for file ID '' + CONVERT (varchar, MAX (file_id))

+ '', database '' + MAX (database_name)

ELSE ''''

END AS policy_violation_message

FROM

(

SELECT TOP 1

io_stall / (num_of_reads + num_of_writes) AS avg_ms_per_io,

io_stall AS io_stall_ms,

(num_of_reads + num_of_writes) AS num_io,

DB_NAME (database_id) AS database_name,

file_id

-- Only check I/O stats for the current database

FROM sys.dm_io_virtual_file_stats (DB_ID(), null)

WHERE

-- Ignore idle databases and files; we do not care about disk

-- wait time if the file has barely seen any I/O.

(num_of_reads + num_of_writes) > 500

-- PBM will use the first column of the first row to evaluate policy

-- compliance, so return the file with the longest avg I/O time.

ORDER BY 1 DESC

) AS slowest_file;

5. Set the Operator to equals (=) and the Value to '' (Note: that’s two single quotes, not a double quote).

Your new policy should look like this. If it does, click OK twice to create the policy.

Now right-click on your policy and select Evaluate. Click the Evaluate button in the lower-right portion of the Evaluate Policies dialog, and acknowledge that you realize that the policy includes a custom script (see the blog post I referenced earlier for more information about the security implications of this approach). This will run the policy against each database on your instance, and will flag any that are suffering from poor disk response time. In the screenshot below, you can see that my [mdw] and [testdatabase] databases are in policy, but my [AdventureWorks] database is out of policy: the average I/O response time for file 1 in this database is an embarrassingly slow 122ms.

Because ExecuteSql takes an arbitrary query, you can define a policy that monitors literally any aspect of server health that you can write a query to detect. Below, for example, is a query that will trigger a policy violation if the value of the “SQLServer:Buffer Manager\Page Life Expectancy” perfmon counter drops too low:

ExecuteSql('String', '

SELECT

CASE

WHEN cntr_value < 300 THEN

''Page life expectancy (sec): ''

+ CONVERT (varchar, cntr_value)

ELSE ''''

END AS policy_violation_message

FROM sys.dm_os_performance_counters

WHERE counter_name = ''Page life expectancy'' AND object_name LIKE ''%Buffer Manager%'';

SQL Health Monitoring Policies and Management Data Warehouse

If you’ve spent much time trying to use SQL DMVs for server health monitoring in the past, you’ve probably run into situations where you needed to compare the live DMV data to an historical snapshot of the same data. For example, consider the “slow I/O” query at the top of this post. The I/O counters and disk “stall” time reported in sys.dm_os_virtual_file_stats are cumulative totals since the SQL instance was started. If the server had been running a long time, these values would be very large, and any disk performance problem would have to exist for a long time before the overall average I/O response time grew high enough to trigger the threshold. What you typically want in a case like this is to assess recent I/O performance, where “recent” means the average within the last several minutes or hours. Here's a more refined version of the policy we started with, that would make this type of monitoring more useful in the real world:

The average disk response time for all data and log files that have a non-trivial number of I/Os should not exceed 100ms (where "average disk response time" is defined as the average I/O wait time for all reads or writes to a file within the last 30 minutes)

This requires historical snapshots of the same data to compare the most recent data to. Luckily, this happens to be exactly what the Data Collector and Management Data Warehouse (MDW) features in SQL 2008 were intended to provide.

Being able to take advantage of historical data in your health monitoring policies has a number of advantages:

· For cumulative-since-server-startup DMV data, the policy will flag problems with less delay

· Similarly, a monitored object can go back “in policy” relatively quickly once the problem has been addressed

· For metrics that are instantaneous measurements, you get fewer false positive policy violations if you can average a set of recent measurements instead of focusing on a single instantaneous data point that might be an atypical value

The query below pulls recent disk stats from a local MDW database (this DMV’s data is collected by the built-in Server Activity collection set, so you don’t need to create a custom collection set). This assumes that (a) the monitored SQL instance is hosting its own MDW database, and (b) the MDW database’s name is “MDW”. If your MDW database is local but is named something other than "MDW", update the query text to reference the correct MDW database. (If you need a walkthough showing how to set up MDW, see this page.)

ExecuteSql('String', '

DECLARE @max_allowed_ms_per_io int;

DECLARE @time_window_min int;

SET @max_allowed_ms_per_io = 100;

SET @time_window_min = 120;

SELECT

CASE

WHEN MAX (recent_avg_ms_per_io) > @max_allowed_ms_per_io THEN

''Excessive disk response time (''

+ CONVERT (varchar, MAX (recent_avg_ms_per_io))

+ ''ms) for file ID '' + CONVERT (varchar, MAX (file_id))

+ '', database '' + MAX (database_name)

ELSE ''''

END AS policy_violation_message

FROM

(

SELECT TOP 1

(recent_io_stall_read_ms + recent_io_stall_write_ms)

/ (recent_read_count + recent_write_count + 1)

AS recent_avg_ms_per_io,

(recent_io_stall_read_ms + recent_io_stall_write_ms)

AS recent_io_stall_ms,

(recent_read_count + recent_write_count) AS recent_io_count,

database_name,

file_id

FROM

(

SELECT

database_name,

file_id,

MAX (io_stall_read_ms) - MIN (io_stall_read_ms)

AS recent_io_stall_read_ms,

MAX (io_stall_write_ms) - MIN (io_stall_write_ms)

AS recent_io_stall_write_ms,

MAX (num_of_reads) - MIN (num_of_reads) AS recent_read_count,

MAX (num_of_writes) - MIN (num_of_writes) AS recent_write_count

FROM mdw.snapshots.io_virtual_file_stats AS fs

INNER JOIN mdw.core.snapshots AS snap

ON fs.snapshot_id = snap.snapshot_id

WHERE

-- MDW data for the local instance only

snap.instance_name = @@SERVERNAME

-- I/O stats only for the current database

AND fs.database_id = DB_ID()

-- within the last 5 minutes

AND fs.collection_time >

DATEADD (minute, @time_window_min, SYSDATETIMEOFFSET())

GROUP BY database_name, file_id

) AS recent_io_stats

WHERE

-- Ignore idle databases and files; we do not care about disk

-- wait time if the file has barely seen any I/O.

(recent_read_count + recent_write_count) > 500

-- PBM will use the first column of the first row to evaluate policy

-- compliance, so return the file with the longest avg I/O time.

ORDER BY 1 DESC

) AS slowest_file;

I think this approach to server health monitoring could be useful, but it does have some limitations that are worth noting:

· This release of SQL doesn’t provide out-of-the-box policies of this sort. You have to write custom out-of-policy detection queries for any complex server health condition you want to monitor.

· Complex policies may have parameters that you might want to tweak on a per-server basis (in this example, the max allowed disk response time is one such parameter). Unfortunately, there is no facility for this in PBM in SQL 2008. If you really need this, you could create your own custom policy configuration table on each server with thresholds particular to that server, and join to this table in your custom condition query.

· When you use ExecuteSql, you can only choose “=” or “!=” for your operators. Because you can’t use inequality operators, you must push your policy violation thresholds down into the query itself.

If you’ve tried this approach to health monitoring and have experiences you can share, good or bad, let me know.

Query Fingerprints and Plan Fingerprints (The Best SQL 2008 Feature That You've Never Heard Of)

bartd — Wed, 03 Sep 2008 22:20:00 GMT

In versions of SQL Server before SQL Server 2008, it can be difficult to determine the cumulative cost of the queries running on a server if the workload includes unparameterized queries. The only truly reliable method is to capture a Profiler trace during a representative time period, then post-process the trace with a utility that strips out any inline literal values from the query text. A number of utilities have sprung up that use this general approach: ReadTrace, SQL Nexus/TraceBuster, and Bill Graziano’s ClearTrace utility are three that I know of. Trace-based query cost analysis is effective, but there are a number of big problems with this approach: capturing a batch- or statement-level profiler trace is expensive and sometimes can slow down your server, a trace can grow up to several GB per minute on servers that have a high transaction rate, and the capture and analysis of the data tends to be a time-consuming and labor-intensive process that is difficult to automate.

In SQL Server 2008, the SQL Server database engine has a powerful new feature that generates an identifier for each query. The identifier is independent of any inline parameter values, so it serves as a very effective query identifier. This identifier – sometimes called a “query fingerprint” – enables a fairly robust method of identifying the most expensive queries on your server based on nothing but DMV queries. I think the feature will eventually form the basis of a query cost analysis approach that requires a much smaller investment of DBA time, has greatly reduced risk, and scales to higher-volume workloads than a traditional trace-based analysis.

This is my personal favorite “sleeper feature” in SQL 2008; the query optimizer team deserves kudos for getting it done in time for the release, and for making it inexpensive enough that fingerprint generation can be on-by-default. It was a very late addition to the release, which I think is the main reason that fingerprints have generated relatively little buzz so far. Plan fingerprints are used to calculate query plan cost in the new Activity Monitor tool that you can launch by right-clicking a server in SQL Server 2008's Management Studio.

To fully appreciate query fingerprints’ value, you first must understand one of the limitations of the sys.dm_exec_query_stats Dynamic Management View (DMV) that was introduced in SQL Server 2005. This DMV lists every statement query plan that is in procedure cache at that moment, along with execution stats for the query plan such as the number of executions of the plan, total CPU cost, physical and logical reads, and so on. It was a groundbreaking addition to SQL Server because, for the first time, it allowed a DBA to examine the cost of the queries in a workload without capturing a Profiler trace. For certain workloads, the original sys.dm_exec_query_stats DMV provides a simple and powerful way to identify the most expensive queries: you capture two snapshots of the DMV, then you join the second snapshot back to the first to calculate the execution cost of each query plan in the time between the two snapshots. But the DMV also has some limitations that constrain its usefulness. In particular, if a query is not explicitly or implicitly parameterized and if the query text contains inline literal values, that query plan will not be reused. Every execution of the query with a different set of parameter values will generate a new compiled plan object. You can see this by running these queries:

-- Run "the same" query twice, but with a different inline parameter

-- value for each execution

SELECT type FROM sys.objects WHERE name = 'sysfiles1'

SELECT type FROM sys.objects WHERE name = 'sysprivs'

-- Find the query execution statistics row(s) for the query

SELECT sql_handle, plan_handle, execution_count

FROM sys.dm_exec_query_stats AS qs

CROSS APPLY sys.dm_exec_sql_text (qs.plan_handle) AS sql

WHERE sql.text LIKE 'SELECT type FROM sys.objects WHERE name %'

Here's the output:

The two queries that look like “SELECT type FROM sys.objects WHERE name = 'x'” are identical except for a different inline parameter value, but each query’s statistics are tracked in different rows in sys.dm_exec_query_stats. This happens because the query is not explicitly parameterized, and the different inline literal values cause SQL Server to compile a separate query plan each time the query is executed with a different parameter value. This behavior makes it difficult to use sys.dm_exec_query_stats to identify the true cumulative cost of a query. Suppose that this query was executed frequently enough to be the most expensive query on your server, but suppose that most executions had a different search parameter value. With the query’s cumulative cost spread out across thousands of rows in sys.dm_exec_query_stats, how would you recognize that each of those thousands of rows actually represents a fraction of the total execution cost of a single query? This is a key problem that the query fingerprint feature in SQL Server 2008 helps to address. To see how it works, add these queries to your query window, and then re-execute the entire script:

-- Execute a second query that is not quite the same shape as the prior query

SELECT type FROM sys.objects WHERE name = 'sysprivs' AND create_date < GETDATE()

-- Find the query execution statistics row(s) for the query

SELECT sql_handle, plan_handle, execution_count, query_hash, query_plan_hash

FROM sys.dm_exec_query_stats AS qs

CROSS APPLY sys.dm_exec_sql_text (qs.plan_handle) AS sql

WHERE sql.text LIKE 'SELECT type FROM sys.objects WHERE name %'

When you re-run the script, you should see output like the following:

Note the query_hash and query_plan_hash columns that are new to sys.dm_exec_query_stats in SQL Server 2008. This is one of two places where the new SQL Server 2008 query fingerprint and query plan fingerprint features are exposed (the other is the sys.dm_exec_requests DMV). Note that the two queries we ran first have the same query_hash value, which indicates that if you were to strip out any inline literals, the two queries have the same “shape”. They also have the same query_plan_hash value, which means that even though they each had their own compiled plan object in the procedure cache, those two query plans also had the same general form. However, the second query doesn’t have quite the same “shape” as the first two queries; it includes an additional filter predicate (“create_date < GETDATE()”), and therefore has different query_hash and query_plan_hash values. Using these two new columns, we can combine the costs of all plans that have the same shape and calculate the true cumulative cost of all executions of a query. This approach generally works well even when faced with an unparameterized workload that features poor plan reuse.

Below I've provided some more detailed information about query fingerprints and query plan fingerprints. This information should be accurate as of the initial release of SQL Server 2008, but it is possible that some of the implementation details will change in subsequent releases. In order to protect the ability to improve the feature, we’re not providing an iron-clad guarantee that a query’s fingerprint won’t change across SQL versions, although we recognize the value of an identifier that remains constant across releases and plan to avoid changes to the fingerprint calculation algorithms unless there’s a very good reason to change them.

Query Fingerprints

A query fingerprint is also called a “query hash”. The queries below are all similar enough to have the same query fingerprint:

SELECT * FROM foo WHERE column1 = 'A' AND column2 = 'B'

SELECT * FROM foo WHERE column1 = 'X' AND column2 = 'Y'

SELECT * FROM foo WHERE column1 = 100 AND column2 = 200

SELECT * FROM foo WHERE foo.column1 = 'X' AND foo.column2 = 'Y'

On the other hand, all of the queries below are have differences that are significant enough to produce different query fingerprints:

SELECT * FROM foo WHERE column1 = 'X'

SELECT * FROM foo AS foo2 WHERE column1 = 'X' AND column2 = 'Y'

SELECT * FROM foo WHERE column1 = 'X' AND column2 IS NULL

SELECT * FROM foo WHERE column1 = 'X' OR column2 = 'Y'

SELECT column1 FROM foo WHERE column1 = 'X' AND column2 > 'Y'

Here are some facts that help clarify how query fingerprints are generated:

Query fingerprints are generated from a tree of logical operators that is used as an input to the query optimizer. Because this tree is created prior to query optimization, a query fingerprint is influenced only by the query’s text, not by the query plan. In other words, two queries may have the same query fingerprint but use two very different query plans.
Similarly, because the query fingerprint is generated from an operator tree *after* parsing, two queries don't have to have the exact same text to have the same fingerprint. Whitespace and comments don't matter, and the queries can even have some small, semantically-irrelevant differences.
Query fingerprints are not affected by the current database context or by the current instance name. This means that the same query run within two different databases, or even two different SQL Server instances, will generally have the same query fingerprint value.
The table, view, and function names referenced by two queries must be identical for the queries to have the same fingerprint. This includes table aliases; if two queries are identical except that one of them uses a table alias and the other query refers to the table using its actual name, the two queries will not have the same fingerprint.
If a query has any table or query hints (including any hints applied via a plan guide), the hints must be identical in order to generate the same query fingerprint.
SET options may influence query fingerprints if they change the query semantics. For example, the logical meaning of a predicate like “column1 = NULL” is influenced by the current ANSI_NULLS setting. If the query includes a predicate like this, then changing the ANSI_NULLS setting may generate a different query fingerprint.

Plan Fingerprints

A query plan fingerprint may also be referred to as a “plan hash”. Plan fingerprints are generated from the tree of physical operators that makes up a compiled query execution plan. Generally speaking, if a user would consider two plans to be different, they will have different plan fingerprint values.

In order to have the same plan fingerprint value, the trees of operators that make up the plans must have the same shape. For each physical operator in one plan, the corresponding node in the other plan must be the same physical operator. For example, if two plans have the same general shape but one plan includes a Hash Join operator where the second plan uses a Loop Join, the two plans will not have the same plan fingerprint.

Certain operator attributes must also be identical in order to generate a matching plan fingerprint. For example, two Table Scan operator must reference the same table name, or they will not match. However, not every attribute of an operator is included in the query plan hash value. For example, the specific number of rows that is estimated to be returned from a Table Scan operator does not influence the plan fingerprint. Two cached plans may have slightly different estimated row counts, yet have the exact same shape and the same execution characteristics, and therefore the same plan fingerprint.

Like query fingerprints, a plan fingerprint is not affected by database context or SQL Server instance name, so if similarly-shaped query plans are used in two different databases, the plans will receive the same plan fingerprint. Plan fingerprints are sensitive to object names, but an exception exists for automatically-generated primary key and unique key constraints. (Unfortunately, due to an issue in the initial release of SQL Server 2008, a query that references a local temporary table will generate different plan and query fingerprints for each execution. Hopefully, this will be addressed in a subsequent Cumulative Update or Service Pack release.)

Finally, in the initial implementation of the plan fingerprint feature, a statement’s query fingerprint value is included in the plan hash. In other words, two different queries with different query fingerprints may result in the “same” plan, but the plan fingerprints will be different simply because the query fingerprints are different.

Limitations

Here are some of the more important restrictions that you should be aware of:

In SQL 2008, the query and plan fingerprints are still tied to the sys.dm_exec_query_stats DMV, which means that the aggregate stats for a fingerprint are associated with cached query plans. Query plans have a transient lifetime, and may be removed at any time in response to internal or external memory pressure on the procedure cache. Any query statistics that are inserted and removed in the interval between two queries against the DMV will not be reflected in your query cost estimates (you can partially compensate for this by querying the DMV at a more frequent interval). Also, certain types of query plan are never inserted into the procedure cache (one example is the plan for a CREATE INDEX). Execution statistics for these types of queries may be undercounted.
The sys.dm_exec_query_stats DMV only shows statistics for completed query executions. In-progress, long-running queries will not show up in the DMV until they finish running. You can merge the stats from this DMV with the the sys.dm_exec_requests DMV (which also exposes the new fingerprint columns) in order to get a more complete view.
Using fingerprints to determine cumulative query cost relies on using the new query_hash and query_plan_hash columns as keys that uniquely identify a particular query’s or plan’s “shape”. It is possible, though unlikely, that two different queries may end up with the same hash value, causing statistics for both of the queries to be charged to one of them. It is also possible that two variations of the “same” query may be assigned different fingerprint hash values, in which case the cost of the query may appear to be spread out over several buckets, making it difficult to recognize the query’s true cumulative cost. At the time of the initial release of SQL Server 2008, we’ve only encountered one instance of this: different executions of a query that references a temp table may be assigned different query_hash values. This problem with temp tables isn’t intentional; it’s a bug, and I’m hopeful that it will be fixed in an upcoming Service Pack or Cumulative Update.

Future Plans

I already mentioned the new SQL 2008 Activity Monitor, which uses plan fingerprints to generate more accurate estimates of query cost than would otherwise be possible. Looking forward, we have plans to publish a new custom collection set based on the SQL Server 2008 Data Collector that uses plan fingerprints to provide a Top N Query identification tool. You will be able to use this new “Query Hash Statistics” collection set as-is, or customize it to meet your needs. The documentation that accompanies this collection set will include a subset of the info in this post, so if you’ve read to this point, you’ll be able to skim over some of the collection set docs.

I suspect that it won’t be terribly long before your favorite SQL monitoring tool starts to take advantage of this feature, but if you’re of the roll-your-own bent, you can start creating your own custom query analysis scripts right now using the info provided here. Below I've provided a query to get you started. It groups on query_plan_hash to calculate query statistics for all plans with a given plan fingerprint, and pulls the statement text from a representative cached plan object that has a given fingerprint.

SELECT TOP 100

    query_hash, query_plan_hash,

    cached_plan_object_count,

    execution_count,

    total_cpu_time_ms, total_elapsed_time_ms,

    total_logical_reads, total_logical_writes, total_physical_reads,

    sample_database_name, sample_object_name,

    sample_statement_text

FROM

(

    SELECT

        query_hash, query_plan_hash,

        COUNT (*) AS cached_plan_object_count,

       MAX (plan_handle) AS sample_plan_handle,

        SUM (execution_count) AS execution_count,

        SUM (total_worker_time)/1000 AS total_cpu_time_ms,

        SUM (total_elapsed_time)/1000 AS total_elapsed_time_ms,

        SUM (total_logical_reads) AS total_logical_reads,

        SUM (total_logical_writes) AS total_logical_writes,

        SUM (total_physical_reads) AS total_physical_reads

    FROM sys.dm_exec_query_stats

    GROUP BY query_hash, query_plan_hash

) AS plan_hash_stats

CROSS APPLY

(

    SELECT TOP 1

        qs.sql_handle AS sample_sql_handle,

        qs.statement_start_offset AS sample_statement_start_offset,

        qs.statement_end_offset AS sample_statement_end_offset,

        CASE

            WHEN [database_id].value = 32768 THEN 'ResourceDb'

            ELSE DB_NAME (CONVERT (int, [database_id].value))

        END AS sample_database_name,

        OBJECT_NAME (CONVERT (int, [object_id].value), CONVERT (int, [database_id].value)) AS sample_object_name,

        SUBSTRING (

            sql.[text],

            (qs.statement_start_offset/2) + 1,

            (

                (

                    CASE qs.statement_end_offset

                        WHEN -1 THEN DATALENGTH(sql.[text])

                        WHEN 0 THEN DATALENGTH(sql.[text])

                        ELSE qs.statement_end_offset

                    END

                    - qs.statement_start_offset

                )/2

            ) + 1

        ) AS sample_statement_text

    FROM sys.dm_exec_sql_text(plan_hash_stats.sample_plan_handle) AS sql

    INNER JOIN sys.dm_exec_query_stats AS qs ON qs.plan_handle = plan_hash_stats.sample_plan_handle

    CROSS APPLY sys.dm_exec_plan_attributes (plan_hash_stats.sample_plan_handle) AS [object_id]

    CROSS APPLY sys.dm_exec_plan_attributes (plan_hash_stats.sample_plan_handle) AS [database_id]

    WHERE [object_id].attribute = 'objectid'

        AND [database_id].attribute = 'dbid'

) AS sample_query_text

ORDER BY total_cpu_time_ms DESC;

Are you using SQL's Missing Index DMVs?

bartd — Thu, 19 Jul 2007 12:29:00 GMT

Did you know that your SQL Server is keeping track of the indexes that it thinks you should create? The "missing index" DMVs in SQL are a really great new feature in SQL Server 2005 that (in my opinion) seem to have been underutilized so far. If you want to see if this feature can spare you the tedium of an afternoon identifying poor performing queries and tuning them, all you have to do is ask:

SELECT

migs.avg_total_user_cost * (migs.avg_user_impact / 100.0) * (migs.user_seeks + migs.user_scans) AS improvement_measure,

'CREATE INDEX [missing_index_' + CONVERT (varchar, mig.index_group_handle) + '_' + CONVERT (varchar, mid.index_handle)

+ '_' + LEFT (PARSENAME(mid.statement, 1), 32) + ']'

+ ' ON ' + mid.statement

+ ' (' + ISNULL (mid.equality_columns,'')

+ CASE WHEN mid.equality_columns IS NOT NULL AND mid.inequality_columns IS NOT NULL THEN ',' ELSE '' END

+ ISNULL (mid.inequality_columns, '')

+ ')'

+ ISNULL (' INCLUDE (' + mid.included_columns + ')', '') AS create_index_statement,

migs.*, mid.database_id, mid.[object_id]

FROM sys.dm_db_missing_index_groups mig

INNER JOIN sys.dm_db_missing_index_group_stats migs ON migs.group_handle = mig.index_group_handle

INNER JOIN sys.dm_db_missing_index_details mid ON mig.index_handle = mid.index_handle

WHERE migs.avg_total_user_cost * (migs.avg_user_impact / 100.0) * (migs.user_seeks + migs.user_scans) > 10

ORDER BY migs.avg_total_user_cost * migs.avg_user_impact * (migs.user_seeks + migs.user_scans) DESC

You'll want to run this after your server has been up and running a normal workload for a while. If this returns no results, that's good news and indicates that you're not missing any indexes that are obvious enough for the DMV to detect. If it does return some suggestions, even better: you just improved your server's perf with almost no work.

While to me this feature is so cool it almost seems magical, it does have a few limitations you should be aware of:

It's not as smart as the Database Engine Tuning Advisor. If you have identified a query that you know is expensive and needs some help, don't pass up DTA just because the missing index DMVs didn't have any suggestions. DTA might still be able to help.
The missing index DMVs don't take into account the overhead that new indexes can create (extra disk space, slight impact on insert/delete perf, etc). DTA does take this into account, however.
The "improvement_measure" column in this query's output is a rough indicator of the (estimated) improvement that might be seen if the index was created. This is a unitless number, and has meaning only relative the same number for other indexes. (It's a combination of the avg_total_user_cost, avg_user_impact, user_seeks, and user_scans columns in sys.dm_db_missing_index_group_stats.)
The missing index DMVs don't make recommendation about whether a proposed index should be clustered or nonclustered. This has workload-wide ramifications, while these DMVs focus only on the indexes that would benefit individual queries. (DTA can do this, however.)
Won't recommend partitioning.
It's possible that the DMVs may not recommend the ideal column order for multi-column indexes.
The DMV tracks information on no more than 500 missing indexes.

If you're a typical SQL user, you may not be using these DMVs yet. If you look around, though, there are a few places where they are in use. One is in the SP2 Performance Dashboard reports. Another is the Perf Stats Script that SQL PSS uses. And if you think the missing index DMVs are useful, check out this set of scripts that builds on the missing index DMVs to simulate an "auto create index" feature. Also, you should be aware there is similar missing index info output in the new XML showplan format in SQL 2005. If you are already focused on a poorly-performing query, I would start with the plan view of missing indexes (followed by DTA) rather than the DMVs.

SELECT from a view slower than "equivalent" SELECT from the base table

bartd — Tue, 15 Aug 2006 01:39:00 GMT

Sometime we get complaints that a query is slower than it could be because a filter isn’t pushed very deeply down into a plan. For example, consider this hypothetical poor performance scenario (my apologies in advance for the lack of normalization):

USE tempdb

IF OBJECT_ID ('Sales') IS NOT NULL DROP TABLE Sales

IF OBJECT_ID ('SalesSummary') IS NOT NULL DROP VIEW SalesSummary

CREATE TABLE Sales (

SalesPerson varchar(30) NOT NULL,

SalesAmount money NOT NULL, Comments char(200))

INSERT INTO Sales VALUES ('Green ', 19011.87, '')

INSERT INTO Sales VALUES ('Green', 2478.42, '')

INSERT INTO Sales VALUES ('Green ', 1975.11, '')

INSERT INTO Sales VALUES ('White', 3007.01, '')

INSERT INTO Sales VALUES ('White', 5312.44, '')

INSERT INTO Sales VALUES ('Brown', 843.20, '')

CREATE INDEX idx ON Sales (SalesPerson, SalesAmount)

CREATE VIEW SalesSummary AS

SELECT SalesPerson, SUM (SalesAmount) AS TotalSales

FROM Sales

GROUP BY SalesPerson

SET STATISTICS PROFILE ON

-- This query uses an index seek to retrieve only the rows

-- where SalesPerson = 'Green'

SELECT *

FROM SalesSummary

WHERE SalesPerson = 'Green'

-- This query uses an index scan, then filters the rows later.

SELECT *

FROM SalesSummary

WHERE SalesPerson LIKE 'Green'

SET STATISTICS PROFILE OFF

The first query filters on “SalesPerson = 'Green'”. The second filters on “SalesPerson LIKE 'Green'”. Here’s the first query’s plan:

|--Stream Aggregate(DEFINE:([Expr1004]=SUM([Sales].[SalesAmount]), [Sales].[SalesPerson]=ANY([Sales].[SalesPerson])))

|--Index Seek(OBJECT:([Sales].[idx]), SEEK:([Sales].[SalesPerson]='Green') ORDERED FORWARD)

You can see that the first thing the plan does is a very efficient index seek to narrow the set of rows down to those that pass the “SalesPerson = ‘Green’” filter. Then a Stream Aggregate operator computes the SUM(SalesAmount) expression for each SalesPerson returned by the index seek.

The second plan, though, scans every row in the table and computes the SUM for every SalesPerson. Only after it has scanned and aggregated every row does it filter out those values that don’t survive the LIKE predicate.

|--Filter(WHERE:([Sales].[SalesPerson] like 'Green'))

|--Stream Aggregate(GROUP BY:([Sales].[SalesPerson]) DEFINE:([Expr1004]=SUM([Sales].[SalesAmount])))

|--Index Scan(OBJECT:([Sales].[idx]), ORDERED FORWARD)

Here’s a general rule off thumb you can follow when looking for tuning opportunities in a query plan: for best performance, you usually want to push the most selective predicates as deeply as possible into the plan. If you do the most selective operation first, the remaining operators have fewer rows to process, and that means faster overall query execution. Based on this rule of thumb, the first plan here is clearly preferable from a performance perspective. So what gives? Why does the use of “LIKE” instead of “=” make SQL refuse to push the filter down? Even more baffling, why do you get the more efficient plan when you bypass the view and select directly from the table, even if you use "LIKE"? Run this and you’ll see what I mean:

SELECT SalesPerson, SUM (SalesAmount) AS TotalSales

FROM Sales

WHERE SalesPerson LIKE 'Green'

GROUP BY SalesPerson

This uses an efficient index seek-based plan, just like the first query.

There are several things going on here:

Views must behave like a table
The “LIKE” and “=” operators use subtly different rules for string matching
GROUP BY uses the same string comparison rules as “=” for the purposes of determining which rows end up in the same bucket

What I mean by “views must behave like a table” is that the output of a select from a view must be the same as what you could get by materializing the view (e.g. selecting it into a temp table), then querying the materialized view. In this case, the GROUP BY in the view could return a different total sales amount for a SalesPerson if SQL chose a plan that pushed a LIKE predicate below the Stream Aggregate. Here’s proof:

SELECT *

FROM SalesSummary

WHERE SalesPerson LIKE 'Green '

-- Query 1 output:

SalesPerson TotalSales

------------------------------ ---------------------

Green 23465.4000

SELECT SalesPerson, SUM (SalesAmount) AS TotalSales

FROM Sales

WHERE SalesPerson LIKE 'Green '

GROUP BY SalesPerson

-- Query 2 output:

SalesPerson TotalSales

------------------------------ ---------------------

Green 20986.9800

The first query selects from the view, while the second moves the view logic into the query and selects directly from the base table. Other than that, they are identical, yet the SUM(SalesAmount) calculation is different. Recall that I mentioned that “=” and “LIKE” have different string comparison semantics. In particular, LIKE considers trailing blanks in the right-hand operand to be significant, while the “=” operator ignores trailing blanks. For the set of three rows with group ID “Green”, only two will qualify for the filter “WHERE SalesPerson LIKE ‘Green ‘” because they have trailing blanks. The third “Green” row doesn’t have any trailing blanks and won’t survive the LIKE filter. When you push this LIKE filter below the aggregate, you end up SUMming a different set of rows. That’s not allowed if the GROUP BY is part of a view; if a WHERE clause applied to a view can change a property of a row instead of just filtering it out, it would mean that the view didn’t behave like a materialized table. Put another way, a filter on a view that includes a GROUP BY is only allowed to eliminate entire groups; it's not legal for it to eliminate some base rows in a group but retain others, changing the group's membership. It’s therefore by design that the slower scan-based plan is selected for the select from the view with LIKE. In contrast, the query from the view with the "SalesPerson='Green'" filter can be pushed because GROUP BY uses the same string comparison rules as the "=" operator. It's safe for the QO to assume that pushing the "=" filter below the Stream Aggregate will not change the view's semantics.

This isn’t just about trailing blanks – you can see the attached script for a couple of examples that demonstrate the exact same thing (pushing LIKE below an aggregate changes the output of the aggregate) for a couple of interesting non-blank characters. And it isn’t only about “LIKE” vs. “=”, either; this is just the example that was close at hand when I wrote this (we have a case open for this scenario right now).

Finally, be aware that derived tables (and CTEs) also provide the same guarantee. For example, note that this query selects from the base table but also does a full scan followed by filter, just like the above select from the view:

SELECT *

FROM (

SELECT SalesPerson, SUM (SalesAmount) AS TotalSales

FROM Sales

GROUP BY SalesPerson) AS t

WHERE SalesPerson LIKE 'Green'

So, to net out all of this: Generally you want your filters to be pushed deep into the query plan -- as deeply as possible. But when you’re selecting from a view, there will be some limits to what can be pushed. Some filters can’t be pushed beneath parts of the view without changing the view’s semantics, and that would break a contract that SQL is required to maintain.

UPDATE (2 March 2009): Fabiano Amorim pointed out that the Query #1 and Query #2 use the same plan on SQL 2008. He's right; a new performance optimization causes the LIKE predicate to be pushed below the GROUP BY's aggregate operator. I think this is actually a bug -- it does result in a faster plan, but it breaks the "views behave like tables" rule that SQL follows in all other cases. The general rule stands: not all predicates can be pushed below a view's GROUP BY, even in SQL 2008 with this fairly aggressive performance optimization. (And don't be surprised if this optimization gets removed in a future release. ;)

Limited Statistics Granularity

bartd — Tue, 25 Jul 2006 19:37:00 GMT

To set up this scenario, run the script below:

USE tempdb
GO
IF OBJECT_ID ('test1') IS NOT NULL DROP TABLE test1
GO
CREATE TABLE test1 (c1 tinyint, c2 smallint)
DECLARE @x int
DECLARE @msg varchar(1000)
SET @x = 1
SET NOCOUNT ON
BEGIN TRAN
WHILE (@x <= 1000000)
BEGIN
INSERT INTO test1 (c1, c2) VALUES (@x % 255, CASE WHEN @x % 1000 = 500 THEN 1000 ELSE @x % 1000 END)
IF @x % 5000 = 0
BEGIN
    COMMIT TRAN
    BEGIN TRAN
    SET @msg = 'Inserted ' + CONVERT(varchar(20), @x) + ' rows ...'
    RAISERROR (@msg, 0, 1) WITH NOWAIT
END
SET @x = @x + 1
END
WHILE (@@TRANCOUNT > 0) COMMIT TRAN
SET NOCOUNT OFF
GO
CREATE INDEX idx1 ON test1 (c2)
UPDATE STATISTICS test1 WITH FULLSCAN
DBCC SHRINKDATABASE (db2)
-- DBCC SHOW_STATISTICS ('test1', 'idx1')

GO

The hypothetical scenario here is that your users complain that the first of the two queries below runs very slowly. You find that if you force SQL to use index [idx1] with an index hint, the query executes much more quickly. But why should you change your app to compensate for what seems to be a SQL Server bug? :)

USE db2
GO
SET STATISTICS PROFILE ON
SET STATISTICS TIME ON
-- Query #1 (slow)
SELECT c1, c2 FROM test1 WHERE c2 = 500

-- Query #2 (fast)
SELECT c1, c2 FROM test1 WITH (INDEX = idx1) WHERE c2 = 500

SET STATISTICS TIME OFF
SET STATISTICS PROFILE OFF

GO

To me, this suggests two questions: 1) Why isn't the fast plan chosen by default? 2) Are there any possible solutions that don't require modifying code to supply an index hint?

Despite appearances, this actually isn't a bug. Here's the explanation:

This table demonstrates uneven data distribution. The [c2] column holds values ranging from 0 to 999. There are exactly 1000 rows with the same [c2] value for each of the integers between 0 and 999. However, the value 500 is an anomaly; there are 0 rows with this value. Run the following query to show the number of rows with each [c2] value and you'll see that there are none with [c2]=500:

SELECT c2, COUNT(*) AS cnt FROM test1 GROUP BY c2

c2     cnt
------ -----------
...
498    1000
499    1000
501    1000
502    1000
...

When the server builds a histogram for the statistics on a column, it attempts to intelligently select range endpoints so that a given step of the histogram represents a range of values with similar density. If you run DBCC SHOW_STATISTICS on the index you can see the histogram:

-- DBCC SHOW_STATISTICS ('test1', 'idx1')

RANGE_HI_KEY RANGE_ROWS EQ_ROWS DISTINCT_RANGE_ROWS AVG_RANGE_ROWS
------------ ---------- ------- ------------------- --------
0            0.0        1000.0 0                   0.0
499          498000.0   1000.0 498                 1000.0
503          2000.0     1000.0 2                   1000.0
1000         496000.0   1000.0 496                 1000.0

RANGE_HI_KEY is the value that sets the upper bound for the range of values represented in each histogram step. The lower bound is RANGE_HI_KEY+1 for the preceding step. For example, consider this row:

499 498000.0 1000.0 498 1000.0

This indicates that there are 498000 rows (RANGE_ROWS) with a value between 1 and 498, inclusive. There are 1000 rows with the exact value of 499 (EQ_ROWS). AVG_RANGE_ROWS tells us that the typical value that falls within the range 1-498 shows up in 1000 rows. Now consider the next step in the histogram:

RANGE_HI_KEY RANGE_ROWS EQ_ROWS DISTINCT_RANGE_ROWS AVG_RANGE_ROWS
------------ ---------- ------- ------------------- --------
503 2000.0 1000.0 2 1000.0

This indicates that there are 2000 rows (RANGE_ROWS) with a value between 500 and 502, inclusive; and there are 1000 rows (EQ_ROWS) with the exact value of 503. The typical c2 value in the range 500-502 shows up average of 1000 times in the table. Recall what we know about the actual number of rows in this range:

c2     cnt
------ -----------
...
499    1000
501    1000
502    1000
503    1000
...

This histogram step describes its range of c2 values exactly. What the histogram doesn't tell us, however, is exactly how many times a particular value in the 500-502 range appears. Remember that each histogram step only summarizes the values within a given range; it doesn't tell you exactly how many of each particular value there are (that is, unless the table's domain is small enough for every distinct value to have its own step in the histogram). SQL knows it needs to search for rows where [c2]=500, so it locates this step in the statistics' histogram, finds that the typical value in this range shows up in 1000 rows, and uses this as its rowcount estimate.

This is a key point: if a value being searched for falls in the middle of a histogram step, the AVG_RANGE_ROWS for that step is used to estimate the number of matches that will be found. SQL is always optimistic and assumes that the value it is searching for is actually one of the range rows. In this case that assumption is incorrect. The QP ends up overestimating the number of rows that will be returned and choosing a table scan when an index seek would have been more efficient.

So if this isn't a bug, what could you do to fix the problem? One solution would be to make the nonclustered index a covering index. Alternately, you could make [idx1] a clustered index. And, if you're using SQL 2005, you could use a plan guide with a USE PLAN hint to force the fast plan without changing the query text. All of these solutions should cause SQL to select the fast index seek plan. The bad cardinality estimate will still be there, but the perf problem won't.

Why a bad plan isn't necessarily a bug

bartd — Tue, 25 Jul 2006 18:54:00 GMT

Everyone that has worked with databases for long enough has run into situations where the query optimizer doesn't select the best possible plan. You may find that you can force SQL to use an index, choose a different join algorithm or join order, or use some other query hint to get a much faster plan. But you're reluctant to try to push through a change in your app in order to compensate for what seems to be a clear SQL Server bug.

We see lots of instances where people have the expectation that if SQL chose a demonstrably inefficient plan, it must be a bug. Unfortunately, it's not quite that simple. At the end of the day, the query processor is just a guessing engine that tries to pick the best possible plan based off statistics that summarize the data in a table. There are some significant limits to how accurately the query optimizer can model the real world when it is costing plans. For starters, the information available to it in statistics only summarizes column data at a fairly high level; it doesn't tell the QO all that there is to know about the distribution of values in a column, or the relationships between data in two or more columns. And there's a tradeoff between the complexity of the QO's modeling rules and the amount of time that it takes to cost the hundreds of thousands of plan possibilities that exist for even modestly complex queries. Even if the QO had full knowledge of everything there was to know about your data, trying to take all of those facts into account when compiling plans would lead to more situations where the optimizer spent more time trying to find that perfect plan than it would have taken to actually execute the query. There are definitely QO bugs out there, but the cases where the QO picks a less-than-ideal plan are not necessarily bugs; some are simply the result of these limits. In fact, our experience is that a majority of bad plan issues fall into the "unfortunate, but by design" bucket.

Dubious? That's OK -- I'll try to illustrate some of the more common "by design" situations where you might see poor plan selections here. Even if you end up taking the perfectly defensible position that the QO should be able to handle some of these situations more gracefully, it'll still benefit you to be able to recognize them.