Hash Warning SQL Profiler Event

queryproc — Fri, 02 Feb 2007 06:18:00 GMT

One of the less well-known warning events that is logged to SQL Profiler trace is the Hash Warning event. Hash Warning events are fired when a hash recursion or hash bailout has occurred during a hashing operation. Both of these situations are less than desirable, as they mean that a Hash Join or Hash Aggregate has run out of memory and been forced to spill information to disk during query execution. When a hashing operation spills to disk, this almost always results in slower query performance and can cause space consumption increase in tempdb.

Note that the Hash Warning event needs to be explicitly enabled within SQL Profiler. It is not one of the “default” set of events. More info on SQL Profiler can be found here

What can be done if you see a lot of Hash Warning events? The recommended actions are:

· Make sure that statistics exist on the columns that are involved in the hashing operation. Without statistics, the hashing operation has no way to know how much memory to pre-allocate.

· Even if statistics do exist, try updating them, as this can give more current information to the hashing operation when it decides how much memory to allocate.

· Try using a different type of join (this can be done by hinting OPTION(MERGE JOIN) or OPTION(LOOP JOIN)). Note that forcing a join type does not necessarily guarantee a better execution plan.

· If all of these fail, you can increase the available memory on the computer.

A sample of what you will see in the profiler would look something like the following. Note the batch starting, followed by a number of Hash Warning events prior to batch completion. Also, the SPID that is causing the events will be recorded

EventClass	StartTime	SPID
SQL:BatchStarting	2007-02-01 18:53:34.703	51
Hash Warning	2007-02-01 18:53:48.267	51
Hash Warning	2007-02-01 18:53:48.283	51
Hash Warning	2007-02-01 18:53:50.097	51
Hash Warning	2007-02-01 18:54:05.300	51
SQL:BatchCompleted	2007-02-01 18:54:19.130	51

- Steve Herbert

SQL Server Query Execution

What's this cost?

queryproc — Thu, 12 Oct 2006 05:27:00 GMT

Oftentimes when people include actual execution plan (Ctrl-M, see previous posts for a good primer on execution plans) while executing a batch in SQL Server Management Studio, and they see this "Query cost (relative to the batch)" thing on top of each query in the batch, they start to ask: What does this mean? But when I run this batch, the first query runs a faster than the second, and yet this crazy SQL Server says the second has a higher cost, what are they talking about?

Before I attempt an explanation, let me quickly outline an example to illustrate this. First, let's create a table with some rows to experiment with:

Use tempdb
Go
If Object_ID('BigT', 'U') Is Not Null
Drop Table BigT
Go
Create Table BigT(pk Int Identity Primary Key Clustered, string Varchar(1000), number Int);
Go
-- Insert 10,000 rows
WITH Digits(D) AS
(
Select 0 Union All Select 1 Union All Select 2 Union All Select 3 Union All Select 4 Union All
Select 5 Union All Select 6 Union All Select 7 Union All Select 8 Union All Select 9
)
Insert Into BigT
Select 'Some text', A.D * 1000 + B.D * 100 + C.D * 10 + D.D
From Digits A Cross Join Digits B Cross Join Digits C Cross Join Digits D
Go

Now, with Include Actual Execution Plan on, we can run the following two-query batch:

Select string, Max(number) From BigT Group By string Order By Max(number)
Select A.string, B.string From BigT A Join BigT B on A.number = B.number * 2

On my machine, I get the attached plan. The main thing to notice here is that the first query has a query cost of 4% (relative to the batch), and the second one has 96% cost. Actually, if you want the numerical basis for these percentages and you're running this experiment yourself (you'll probably have different numbers depending on your machine), just hover over the root SELECT operator in the plans. I have Estimated Subtree Cost of 0.135437 for the first SELECT and 3.154 for the second SELECT, which have the ratio ~4:96. Notice the lack of units for these costs: they're not in seconds, MB or anything.

The reason these costs exist is because of the query optimization SQL Server does: it does cost-based optimization, which means that the optimizer formulates a lot of different ways to execute the query, assigns a cost to each of these alternatives, and chooses the one with the least cost. The cost tagged on each alternative is heurestically calculated, and is supposed to roughly reflect the amount of processing and I/O this alternative is going to take. For example, based on the statistical information we automatically collect on the number column, we estimated that we'll have 5000.5 rows coming out of the join in the second query (a really good guess in this case, the join produces 5000 rows), and based on this we assigned a cost 1.8 on the Merge Join operator in my machine. This number, again, is unit-less and is meant only for comparison purposes against other alternatives.

So when you get the execution plan in the end, we display this relative costs of queries in a batch, and for different operators within the same query, for informational purposes. They are not meant as accurate representations of query run-times. The actual run-time of a query depends on many things: whether the buffers are warm (i.e. the contents required by the query are cached by the server), what other queries are running on the server, locking, etc. Also, the cost estimation is a heurestic process based on statistical sampling and various best-effort guesses, and could easily go wrong especially for large queries.

For example, setting Statistics Time on:

Set Statistics Time On

And running these queries again I get a run time of 15 ms for the first query and 70 ms for the second. The second did take longer, but not a 96:4 ratio. Then again, these are really short times and hardly precise. For example, a second run yielded run times of 16 ms & 111 ms.

Mostafa Elhemali

Tips, Tricks, and Advice from the SQL Server Query Processing Team : Query plan

Hash Warning SQL Profiler Event

What's this cost?