Craig Freedman's SQL Server Blog : Aggregation

Query Processing Presentation

craigfr — Thu, 15 May 2008 19:39:00 GMT

Last week, I had the opportunity to talk to the New England SQL Server Users Group. I would like to thank the group for inviting me, Adam Machanic for organizing the event, and Red Gate for sponsoring it. My talk was an introduction to query processing, query execution, and query plans in SQL Server. I've had a request for the slides, so here they are. Enjoy!

Ranking Functions: RANK, DENSE_RANK, and NTILE

craigfr — Tue, 01 Apr 2008 00:04:00 GMT

In my previous post, I discussed the ROW_NUMBER ranking function which was introduced in SQL Server 2005. In this post, I'll take a look at the other ranking functions - RANK, DENSE_RANK, and NTILE. Let's begin with RANK and DENSE_RANK. These functions are similar - both in functionality and implementation - to ROW_NUMBER. The difference is that while the ROW_NUMBER function assigns a unique (and ascending) value to each row without regard for ties in the ORDER BY values, the RANK and DENSE_RANK functions assign the same value to rows that have the same ORDER BY value. The difference between the RANK and DENSE_RANK functions is in how values are assigned to rows following a tie. The easiest way to illustrate the difference between all of these functions is with a simple example:

CREATE TABLE T (PK INT IDENTITY, A INT, B INT, C INT)
CREATE UNIQUE CLUSTERED INDEX TPK ON T(PK)

INSERT T VALUES (0, 1, 6)
INSERT T VALUES (0, 1, 4)
INSERT T VALUES (0, 3, 2)
INSERT T VALUES (0, 3, 0)
INSERT T VALUES (1, 0, 7)
INSERT T VALUES (1, 0, 5)
INSERT T VALUES (0, 2, 3)
INSERT T VALUES (0, 2, 1)

SELECT *,
    ROW_NUMBER() OVER (ORDER BY B) AS RowNumber,
   RANK() OVER (ORDER BY B) AS Rank,
   DENSE_RANK() OVER (ORDER BY B) AS DenseRank
FROM T

PK    A     B     C     RowNumber  Rank       DenseRank
----- ----- ----- ----- ---------- ---------- ----------
5     1     0     7     1          1          1
6     1     0     5     2          1          1
1     0     1     6     3          3          2
2     0     1     4     4          3          2
7     0     2     3     5          5          3
8     0     2     1     6          5          3
3     0     3     2     7          7          4
4     0     3     0     8          7          4

Notice how the ROW_NUMBER function ignores the duplicate values for column B and assigns the unique integers from 1 to 8 to the 8 rows while the RANK and DENSE_RANK functions assigns the same value to each of the pairs of duplicate rows. Moreover, notice how the RANK function counts the duplicate rows even while it assigns the same value to each duplicate row whereas the DENSE_RANK function does not count the duplicate rows. For example, both the RANK and DENSE_RANK functions assign a rank of 1 to the first two rows, but the RANK function assigns a rank of 3 to the third row - as it is the third row - while the DENSE_RANK function assigns a rank of 2 to the third row - as it contains the second distinct value for column B. Note that the maximum value returned by the DENSE_RANK function is exactly equal to the number of distinct values in column B.

The query plan for computing all three of these functions is essentially the same and, in fact, as long as all three of the functions share the same PARTITION BY and ORDER BY clauses, a single sequence project operator can compute all three functions simultaneously:

|--Sequence Project(DEFINE:([Expr1003]=row_number, [Expr1004]=rank, [Expr1005]=dense_rank))
       |--Segment [GROUP BY:([T].[B])]
            |--Segment [GROUP BY:()]
                 |--Sort(ORDER BY:([T].[B] ASC))
                      |--Clustered Index Scan(OBJECT:([T].[TPK]))

The only real difference between this plan and a plan that computes only the ROW_NUMBER function is the presence of the extra segment operator that groups by column B. This segment operator detects ties and notifies the sequence project operator when to output the same value and when to output a new value for the RANK and DENSE_RANK functions.

All of the issues from my previous post on ROW_NUMBER (such as the use of an index to avoid the sort or the impact of mixing ranking functions with different PARTITION BY and/or ORDER BY clauses) apply equally well to the RANK and DENSE_RANK functions so I will not repeat them here.

Now let's take a look at the NTILE function. This function breaks an input set down into N equal sized groups. To determine how many rows belong in each group, SQL Server must first determine the total number of rows in the input set. If the NTILE function includes a PARTITION BY clause, SQL Server must compute the number of rows in each partition separately. Once we know the number of rows in each partition, we can write the NTILE function as

NTILE(N) := (N * (ROW_NUMBER() - 1) / COUNT(*)) + 1

where COUNT(*) is the number of rows in each partition. We need the -1 and +1 in this equation as the ROW_NUMBER and NTILE functions are 1 not 0 based.

We already know how to compute the ROW_NUMBER function, so the real question is how to compute the COUNT(*) function. It turns out that SQL Server supports use of the OVER clause that we've already seen with the ranking functions with nearly all aggregate functions as well. This type of aggregate function is sometimes referred to as a "window aggregate function." With a window aggregate function, instead of returning a single row per group, SQL Server returns the same aggregate result with each row in the group. Let's look at an example:

SELECT *, COUNT(*) OVER (PARTITION BY A) AS Cnt FROM T

PK    A     B     C     Cnt
----- ----- ----- ----- ----------
1     0     1     6     6
2     0     1     4     6
3     0     3     2     6
4     0     3     0     6
7     0     2     3     6
8     0     2     1     6
5     1     0     7     2
6     1     0     5     2

Notice that unlike a scalar aggregate query which can return only aggregate functions or an aggregate query with a GROUP BY clause which can return only columns from the GROUP BY clause and aggregate functions, this query can return any columns from the table. Moreover, since each row in the group gets the same result, the ORDER BY clause is not supported for this scenario. Now let's take a look at the plan for this query:

While this plan is certainly more complex than a typical aggregation query plan, it's actually fairly straightforward. The sort and segment operators merely break the rows into groups in the same way as a stream aggregate. The table spool above the segment operator is a special type of common subexpression spool known as a segment spool. This spool operator makes a copy of all rows in one group and then, when the segment operator indicates the start of a new group, it returns a single row to the topmost nested loops join which immediately executes its inner side. At this point, the table spool replays the rows from the current group and the stream aggregate counts them. Finally, the stream aggregate returns the count to the bottommost nested loops join which joins this count with the rows from the current group - the table spool replays these rows a second time.

Note that this plan needs the segment spool because there is no way to know a priori how many rows are in the same group with the current row without scanning the entire group. But after scanning the entire group and determining the count, we need some way to go back and apply the count to each of the original rows. The spool makes it possible to go back. A typical aggregate query with a GROUP BY clause does not have this issue because after computing the aggregate function (be it a COUNT or any other function), it simply outputs the result without go back to the original rows.

For an example of a different query that yields a very similar query plan involving a segment spool, check out my discussion of subqueries on pages 171 through 173 of Inside Microsoft SQL Server 2005: Query Tuning and Optimization.

It's now a simple matter to see how to compute the NTILE function. For example, the following query groups the rows by column A and then for each group computes which rows are below the median (denoted by a 1) and which rows are above the median (denoted by a 2):

SELECT *, NTILE(2) OVER (PARTITION BY A ORDER BY B) AS NTile FROM T

Here are results for this query:

PK    A     B     C     NTile
----- ----- ----- ----- ------
1     0     1     6     1
2     0     1     4     1
7     0     2     3     1
8     0     2     1     2
3     0     3     2     2
4     0     3     0     2
5     1     0     7     1
6     1     0     5     2

And here is the plan for this query:

This plan is basically the same as the plan for the previous COUNT(*) query. The only difference is the addition of the extra segment and sequence project operators which compute the NTILE function by computing ROW_NUMBER and applying the formula I provided above. Using this formula, we could also compute NTILE manually as follows:

SELECT *, (2*(RowNumber-1)/Cnt)+1 AS MyNTile
FROM
    (
    SELECT *,
        ROW_NUMBER() OVER (PARTITION BY A ORDER BY B) AS RowNumber,
        COUNT(*) OVER (PARTITION BY A ) AS Cnt,
        NTILE(2) OVER (PARTITION BY A ORDER BY B) AS NTile
    FROM T
    ) TSeq

Unfortunately, the optimizer does not recognize that it can use the same plan fragment to compute both the ranking functions and the COUNT(*) window aggregate function and ends up generating the following inefficient plan which computes the COUNT(*) twice:

I'll leave parsing this query plan as an exercise. Enjoy!

Partial Aggregation

craigfr — Fri, 18 Jan 2008 22:12:00 GMT

In some of my past posts, I've discussed how SQL Server implements aggregation including the stream aggregate and hash aggregate operators. I also used hash aggregation as an initial example in my introductory post on parallel query execution. In this post, I'll look at a partial aggregation. Partial aggregation is a technique that SQL Server uses to optimize parallel aggregation. Before I begin, I just want to note that I also discuss partial aggregation in Inside Microsoft SQL Server 2005: Query Tuning and Optimization. (See the bottom of page 187.)

Let's begin with a simple scalar aggregation example. Recall that a scalar aggregate is an aggregate without a GROUP BY clause. A scalar aggregate always produces a single output row.

CREATE TABLE T (A INT, B INT IDENTITY, C INT, D INT)
CREATE CLUSTERED INDEX TA ON T(A)

SELECT COUNT(*) FROM T

Not surprisingly, this query yields a trivial stream aggregate plan:

|--Compute Scalar(DEFINE:([Expr1004]=CONVERT_IMPLICIT(int,[Expr1005],0)))
|--Stream Aggregate(DEFINE:([Expr1005]=Count(*)))
|--Clustered Index Scan(OBJECT:([T].[TA]))

Now, suppose that we want to parallelize this query. Because this query outputs a single row, we cannot simply put an exchange (i.e., parallelism operator) at the root of the plan and divide the work of counting among multiple threads. Such a strategy would yield one output row per thread which is clearly not the correct result.

Alternatively, we could put a gather streams exchange between the stream aggregate and clustered index scan operators. This strategy would permit us to use a parallel scan while still counting in a single thread and outputting a single row. However, we would end up moving every row from the scan through the exchange - a rather costly operation. Thus, this option, while valid, would not yield nearly the performance we'd like to see.

Fortunately, there is a third option. We can use a parallel scan, divide the work of counting among multiple threads (as in the first option), use an exchange to gather the per-thread counts into a single thread, and finally sum the per-thread counts to generate the grand total. This strategy is much more efficient as we need only move a single row per thread through the exchange. To get the optimizer to generate this plan, we need to add lots of data to the table. To save time, I'm going to use UPDATE STATISTICS to trick the optimizer into thinking that we've added rows to the table:

UPDATE STATISTICS T WITH ROWCOUNT = 1000000, PAGECOUNT = 100000
SELECT COUNT(*) FROM T OPTION (RECOMPILE)

We need the RECOMPILE query hint to force the optimizer to generate a new plan with the new statistics. Here is the plan we get:

|--Compute Scalar(DEFINE:([Expr1004]=CONVERT_IMPLICIT(int,[globalagg1006],0)))
       |--Stream Aggregate(DEFINE:([globalagg1006]=SUM([partialagg1005])))
            |--Parallelism(Gather Streams)
                 |--Stream Aggregate(DEFINE:([partialagg1005]=Count(*)))
                      |--Clustered Index Scan(OBJECT:([T].[TA]))

We call the bottommost aggregate operator a partial aggregate because it computes only part of the result. We also sometimes refer to this operator as a local aggregate because it computes the portion of the result that is local to the thread where it executes. We refer to the topmost aggregate as a global aggregate because it computes the full result.

SQL Server is able to use partial aggregation for most aggregate functions including the standard built-ins: COUNT, SUM, AVG, MIN, and MAX. While partial aggregation is necessary to parallelize scalar aggregates, it is also useful even for aggregates with a GROUP BY clause. Whether the optimizer chooses to use partial aggregation depends on the number of unique groups and the size of these groups. If the optimizer anticipates that a query will generate few large groups (such as in the scalar aggregation case), it will use partial aggregation. However, if the optimizer expects that a query will generate many small groups, it may choose to use a single level aggregation. With small groups, a partial aggregate cannot reduce the number of rows significantly and merely adds overhead to the query. Moreover, with many groups it is easy to parallelize the aggregation by hashing on the GROUP BY keys and distributing different groups to different threads.

Let's see how the optimizer makes this choice. Column B in our example has the IDENTITY property. Although we have no real data, this property is sufficient to trick the optimizer into concluding that this column is mostly unique. (Without a unique index, the optimizer cannot be certain that the column is indeed unique and must assume that it is not.) Suppose we aggregate on this column:

SELECT COUNT(*) FROM T GROUP BY B

|--Parallelism(Gather Streams)
       |--Compute Scalar(DEFINE:([Expr1004]=CONVERT_IMPLICIT(int,[Expr1007],0)))
            |--Hash Match(Aggregate, HASH:([T].[B]) DEFINE:([Expr1007]=COUNT(*)))
                 |--Parallelism(Repartition Streams, Hash Partitioning, PARTITION COLUMNS:([T].[B]))
                      |--Clustered Index Scan(OBJECT:([T].[TA]))

Notice that this query yields a normal single level, albeit parallel, aggregate operator. Now suppose we aggregate on column C which does not have the IDENTITY property:

SELECT COUNT(*) FROM T GROUP BY C

|--Compute Scalar(DEFINE:([Expr1004]=CONVERT_IMPLICIT(int,[globalagg1006],0)))
       |--Parallelism(Gather Streams)
            |--Stream Aggregate(GROUP BY:([T].[C]) DEFINE:([globalagg1006]=SUM([partialagg1005])))
                 |--Sort(ORDER BY:([T].[C] ASC))
                      |--Parallelism(Repartition Streams, Hash Partitioning, PARTITION COLUMNS:([T].[C]))
                           |--Hash Match(Partial Aggregate, HASH:([T].[C]), RESIDUAL:([T].[C] = [T].[C]) DEFINE:([partialagg1005]=COUNT(*)))
                                |--Clustered Index Scan(OBJECT:([T].[TA]))

This time we get a partial aggregate. Also, observe that the partial aggregate is a hash aggregate while the global aggregate is a stream aggregate. The optimizer is free to choose either physical aggregation operator (stream or hash) for the partial and global aggregates in a plan with partial aggregation. The decision of which operator to use is cost based.

Finally, it is worth noting that, while a stream aggregate behaves identically whether it is computing a partial or global aggregate, a partial hash aggregate does differ slightly from a normal hash aggregate. First, a partial hash aggregate requests only a fixed minimal memory grant as it presumes that it will be computing a relatively small number of groups. Second, a partial hash aggregate never spills rows to tempdb. If a partial hash aggregate runs out of memory, it simply stops aggregating and begins returning non-aggregated rows. This behavior is safe since the global aggregate will always compute the correct final results regardless of what the partial aggregate does. The partial aggregate is merely a performance optimization and the goal is to prevent it from stealing resources from the global aggregate.

GROUPING SETS in SQL Server 2008

craigfr — Thu, 11 Oct 2007 19:13:00 GMT

In my last two posts, I gave examples of aggregation WITH ROLLUP and CUBE. SQL Server 2008 continues to support this syntax, but also introduces new more powerful ANSI SQL 2006 compliant syntax. In this post, I'll give an overview of the changes.

First, let's see how we rewrite simple WITH ROLLUP and CUBE queries using the new syntax. I'll use the same schema and queries as in my previous posts:

CREATE TABLE Sales (EmpId INT, Yr INT, Sales MONEY)
INSERT Sales VALUES(1, 2005, 12000)
INSERT Sales VALUES(1, 2006, 18000)
INSERT Sales VALUES(1, 2007, 25000)
INSERT Sales VALUES(2, 2005, 15000)
INSERT Sales VALUES(2, 2006, 6000)
INSERT Sales VALUES(3, 2006, 20000)
INSERT Sales VALUES(3, 2007, 24000)

SELECT EmpId, Yr, SUM(Sales) AS Sales
FROM Sales
GROUP BY EmpId, Yr WITH ROLLUP

SELECT EmpId, Yr, SUM(Sales) AS Sales
FROM Sales
GROUP BY EmpId, Yr WITH CUBE

We can rewrite these two queries using the new syntax as:

SELECT EmpId, Yr, SUM(Sales) AS Sales
FROM Sales
GROUP BY ROLLUP(EmpId, Yr)

SELECT EmpId, Yr, SUM(Sales) AS Sales
FROM Sales
GROUP BY CUBE(EmpId, Yr)

These new queries are semantically equivalent to and use the same query plans as the original queries. Note that the new ROLLUP and CUBE syntax is only available in compatibility level 100. The more general GROUPING SETS syntax, which I will discuss next, is also available in earlier compatibility levels.

The new GROUPING SETS syntax is considerably more powerful. It allows us to specify precisely which aggregations we want to compute. As the following table illustrates, our simple two dimensional schema has a total of only four possible aggregations:

		Yr
		2005	2006	2007	ALL
EmpId	1	GROUP BY (EmpId, Yr)			GROUP BY (EmpId)
	2
	3
	ALL	GROUP BY (Yr)			GROUP BY ()

ROLLUP and CUBE are just shorthand for two common usages of GROUPING SETS. We can express the above ROLLUP query as:

SELECT EmpId, Yr, SUM(Sales) AS Sales
FROM Sales
GROUP BY GROUPING SETS((EmpId, Yr), (EmpId), ())

EmpId       Yr          Sales
----------- ----------- ---------------------
1           2005        12000.00
1           2006        18000.00
1           2007        25000.00
1           NULL        55000.00
2           2005        15000.00
2           2006        6000.00
2           NULL        21000.00
3           2006        20000.00
3           2007        24000.00
3           NULL        44000.00
NULL        NULL        120000.00

This query explicitly asks SQL Server to aggregate sales by employee and year, to aggregate by employee only, and to compute the total for all employees for all years. The () syntax with no GROUP BY columns denotes the total. Similarly, we can express the above CUBE query by asking SQL Server to compute all possible aggregate combinations:

SELECT EmpId, Yr, SUM(Sales) AS Sales
FROM Sales
GROUP BY GROUPING SETS((EmpId, Yr), (EmpId), (Yr), ())

EmpId       Yr          Sales
----------- ----------- ---------------------
1           2005        12000.00
2           2005        15000.00
NULL        2005        27000.00
1           2006        18000.00
2           2006        6000.00
3           2006        20000.00
NULL        2006        44000.00
1           2007        25000.00
3           2007        24000.00
NULL        2007        49000.00
NULL        NULL        120000.00
1           NULL        55000.00
2           NULL        21000.00
3           NULL        44000.00

We can also use GROUPING SETS to compute other results. For example, we can perform a partial rollup aggregating sales by employee and year and by employee only but without computing the total for all employees for all years:

SELECT EmpId, Yr, SUM(Sales) AS Sales
FROM Sales
GROUP BY GROUPING SETS((EmpId, Yr), (EmpId))

EmpId       Yr          Sales
----------- ----------- ---------------------
1           2005        12000.00
1           2006        18000.00
1           2007        25000.00
1           NULL        55000.00
2           2005        15000.00
2           2006        6000.00
2           NULL        21000.00
3           2006        20000.00
3           2007        24000.00
3           NULL        44000.00

We can skip certain rollup levels. For example, we can compute the total sales by employee and year and the total sales for all employees and all years without computing any of the intermediate results:

SELECT EmpId, Yr, SUM(Sales) AS Sales
FROM Sales
GROUP BY GROUPING SETS((EmpId, Yr), ())

EmpId       Yr          Sales
----------- ----------- ---------------------
1           2005        12000.00
1           2006        18000.00
1           2007        25000.00
2           2005        15000.00
2           2006        6000.00
3           2006        20000.00
3           2007        24000.00
NULL        NULL        120000.00

We can even compute multiple unrelated aggregations along disparate dimensions. For example, we can compute the total sales by employee and the total sales by year:

SELECT EmpId, Yr, SUM(Sales) AS Sales
FROM Sales
GROUP BY GROUPING SETS((EmpId), (Yr))

EmpId       Yr          Sales
----------- ----------- ---------------------
NULL        2005        27000.00
NULL        2006        44000.00
NULL        2007        49000.00
1           NULL        55000.00
2           NULL        21000.00
3           NULL        44000.00

Note that we could also write GROUPING SETS (EmpId, Yr) without the extra set of parenthesis, but the extra parenthesis make the intent of the query more explicit and clearly differentiate the previous query from the following query which just performs a normal aggregation by employee and year:

SELECT EmpId, Yr, SUM(Sales) AS Sales
FROM Sales
GROUP BY GROUPING SETS((EmpId, Yr))

EmpId       Yr          Sales
----------- ----------- ---------------------
1           2005        12000.00
2           2005        15000.00
1           2006        18000.00
2           2006        6000.00
3           2006        20000.00
1           2007        25000.00
3           2007        24000.00

Here are some additional points worth noting about the GROUPING SETS syntax:

As with any other aggregation query, if a column appears in the SELECT list and is not part of an aggregate function, it must appear somewhere in the GROUP BY clause. Thus, the following is not valid:

SELECT EmpId, Yr, SUM(Sales) AS Sales
FROM Sales
GROUP BY GROUPING SETS((EmpId), ())

Msg 8120, Level 16, State 1, Line 1
Column 'Sales.Yr' is invalid in the select list because it is not contained in either an aggregate function or the GROUP BY clause.

The order of the columns within each GROUPING SET and the order of the GROUPING SETS does not matter. So both of the following queries compute the same CUBE although the order that the rows are output differs:

SELECT EmpId, Yr, SUM(Sales) AS Sales
FROM Sales
GROUP BY GROUPING SETS((EmpId, Yr), (EmpId), (Yr), ())

SELECT EmpId, Yr, SUM(Sales) AS Sales
FROM Sales
GROUP BY GROUPING SETS((), (Yr), (EmpId), (Yr, EmpId))

If the order that the rows are output matters, use an explicit ORDER BY clause to enforce that order.

We can nest CUBE and ROLLUP within a GROUPING SETS clause as shorthand for expressing more complex GROUPING SETS. This shorthand is most useful when we have more than three dimensions in our schema. For example, suppose we add a month column to our sales table:

CREATE TABLE Sales (EmpId INT, Month INT, Yr INT, Sales MONEY)

Now, suppose we want to compute sales for each employee by month and year, by year, and total. We could write out all of the GROUPING SETS explicitly:

SELECT EmpId, Month, Yr, SUM(Sales) AS Sales
FROM Sales
GROUP BY GROUPING SETS((EmpId, Yr, Month), (EmpId, Yr), (EmpId))

Or we can use ROLLUP to simplify the query:

SELECT EmpId, Month, Yr, SUM(Sales) AS Sales
FROM Sales
GROUP BY GROUPING SETS((EmpId, ROLLUP(Yr, Month)))

Note that once again the correct use of parenthesis is critical. If we omit one set of parenthesis from the above query, the meaning changes significantly and we end up separately aggregating by employee and then computing the year and month ROLLUP for all employees.

The new GROUPING SETS syntax is available in all of SQL Server 2008 Community Technology Preview (CTP) releases.

Aggregation WITH CUBE

craigfr — Thu, 27 Sep 2007 21:30:00 GMT

In my last post, I wrote about how aggregation WITH ROLLUP works. In this post, I will discuss how aggregation WITH CUBE works. Like the WITH ROLLUP clause, the WITH CUBE clause permits us to compute multiple "levels" of aggregation in a single statement. To understand the difference between these two clauses, let's look at an example. We'll use the same fictitious sales data from last week's example.

CREATE TABLE Sales (EmpId INT, Yr INT, Sales MONEY)
INSERT Sales VALUES(1, 2005, 12000)
INSERT Sales VALUES(1, 2006, 18000)
INSERT Sales VALUES(1, 2007, 25000)
INSERT Sales VALUES(2, 2005, 15000)
INSERT Sales VALUES(2, 2006, 6000)
INSERT Sales VALUES(3, 2006, 20000)
INSERT Sales VALUES(3, 2007, 24000)

Consider the following query from last week:

SELECT EmpId, Yr, SUM(Sales) AS Sales
FROM Sales
GROUP BY EmpId, Yr WITH ROLLUP

It will be easier to see what is happening if we pivot the sales data:

		Yr
		2005	2006	2007	ALL
EmpId	1	12000.00	18000.00	25000.00	55000.00
	2	15000.00	6000.00		21000.00
	3		20000.00	24000.00	44000.00
	ALL				120000.00

The table clearly shows that the WITH ROLLUP clause computes the total for each employee for all years and the grand total for all employees and all years. The query does not compute the totals for each year for all employees. Moreover, the order of the columns in the GROUP BY clause determines in which order the data is totaled.

Now let's repeat the same query but replace the WITH ROLLUP clause with a WITH CUBE clause:

SELECT EmpId, Yr, SUM(Sales) AS Sales
FROM Sales
GROUP BY EmpId, Yr WITH CUBE

This query computes all possible sub-totals and totals:

		Yr
		2005	2006	2007	ALL
EmpId	1	12000.00	18000.00	25000.00	55000.00
	2	15000.00	6000.00		21000.00
	3		20000.00	24000.00	44000.00
	ALL	*27000.00*	*44000.00*	*49000.00*	120000.00

Because the WITH CUBE clause causes the query to compute all possible totals, the order of the columns in the GROUP BY clause does not matter. Of course, by default, SQL Server does not pivot the results of either of the above queries. Here is the actual output from the WITH CUBE query:

EmpId       Yr          Sales
----------- ----------- ---------------------
1           2005        12000.00
1           2006        18000.00
1           2007        25000.00
1           NULL        55000.00
2           2005        15000.00
2           2006        6000.00
2           NULL        21000.00
3           2006        20000.00
3           2007        24000.00
3           NULL        44000.00
NULL        NULL        120000.00
NULL        2005        27000.00
NULL        2006        44000.00
NULL        2007        49000.00

Next, let's look at the query plan for the WITH CUBE query:

|--Compute Scalar(DEFINE:([Expr1004]=CASE WHEN [Expr1005]=(0) THEN NULL ELSE [Expr1006] END))
       |--Concatenation
            |--Stream Aggregate(GROUP BY:([Sales].[EmpId], [Sales].[Yr]) DEFINE:([Expr1005]=SUM([Expr1007]), [Expr1006]=SUM([Expr1008])))
            |    |--Sort(ORDER BY:([Sales].[EmpId] ASC, [Sales].[Yr] ASC))
            |         |--Table Spool
            |              |--Stream Aggregate(GROUP BY:([Sales].[Yr], [Sales].[EmpId]) DEFINE:([Expr1007]=COUNT_BIG([Sales].[Sales]), [Expr1008]=SUM([Sales].[Sales])))
            |                   |--Sort(ORDER BY:([Sales].[Yr] ASC, [Sales].[EmpId] ASC))
            |                        |--Table Scan(OBJECT:([Sales]))
            |--Compute Scalar(DEFINE:([Expr1012]=NULL))
                 |--Stream Aggregate(GROUP BY:([Sales].[Yr]) DEFINE:([Expr1005]=SUM([Expr1007]), [Expr1006]=SUM([Expr1008])))
                      |--Table Spool

This plan consists of two parts. SQL Server has effectively rewritten our query as follows:

SELECT EmpId, Yr, SUM(Sales) AS Sales
FROM Sales
GROUP BY EmpId, Yr WITH ROLLUP
UNION ALL
SELECT NULL, Yr, SUM(Sales)
FROM Sales
GROUP BY Yr

The first part of the plan computes the result for the WITH ROLLUP query above. I described how this query plan works in last week's post. The second part of this plan computes the missing year sub-totals yielding the entire CUBE result. Note that this plan employs a common sub-expression spool. As I discussed in this post, a common sub-expression spool copies its input rows into a worktable and then reads and returns the rows from the worktable multiple times - in this case twice. The spool is meant to improve performance although, in this example, it has little impact since the server could just as easily have re-read the original Sales table. However, if the input to the aggregation was more complex and cost more to evaluate, the spool would help.

If we use the WITH CUBE clause when aggregating on more than two columns, SQL Server simply generates increasingly complex plans with additional inputs to the concatentation operator. As with the simple two column example, the idea is to compute the whole CUBE by computing all of the individual ROLLUPs that compose it.

Finally, we can actually combine WITH CUBE and PIVOT to generate the above table in a single simple statement. (I actually proposed a variation of this query in an answer to a reader's comment on my post about the PIVOT operator but I like this solution better.)

SELECT EmpId, [2005], [2006], [2007], [ALL]
FROM
      (
      SELECT
            CASE WHEN GROUPING(EmpId) = 0
                  THEN CAST (EmpId AS CHAR(7))
                  ELSE 'ALL'
            END AS EmpId,
            CASE WHEN GROUPING(Yr) = 0
                  THEN CAST (Yr AS CHAR(7))
                  ELSE 'ALL'
            END AS Yr,
            SUM(Sales) AS Sales
      FROM Sales
      GROUP BY EmpId, Yr WITH CUBE
      ) AS s
PIVOT (SUM(Sales) FOR Yr IN ([2005], [2006], [2007], [ALL])) AS p

Here is the output from this query:

EmpId   2005                  2006                  2007                  ALL
------- --------------------- --------------------- --------------------- ---------------------
1       12000.00              18000.00              25000.00              55000.00
2       15000.00              6000.00               NULL                  21000.00
3       NULL                  20000.00              24000.00              44000.00
ALL     27000.00              44000.00              49000.00              120000.00

Aggregation WITH ROLLUP

craigfr — Fri, 21 Sep 2007 20:36:00 GMT

In this post, I'm going to discuss how aggregation WITH ROLLUP works. The WITH ROLLUP clause permits us to execute multiple "levels" of aggregation in a single statement. For example, suppose we have the following fictitious sales data. (This is the same data that I used for my series of posts on the PIVOT operator.)

CREATE TABLE Sales (EmpId INT, Yr INT, Sales MONEY)
INSERT Sales VALUES(1, 2005, 12000)
INSERT Sales VALUES(1, 2006, 18000)
INSERT Sales VALUES(1, 2007, 25000)
INSERT Sales VALUES(2, 2005, 15000)
INSERT Sales VALUES(2, 2006, 6000)
INSERT Sales VALUES(3, 2006, 20000)
INSERT Sales VALUES(3, 2007, 24000)

We can write a simple aggregation query to compute the total sales by year:

SELECT Yr, SUM(Sales) AS Sales
FROM Sales
GROUP BY Yr

As expected, this query returns three rows - one for each year:

Yr          Sales
----------- ---------------------
2005        27000.00
2006        44000.00
2007        49000.00

The query plan is a simple stream aggregate:

|--Compute Scalar(DEFINE:([Expr1004]=CASE WHEN [Expr1010]=(0) THEN NULL ELSE [Expr1011] END))
       |--Stream Aggregate(GROUP BY:([Sales].[Yr]) DEFINE:([Expr1010]=COUNT_BIG([Sales].[Sales]), [Expr1011]=SUM([Sales].[Sales])))
            |--Sort(ORDER BY:([Sales].[Yr] ASC))
                 |--Table Scan(OBJECT:([Sales]))

Now suppose that we want to compute not just the sale by year but the total sales as well. We could write a UNION ALL query:

SELECT Yr, SUM(Sales) AS Sales
FROM Sales
GROUP BY Yr
UNION ALL
SELECT NULL, SUM(Sales) AS Sales
FROM Sales

This query works and does give the correct result:

Yr          Sales
----------- ---------------------
2005        27000.00
2006        44000.00
2007        49000.00
NULL        120000.00

However, the query plan performs two scans and two aggregations (one to compute the sales by year and one to compute the total sales):

|--Concatenation
       |--Compute Scalar(DEFINE:([Expr1004]=CASE WHEN [Expr1023]=(0) THEN NULL ELSE [Expr1024] END))
       |    |--Stream Aggregate(GROUP BY:([Sales].[Yr]) DEFINE:([Expr1023]=COUNT_BIG([Sales].[Sales]), [Expr1024]=SUM([Sales].[Sales])))
       |         |--Sort(ORDER BY:([Sales].[Yr] ASC))
       |              |--Table Scan(OBJECT:([Sales]))
       |--Compute Scalar(DEFINE:([Expr1010]=NULL))
            |--Compute Scalar(DEFINE:([Expr1009]=CASE WHEN [Expr1025]=(0) THEN NULL ELSE [Expr1026] END))
                 |--Stream Aggregate(DEFINE:([Expr1025]=COUNT_BIG([Sales].[Sales]), [Expr1026]=SUM([Sales].[Sales])))
                      |--Table Scan(OBJECT:([Sales]))

We can do better by adding a WITH ROLLUP clause to the original query:

SELECT Yr, SUM(Sales) AS Sales
FROM Sales
GROUP BY Yr WITH ROLLUP

This query is simpler to write and uses a more efficient query plan with only a single scan:

|--Compute Scalar(DEFINE:([Expr1004]=CASE WHEN [Expr1005]=(0) THEN NULL ELSE [Expr1006] END))
       |--Stream Aggregate(GROUP BY:([Sales].[Yr]) DEFINE:([Expr1005]=SUM([Expr1007]), [Expr1006]=SUM([Expr1008])))
            |--Stream Aggregate(GROUP BY:([Sales].[Yr]) DEFINE:([Expr1007]=COUNT_BIG([Sales].[Sales]), [Expr1008]=SUM([Sales].[Sales])))
                 |--Sort(ORDER BY:([Sales].[Yr] ASC))
                      |--Table Scan(OBJECT:([Sales]))

The bottom stream aggregate in this query plan is the same as the stream aggregate in the original non-ROLLUP query. This aggregation is a normal aggregation and, as such, it can be implemented using a stream aggregate (as in this example) or a hash aggregate (try adding an OPTION (HASH GROUP) clause to the above query). It can also be parallelized.

The top stream aggregate is a special aggregate that computes the ROLLUP. (Unfortunately, in SQL Server 2005 there is no way to discern from the query plan that this aggregate implements a ROLLUP. This issue will be fixed in SQL Server 2008 graphical and XML plans.) A ROLLUP aggregate is always implemented using stream aggregate and cannot be parallelized. In this simple example, the ROLLUP stream aggregate merely returns each pre-aggregated input row while maintaining a running total of the Sales column. After outputting the final input row, the aggregate also returns one additional row with the final sum. Since SQL lacks a concept of an ALL value, the Yr column is set to NULL for this final row. If NULL is valid value for Yr, we can identify the ROLLUP row using the GROUPING(Yr) construct.

SELECT
      CASE WHEN GROUPING(Yr) = 0
            THEN CAST (Yr AS CHAR(5))
            ELSE 'ALL'
      END AS Yr,
      SUM(Sales) AS Sales
FROM Sales
GROUP BY Yr WITH ROLLUP

Yr    Sales
----- ---------------------
2005  27000.00
2006  44000.00
2007  49000.00
ALL   120000.00

We can also compute multiple ROLLUP levels in a single query. For example, suppose that we want to compute the sales first by employee and then for each employee by year:

SELECT EmpId, Yr, SUM(Sales) AS Sales
FROM Sales
GROUP BY EmpId, Yr WITH ROLLUP

EmpId       Yr          Sales
----------- ----------- ---------------------
1           2005        12000.00
1           2006        18000.00
1           2007        25000.00
1           NULL        55000.00
2           2005        15000.00
2           2006        6000.00
2           NULL        21000.00
3           2006        20000.00
3           2007        24000.00
3           NULL        44000.00
NULL        NULL        120000.00

There are a couple of points worth noting about this query. First, since the combination of the EmpId and Yr columns is unique, in the absence of the WITH ROLLUP clause, this query would just return the original data. However, with the WITH ROLLUP clause the query produces a useful result. Second, the order of the columns in the GROUP BY clause is relevant with the WITH ROLLUP clause. To see why simply try the same query but reverse the EmpId and Yr columns. Instead of computing the sales first by employee it will compute the sales first by year.

The query plan for this query is identical to the query plan for the prior query except that it groups on both the EmpId and Yr columns instead of on just the EmpId column. Like the prior query plan, this query plan includes two stream aggregates: the bottom one which is a normal stream aggregate and the top one which computes the ROLLUP. This ROLLUP stream aggregate actually computes two running totals: one which computes the total sales for an employee for all years and one which compute the total sales for all employees and all years. This table shows how the ROLLUP computation proceeds:

EmpId	Yr	SUM(Sales) BY EmpId, Yr	SUM(Sales) BY EmpId	SUM(Sales)
1	2005	12000.00	12000.00	12000.00
1	2006	18000.00	30000.00	30000.00
1	2007	25000.00	55000.00	55000.00
1	NULL		55000.00	55000.00
2	2005	15000.00	15000.00	70000.00
2	2006	6000.00	21000.00	76000.00
2	NULL		21000.00	76000.00
3	2006	20000.00	20000.00	96000.00
3	2007	24000.00	44000.00	120000.00
3	NULL		44000.00	120000.00
NULL	NULL			120000.00

In my next post, I'll take a look at the WITH CUBE clause. I'll discuss how it differs from WITH ROLLUP both in terms of function and in terms of its implementation.

PIVOT Query Plans

craigfr — Mon, 09 Jul 2007 21:05:00 GMT

In my last post, I gave an overview of the PIVOT operator. In this post, I'm going to take a look at the query plans generated by the PIVOT operator. As we'll see, SQL Server generates a surprisingly simple query plan that is essentially just a fancy aggregation query plan.

Let's use the same schema and queries from my previous post:

CREATE TABLE Sales (EmpId INT, Yr INT, Sales MONEY)
INSERT Sales VALUES(1, 2005, 12000)
INSERT Sales VALUES(1, 2006, 18000)
INSERT Sales VALUES(1, 2007, 25000)
INSERT Sales VALUES(2, 2005, 15000)
INSERT Sales VALUES(2, 2006, 6000)
INSERT Sales VALUES(3, 2006, 20000)
INSERT Sales VALUES(3, 2007, 24000)

SELECT [2005], [2006], [2007]
FROM (SELECT Yr, Sales FROM Sales) AS s
PIVOT (SUM(Sales) FOR Yr IN ([2005], [2006], [2007])) AS p

This query generates the following query plan:

  |--Compute Scalar(DEFINE:(
                 [Expr1006]=CASE WHEN [Expr1024]=(0) THEN NULL ELSE [Expr1025] END,
                 [Expr1007]=CASE WHEN [Expr1026]=(0) THEN NULL ELSE [Expr1027] END,
                 [Expr1008]=CASE WHEN [Expr1028]=(0) THEN NULL ELSE [Expr1029] END))
       |--Stream Aggregate(DEFINE:(
                      [Expr1024]=COUNT_BIG(CASE WHEN [Sales].[Yr]=(2005) THEN [Sales].[Sales] ELSE NULL END),
                      [Expr1025]=SUM(CASE WHEN [Sales].[Yr]=(2005) THEN [Sales].[Sales] ELSE NULL END),
                      [Expr1026]=COUNT_BIG(CASE WHEN [Sales].[Yr]=(2006) THEN [Sales].[Sales] ELSE NULL END),
                      [Expr1027]=SUM(CASE WHEN [Sales].[Yr]=(2006) THEN [Sales].[Sales] ELSE NULL END),
                      [Expr1028]=COUNT_BIG(CASE WHEN [Sales].[Yr]=(2007) THEN [Sales].[Sales] ELSE NULL END),
                      [Expr1029]=SUM(CASE WHEN [Sales].[Yr]=(2007) THEN [Sales].[Sales] ELSE NULL END)))
            |--Table Scan(OBJECT:([Sales]))

This is just a basic scalar aggregation query plan! It calculates one SUM aggregate for each year. Like any SUM aggregate, each aggregate actually computes both the count and the sum. If the count is zero, the query plan returns NULL else it returns the sum. (The compute scalar handles this logic.)

The only twist is that each SUM aggregate is actually computed over a CASE statement that filter for those rows that match the year for which it is summing sales. The CASE statement returns the value of the Sales column for those rows that match the year and NULLs for all other rows. To clarify what is happening, we can view the results of the CASE statements without the aggregation:

SELECT EmpId, Yr,
    CASE WHEN Yr = 2005 THEN Sales END AS [2005],
    CASE WHEN Yr = 2006 THEN Sales END AS [2006],
    CASE WHEN Yr = 2007 THEN Sales END AS [2007]
FROM Sales

EmpId       Yr          2005                  2006                  2007
----------- ----------- --------------------- --------------------- ---------------------
1           2005        12000.00              NULL                  NULL
1           2006        NULL                  18000.00              NULL
1           2007        NULL                  NULL                  25000.00
2           2005        15000.00              NULL                  NULL
2           2006        NULL                  6000.00               NULL
3           2006        NULL                  20000.00              NULL
3           2007        NULL                  NULL                  24000.00
2           2007        NULL                  NULL                  NULL

When computing the sums of each of the year columns, the query plan relies on the fact that aggregate functions discard NULLs; that is, the NULL values are not included in the results. Although this point may seem intuitive for a SUM aggregate, the significance is clearer for a COUNT aggregate:

CREATE TABLE T (A INT)
INSERT T VALUES(NULL)

-- Returns 1: the number rows in table T
SELECT COUNT(*) FROM T

-- Returns 0: the number of non-NULL values of column A
SELECT COUNT(A) FROM T

Note that we could just as easily have written the original query as:

SELECT
    SUM(CASE WHEN Yr = 2005 THEN Sales END) AS [2005],
    SUM(CASE WHEN Yr = 2006 THEN Sales END) AS [2006],
    SUM(CASE WHEN Yr = 2007 THEN Sales END) AS [2007]
FROM Sales

This query gets a nearly identical query plan. The only visible difference is the use of an extra compute scalar to evaluate the CASE statements.

  |--Compute Scalar(DEFINE:(
                 [Expr1004]=CASE WHEN [Expr1013]=(0) THEN NULL ELSE [Expr1014] END,
                 [Expr1005]=CASE WHEN [Expr1015]=(0) THEN NULL ELSE [Expr1016] END,
                 [Expr1006]=CASE WHEN [Expr1017]=(0) THEN NULL ELSE [Expr1018] END))
       |--Stream Aggregate(DEFINE:(
                      [Expr1013]=COUNT_BIG([Expr1007]), [Expr1014]=SUM([Expr1007]),
                      [Expr1015]=COUNT_BIG([Expr1008]), [Expr1016]=SUM([Expr1008]),
                      [Expr1017]=COUNT_BIG([Expr1009]), [Expr1018]=SUM([Expr1009])))
            |--Compute Scalar(DEFINE:(
                           [Expr1007]=CASE WHEN [Sales].[Yr]=(2005) THEN [Sales].[Sales] ELSE NULL END,
                           [Expr1008]=CASE WHEN [Sales].[Yr]=(2006) THEN [Sales].[Sales] ELSE NULL END,
                           [Expr1009]=CASE WHEN [Sales].[Yr]=(2007) THEN [Sales].[Sales] ELSE NULL END))
                 |--Table Scan(OBJECT:([Sales]))

The other big difference between the PIVOT syntax and query plan and the alternative syntax and query plan is that the PIVOT query suppresses the following warning about NULLs:

Warning: Null value is eliminated by an aggregate or other SET operation.

We could also suppress this warning by executing the following statement:

SET ANSI_WARNINGS OFF

At this point, the query plan for a multi-row PIVOT operation should not come as a surprise:

SELECT EmpId, [2005], [2006], [2007]
FROM (SELECT EmpId, Yr, Sales FROM Sales) AS s
PIVOT (SUM(Sales) FOR Yr IN ([2005], [2006], [2007])) AS p

The query plan for this PIVOT operation uses CASE statements to compute the same intermediate result that we saw above. Then, like any other GROUP BY query, it uses either stream or hash aggregate to group by EmpId and to compute the final result. In this case, the optimizer chooses a stream aggregate. Since we do not have an index to provide order, it must also introduce a sort.

  |--Compute Scalar(DEFINE:(
                 [Expr1007]=CASE WHEN [Expr1025]=(0) THEN NULL ELSE [Expr1026] END,
                 [Expr1008]=CASE WHEN [Expr1027]=(0) THEN NULL ELSE [Expr1028] END,
                 [Expr1009]=CASE WHEN [Expr1029]=(0) THEN NULL ELSE [Expr1030] END))
       |--Stream Aggregate(GROUP BY:([Sales].[EmpId]) DEFINE:(
                      [Expr1025]=COUNT_BIG(CASE WHEN [Sales].[Yr]=(2005) THEN [Sales].[Sales] ELSE NULL END),
                      [Expr1026]=SUM(CASE WHEN [Sales].[Yr]=(2005) THEN [Sales].[Sales] ELSE NULL END),
                      [Expr1027]=COUNT_BIG(CASE WHEN [Sales].[Yr]=(2006) THEN [Sales].[Sales] ELSE NULL END),
                      [Expr1028]=SUM(CASE WHEN [Sales].[Yr]=(2006) THEN [Sales].[Sales] ELSE NULL END),
                      [Expr1029]=COUNT_BIG(CASE WHEN [Sales].[Yr]=(2007) THEN [Sales].[Sales] ELSE NULL END),
                      [Expr1030]=SUM(CASE WHEN [Sales].[Yr]=(2007) THEN [Sales].[Sales] ELSE NULL END)))
            |--Compute Scalar(DEFINE:([Sales].[EmpId]=[Sales].[EmpId]))
                 |--Sort(ORDER BY:([Sales].[EmpId] ASC))
                      |--Table Scan(OBJECT:([Sales]))

To see how this query really is no different than any other GROUP BY query or to see some alternative query plans, try creating a clustered index on EmpId to eliminate the sort or using an OPTION(HASH GROUP) hint to force a hash aggregate.

The PIVOT Operator

craigfr — Tue, 03 Jul 2007 21:31:00 GMT

In my next few posts, I'm going to look at how SQL Server implements the PIVOT and UNPIVOT operators. Let's begin with the PIVOT operator. The PIVOT operator takes a normalized table and transforms it into a new table where the columns of the new table are derived from the values in the original table. For example, suppose we want to store annual sales data by employee. We might create a schema such as the following:

CREATE TABLE Sales (EmpId INT, Yr INT, Sales MONEY)
INSERT Sales VALUES(1, 2005, 12000)
INSERT Sales VALUES(1, 2006, 18000)
INSERT Sales VALUES(1, 2007, 25000)
INSERT Sales VALUES(2, 2005, 15000)
INSERT Sales VALUES(2, 2006, 6000)
INSERT Sales VALUES(3, 2006, 20000)
INSERT Sales VALUES(3, 2007, 24000)

Notice that this schema has one row per employee per year. Moreover, notice that in the sample data employees 2 and 3 only have sales data for two of the three years worth of data. Now suppose that we'd like to transform this data into a table that has one row per employee with all three years of sales data in each row. We can achieve this conversion very easily using PIVOT:

SELECT EmpId, [2005], [2006], [2007]
FROM (SELECT EmpId, Yr, Sales FROM Sales) AS s
PIVOT (SUM(Sales) FOR Yr IN ([2005], [2006], [2007])) AS p

I'm not going to delve into the PIVOT syntax which is already documented in Books Online. Suffice it to say that this statement sums up the sales for each employee for each of the specified years and outputs one row per employee. The resulting output is:

EmpId       2005                  2006                  2007
----------- --------------------- --------------------- ---------------------
1           12000.00              18000.00              25000.00
2           15000.00              6000.00               NULL
3           NULL                  20000.00              24000.00

Notice that SQL Server inserts NULLs for the missing sales data for employees 2 and 3.

The SUM keyword (or some other aggregate) is required. If the Sales table includes multiple rows for a particular employee for a particular year, PIVOT does aggregate them - in this case by summing them - into a single data point in the result. Of course, in this example, since the entry in each "cell" of the output table is the result of summing a single input row, we could just as easily have used another aggregate such as MIN or MAX. I've used SUM since it is more intuitive.

This PIVOT example is reversible. The information in the output table can be used to reconstruct the original input table using an UNPIVOT operation (which I will cover in a later post). However, not all PIVOT operations are reversible. To be reversible, a PIVOT operation must meet the following criteria:

All of the input data must be transformed. If we include a filter of any kind including on the IN clause, some data may be omitted from the PIVOT result. For example, if we altered the above example only to output sales for 2006 and 2007, clearly we could not reconstruct the 2005 sales data from the result.
Each cell in the output table must derive from a single input row. If multiple input rows are aggregated into a single cell, there is no way to reconstruct the original input rows.
The aggregate function must be an identity function (when used on a single input row). SUM, MIN, MAX, and AVG all return the single input value unchanged and, thus, can be reversed. COUNT does not return its input value unchanged and, thus, cannot be reversed.

Here is an example of a non-reversible PIVOT operation. This example, calculates the total sales for all employees for all three years. It does not itemize the output by employee.

SELECT [2005], [2006], [2007]
FROM (SELECT Yr, Sales FROM Sales) AS s
PIVOT (SUM(Sales) FOR Yr IN ([2005], [2006], [2007])) AS p

Here is the output. Each cell represents the sum of two or three rows from the input table.

2005                  2006                  2007
--------------------- --------------------- ---------------------
27000.00              44000.00              49000.00

In my next post, I'll look at some example PIVOT query plans.

Hash Aggregate

craigfr — Wed, 20 Sep 2006 21:09:00 GMT

In my prior two posts, I wrote about the stream aggregate operator. Stream aggregate is great for scalar aggregates and for aggregations where we have an index to provide a sort order on the group by column(s) or where we need to sort anyhow (e.g., due to an order by clause).

The other aggregation operator, hash aggregate, is similar to hash join. It does not require (or preserve) sort order, requires memory, and is blocking (i.e., it does not produce any results until it has consumed its entire input). Hash aggregate excels at efficiently aggregating very large data sets.

Here is pseudo-code for the hash aggregate algorithm:

for each input row
begin
    calculate hash value on group by column(s)
    check for a matching row in the hash table
    if we do not find a match
      insert a new row into the hash table
    else
      update the matching row with the input row
end
output all rows in the hash table

While stream aggregate computes just one group at a time, hash aggregate computes all of the groups simultaneously. We use a hash table to store these groups. With each new input row, we check the hash table to see whether the new row belongs to an existing group. If it does, we simply update the existing group. If not, we create a new group. Since the input data is unsorted, any row can belong to any group. Thus, we cannot output any results until we’ve finished processing every input row.

Memory and spilling

As with hash join, the hash aggregate requires memory. Before executing a query with a hash aggregate, SQL Server uses cardinality estimates to estimate how much memory we need to execute the query. With a hash join, we store each build row, so the total memory requirement is proportional to the number and size of the build rows. The number of rows that join and the output cardinality of the join have no effect on the memory requirement of the join. With a hash aggregate, we store one row for each group, so the total memory requirement is actually proportional to the number and size of the output groups or rows. If we have fewer unique values of the group by column(s) and fewer groups, we need less memory. If we have more unique values of the group by column(s) and more groups, we need more memory.

So, what happens if we run out of memory? Again, like hash join, if we run out of memory, we must begin spilling rows to tempdb. We spill one or more buckets or partitions including any partially aggregated results along with any additional new rows that hash to the spilled buckets or partitions. Although we do not attempt to aggregate the spilled new rows, we do hash them and divide them up into several buckets or partitions. Once we’ve finished processing all input groups, we output the completed in-memory groups and repeat the algorithm by reading back and aggregating one spilled partition at a time. By dividing the spilled rows into multiple partitions, we reduce the size of each partition and, thus, reduce the risk that the algorithm will need to repeat many times.

Note that while duplicate rows are a big problem for hash join as they lead to skew in the size of the different hash buckets and make it difficult to divide the work into small uniform portions, duplicates are actually quite helpful for hash aggregate since they collapse into a single group.

Examples

The optimizer tends to favor hash aggregation for tables with more rows and groups. For example, with only 100 rows and 10 groups, we get a sort and a stream aggregate:

create table t (a int, b int, c int)

set nocount on

declare @i int

set @i = 0

while @i < 100

begin

insert t values (@i % 10, @i, @i * 3)

set @i = @i + 1

end

select sum(b) from t group by a

|--Compute Scalar(DEFINE:([Expr1004]=CASE WHEN [Expr1010]=(0) THEN NULL ELSE [Expr1011] END))
       |--Stream Aggregate(GROUP BY:([t].[a]) DEFINE:([Expr1010]=COUNT_BIG([t].[b]), [Expr1011]=SUM([t].[b])))
            |--Sort(ORDER BY:([t].[a] ASC))
                 |--Table Scan(OBJECT:([t]))

But, with 1000 rows and 100 groups, we get a hash aggregate:

truncate table t

declare @i int

set @i = 100

while @i < 1000

begin

insert t values (@i % 100, @i, @i * 3)

set @i = @i + 1

end

select sum(b) from t group by a

|--Compute Scalar(DEFINE:([Expr1004]=CASE WHEN [Expr1010]=(0) THEN NULL ELSE [Expr1011] END))
|--Hash Match(Aggregate, HASH:([t].[a]), RESIDUAL:([t].[a] = [t].[a]) DEFINE:([Expr1010]=COUNT_BIG([t].[b]), [Expr1011]=SUM([t].[b])))
|--Table Scan(OBJECT:([t]))

Notice that we hash on the group by column. The residual predicate from the hash aggregate is used to compare rows in the hash table to input rows in case we have a hash value collision.

Notice also that with the hash aggregate we do not need a sort. A sort requires more memory than the hash aggregate since we must sort 1000 rows but only need memory in the hash aggregate for 100 groups. However, if we explicitly request a sort using an order by clause in the query, we get the stream aggregate again:

select sum(b) from t group by a order by a

If the table gets big enough and the number of groups remains small enough, eventually the optimizer will decide that it is cheaper to use the hash aggregate and sort after we aggregate. For example, with 10,000 rows and only 100 groups, the optimizer decides that it is better to hash and sort only 100 group than to sort 10,000 rows:

truncate table t

set nocount on

declare @i int

set @i = 0

while @i < 10000

begin

insert t values (@i % 100, @i, @i * 3)

set @i = @i + 1

end

select sum(b) from t group by a order by a

|--Sort(ORDER BY:([t].[a] ASC))
       |--Compute Scalar(DEFINE:([Expr1004]=CASE WHEN [Expr1010]=(0) THEN NULL ELSE [Expr1011] END))
            |--Hash Match(Aggregate, HASH:([t].[a]), RESIDUAL:([t].[a] = [t].[a]) DEFINE:([Expr1010]=COUNT_BIG([t].[b]), [Expr1011]=SUM([t].[b])))
                 |--Table Scan(OBJECT:([t]))

Distinct

Just like stream aggregate, hash aggregate can be used to implement distinct operations:

select distinct a from t

|--Hash Match(Aggregate, HASH:([t].[a]), RESIDUAL:([t].[a] = [t].[a]))
|--Table Scan(OBJECT:([t]))

Or for something a little more interesting:

select sum(distinct b), sum(distinct c) from t group by a

|--Hash Match(Inner Join, HASH:([t].[a])=([t].[a]), RESIDUAL:([t].[a] = [t].[a]))
       |--Compute Scalar(DEFINE:([t].[a]=[t].[a]))
       |    |--Compute Scalar(DEFINE:([Expr1004]=CASE WHEN [Expr1018]=(0) THEN NULL ELSE [Expr1019] END))
       |         |--Hash Match(Aggregate, HASH:([t].[a]), RESIDUAL:([t].[a] = [t].[a]) DEFINE:([Expr1018]=COUNT_BIG([t].[b]), [Expr1019]=SUM([t].[b])))
       |              |--Hash Match(Aggregate, HASH:([t].[a], [t].[b]), RESIDUAL:([t].[a] = [t].[a] AND [t].[b] = [t].[b]))
       |                   |--Table Scan(OBJECT:([t]))
       |--Compute Scalar(DEFINE:([t].[a]=[t].[a]))
            |--Compute Scalar(DEFINE:([Expr1005]=CASE WHEN [Expr1020]=(0) THEN NULL ELSE [Expr1021] END))
                 |--Hash Match(Aggregate, HASH:([t].[a]), RESIDUAL:([t].[a] = [t].[a]) DEFINE:([Expr1020]=COUNT_BIG([t].[c]), [Expr1021]=SUM([t].[c])))
                      |--Hash Match(Aggregate, HASH:([t].[a], [t].[c]), RESIDUAL:([t].[a] = [t].[a] AND [t].[c] = [t].[c]))
                           |--Table Scan(OBJECT:([t]))

This plan is logically equivalent to the last plan from my stream aggregate post, but it uses hash aggregate and hash join instead of sorts, stream aggregates, and merge join. We have two hash aggregates to eliminate duplicates (one for "distinct b" and one for "distinct c") and two more hash aggregates to compute the two sums. The hash join "glues" the resulting rows together to form the final result.

Hints

You can use the “order group” and “hash group” query hints to force stream aggregate and hash aggregate respectively. These hints affect all aggregation operations in the entire query. For example:

select sum(b) from t group by a option(order group)

select sum(b) from t group by a option(hash group)

Sql profiler

You can use the SQL profiler “Hash Warning” event class (in the “Errors and Warnings” event category) to detect when a query with a hash join or hash aggregate spills. Spilling generates I/O and can adversely affect performance. See Books Online for more information about this event.

Stream Aggregate

craigfr — Wed, 13 Sep 2006 23:42:00 GMT

There are two physical operators that SQL Server uses to compute general purpose aggregates where we have a GROUP BY clause. One of these operators is stream aggregate which as we saw last week is used for scalar aggregates. The other operator is hash aggregate. In this post, I’ll take a closer look at how stream aggregate works.

The algorithm

Stream aggregate relies on data arriving sorted by the group by column(s). If we are grouping on more than one column, we can choose any sort order that includes all of the columns. For example, if we are grouping on columns a and b, we can sort on “(a, b)” or on “(b, a)”. As with merge join, the sort order may be delivered by an index or by an explicit sort operator. The sort order ensures that sets of rows with the same value for the group by columns will be adjacent to one another.

Here is pseudo-code for the stream aggregate algorithm:

clear the current aggregate results
clear the current group by columns
for each input row
begin
    if the input row does not match the current group by columns
      begin
        output the aggregate results
        clear the current aggregate results
        set the current group by columns to the input row
      end
    update the aggregate results with the input row
end

For example, if we are computing a SUM, the stream aggregate considers each input row. If the input row belongs to the current group (i.e., the group by columns of the input row match the group by columns of the previous row), we update the current SUM by adding the appropriate value from the input row to the running total. If the input row belongs to a new group (i.e., the group by columns of the input row do not match the group by columns of the previous row), we output the current SUM, reset the SUM to zero, and start a new group.

Simple examples

create table t (a int, b int, c int)

select sum(c) from t group by a, b

|--Compute Scalar(DEFINE:([Expr1004]=CASE WHEN [Expr1010]=(0) THEN NULL ELSE [Expr1011] END))
       |--Stream Aggregate(GROUP BY:([t].[b], [t].[a]) DEFINE:([Expr1010]=COUNT_BIG([t].[c]), [Expr1011]=SUM([t].[c])))
            |--Sort(ORDER BY:([t].[b] ASC, [t].[a] ASC))
                 |--Table Scan(OBJECT:([t]))

This is basically the same plan that we saw for the scalar aggregate SUM query except that we need to sort the data before we aggregate. (You can think of the scalar aggregate as one big group containing all of the rows; thus, for a scalar aggregate there is no need to sort the rows into different groups.)

Stream aggregate preserves the input sort order so if we request order on the group by columns or on a subset of the group by columns, we do not need to sort again:

select sum(c) from t group by a, b order by a

|--Compute Scalar(DEFINE:([Expr1004]=CASE WHEN [Expr1010]=(0) THEN NULL ELSE [Expr1011] END))
       |--Stream Aggregate(GROUP BY:([t].[a], [t].[b]) DEFINE:([Expr1010]=COUNT_BIG([t].[c]), [Expr1011]=SUM([t].[c])))
            |--Sort(ORDER BY:([t].[a] ASC, [t].[b] ASC))
                 |--Table Scan(OBJECT:([t]))

Note that the sort columns are reversed from the first example. Previously, we didn’t care whether we sorted on “(a, b)” or “(b, a)”. Now since the query includes an order by clause, we choose to sort on column a first.

If we have an appropriate index, we do not need to sort at all:

create clustered index tab on t(a,b)

select sum(c) from t group by a, b

|--Compute Scalar(DEFINE:([Expr1004]=CASE WHEN [Expr1010]=(0) THEN NULL ELSE [Expr1011] END))
|--Stream Aggregate(GROUP BY:([t].[a], [t].[b]) DEFINE:([Expr1010]=COUNT_BIG([t].[c]), [Expr1011]=SUM([t].[c])))
|--Clustered Index Scan(OBJECT:([t].[tab]), ORDERED FORWARD)

Select distinct

If we have an index to provide order, stream aggregate can also be used to implement select distinct. (If we need to sort to get the order, we can just let the sort distinct directly; there is no reason to use the stream aggregate.) Select distinct is essentially the same as group by on all selected columns with no aggregate functions. For example:

select distinct a, b from t

Can also be written as:

select a, b from t group by a, b

Both queries use the same plan:

|--Stream Aggregate(GROUP BY:([t].[a], [t].[b]))
|--Clustered Index Scan(OBJECT:([t].[tab]), ORDERED FORWARD)

Notice that the stream aggregate has a group by clause, but no defined columns.

Distinct aggregates

Consider this query:

select sum(distinct b) from t group by a

We need to eliminate duplicate values of column b from each group. In my last post, we saw one way to do this using a sort distinct. However, if we have an appropriate index, we can also use the stream aggregate to eliminate duplicates:

|--Compute Scalar(DEFINE:([Expr1004]=CASE WHEN [Expr1010]=(0) THEN NULL ELSE [Expr1011] END))
       |--Stream Aggregate(GROUP BY:([t].[a]) DEFINE:([Expr1010]=COUNT_BIG([t].[b]), [Expr1011]=SUM([t].[b])))
            |--Stream Aggregate(GROUP BY:([t].[a], [t].[b]))
                 |--Clustered Index Scan(OBJECT:([t].[tab]), ORDERED FORWARD)

The bottommost stream aggregate eliminates duplicates while the topmost one performs the aggregation.

Multiple distincts

Finally, consider this query:

select sum(distinct b), sum(distinct c) from t group by a

|--Merge Join(Inner Join, MANY-TO-MANY MERGE:([t].[a])=([t].[a]), RESIDUAL:([t].[a] = [t].[a]))
       |--Compute Scalar(DEFINE:([t].[a]=[t].[a]))
       |    |--Compute Scalar(DEFINE:([Expr1005]=CASE WHEN [Expr1018]=(0) THEN NULL ELSE [Expr1019] END))
       |         |--Stream Aggregate(GROUP BY:([t].[a]) DEFINE:([Expr1018]=COUNT_BIG([t].[c]), [Expr1019]=SUM([t].[c])))
       |              |--Sort(DISTINCT ORDER BY:([t].[a] ASC, [t].[c] ASC))
       |                   |--Clustered Index Scan(OBJECT:([t].[tab]))
       |--Compute Scalar(DEFINE:([t].[a]=[t].[a]))
            |--Compute Scalar(DEFINE:([Expr1004]=CASE WHEN [Expr1020]=(0) THEN NULL ELSE [Expr1021] END))
                 |--Stream Aggregate(GROUP BY:([t].[a]) DEFINE:([Expr1020]=COUNT_BIG([t].[b]), [Expr1021]=SUM([t].[b])))
                      |--Stream Aggregate(GROUP BY:([t].[a], [t].[b]))
                           |--Clustered Index Scan(OBJECT:([t].[tab]), ORDERED FORWARD)

As in the multiple scalar distinct example from my prior post, we need to break this query into two parts – one for each distinct set. Note that the computation of “sum(distinct c)” requires a sort to distinct on column c while the computation of “sum(distinct b)” uses an ordered scan of the clustered index and a stream aggregate as shown above. We join these two sets of sums on the group by column (in this case column a) to generate the final result. Since the two input sets are already sorted on the group by column, we can use a merge join. (The compute scalar operators that appear to define “[t].[a] = [t].[a]” are needed for internal purposes and can be disregarded.)

The merge join ought to be one-to-many not many-to-many since the aggregates ensure uniqueness on the group by column (and join column). This is a minor performance issue not a correctness issue. If we rewrite the original query as an explicit join, we do get a one-to-many join:

select sum_b, sum_c

from

(select a, sum(distinct b) as sum_b from t group by a) r

join

(select a, sum(distinct c) as sum_c from t group by a) s

on r.a = s.a

|--Merge Join(Inner Join, MERGE:([t].[a])=([t].[a]), RESIDUAL:([t].[a]=[t].[a]))
       |--Compute Scalar(DEFINE:([Expr1009]=CASE WHEN [Expr1020]=(0) THEN NULL ELSE [Expr1021] END))
       |    |--Stream Aggregate(GROUP BY:([t].[a]) DEFINE:([Expr1020]=COUNT_BIG([t].[c]), [Expr1021]=SUM([t].[c])))
       |         |--Sort(DISTINCT ORDER BY:([t].[a] ASC, [t].[c] ASC))
       |              |--Clustered Index Scan(OBJECT:([t].[tab]))
       |--Compute Scalar(DEFINE:([Expr1004]=CASE WHEN [Expr1022]=(0) THEN NULL ELSE [Expr1023] END))
            |--Stream Aggregate(GROUP BY:([t].[a]) DEFINE:([Expr1022]=COUNT_BIG([t].[b]), [Expr1023]=SUM([t].[b])))
                 |--Stream Aggregate(GROUP BY:([t].[a], [t].[b]))
                      |--Clustered Index Scan(OBJECT:([t].[tab]), ORDERED FORWARD)

Next …

In my next post, I will write about the other aggregation operator: hash aggregate.

Aggregation

craigfr — Wed, 06 Sep 2006 22:12:00 GMT

Aggregation refers to the collapse of a larger set of rows into a smaller set of rows. Typical aggregate functions are COUNT, MIN, MAX, SUM, and AVG. SQL Server also supports other aggregates such as STDEV and VAR.

I’m going to break this topic down into multiple posts. In this post, I’m going to focus on “scalar aggregates.” Scalar aggregates are queries with aggregate functions in the select list and no GROUP BY clause. Scalar aggregates always return a single row.

Scalar Aggregation

There is only one operator for scalar aggregation: stream aggregate. For example:

create table t (a int, b int, c int)

select count(*) from t

|--Compute Scalar(DEFINE:([Expr1004]=CONVERT_IMPLICIT(int,[Expr1005],0)))
|--Stream Aggregate(DEFINE:([Expr1005]=Count(*)))
|--Table Scan(OBJECT:([t]))

This is the aggregation equivalent of “Hello World!” The stream aggregate just counts the number of input rows and returns this result. The stream aggregate actually computes the count ([Expr1005]) as a bigint. The compute scalar is needed to convert this result to the expected output type of int. Note that a scalar stream aggregate is one of the only examples (maybe the only example – I can’t think of any others right now) of a non-leaf operator that can produce an output row even with an empty input set.

It is easy to see how to implement other simple scalar aggregate functions such as MIN, MAX, and SUM. We can also calculate multiple scalar aggregates at the same time using a single stream aggregate operator:

select min(a), max(b) from t

|--Stream Aggregate(DEFINE:([Expr1004]=MIN([t].[a]), [Expr1005]=MAX([t].[b])))
|--Table Scan(OBJECT:([t]))

This plan just reads each row of table t and keeps track of the minimum value of column a and the maximum value of column b. Note that we do not need to convert the result for the MIN and MAX aggregates since the types of these aggregates are computed based on the types of columns a and b.

Some aggregates such as AVG are actually calculated from two other aggregates such as SUM and COUNT:

select avg(a) from t

|--Compute Scalar(DEFINE:([Expr1004]=CASE WHEN [Expr1005]=(0) THEN NULL ELSE [Expr1006]/CONVERT_IMPLICIT(int,[Expr1005],0) END))
|--Stream Aggregate(DEFINE:([Expr1005]=COUNT_BIG([t].[a]), [Expr1006]=SUM([t].[a])))
|--Table Scan(OBJECT:([t]))

This time the compute scalar also calculates the average from the sum and count. The CASE expression is needed to make sure that we do not divide by zero.

Although SUM does not need to be computed per se, we still need the count:

select sum(a) from t

|--Compute Scalar(DEFINE:([Expr1004]=CASE WHEN [Expr1005]=(0) THEN NULL ELSE [Expr1006] END))
|--Stream Aggregate(DEFINE:([Expr1005]=COUNT_BIG([t].[a]), [Expr1006]=SUM([t].[a])))
|--Table Scan(OBJECT:([t]))

The CASE expression uses the count to ensure that SUM returns NULL instead of zero if we have no rows.

Scalar Distinct

Now let’s take a look at what happens if we add a DISTINCT keyword to the aggregate:

select count(distinct a) from t

|--Compute Scalar(DEFINE:([Expr1004]=CONVERT_IMPLICIT(int,[Expr1007],0)))
       |--Stream Aggregate(DEFINE:([Expr1007]=COUNT([t].[a])))
            |--Sort(DISTINCT ORDER BY:([t].[a] ASC))
                 |--Table Scan(OBJECT:([t]))

This query must only count rows that have a unique value for column a. We use the sort operator to eliminate rows with duplicate values of column a. It is easy to remove duplicate rows once we sort the input set since the duplicates will be adjacent to one another.

Not all distinct aggregates require duplicate elimination. For example, MIN and MAX behave identically with and without the distinct keyword. The minimum and maximum values of a set remain the same whether or not the set includes duplicate values.

select min(distinct a), max(distinct b) from t

|--Stream Aggregate(DEFINE:([Expr1004]=MIN([t].[a]), [Expr1005]=MAX([t].[b])))
|--Table Scan(OBJECT:([t]))

If we have a unique index, we also can skip the duplicate elimination since the index guarantees that there are no duplicates:

create unique index ta on t(a)

select count(distinct a) from t

drop index t.ta

|--Compute Scalar(DEFINE:([Expr1004]=CONVERT_IMPLICIT(int,[Expr1007],0)))
|--Stream Aggregate(DEFINE:([Expr1007]=COUNT([t].[a])))
|--Index Scan(OBJECT:([t].[ta]))

Multiple Distinct

Consider this query:

select count(distinct a), count(distinct b) from t

As we saw above, we can compute “count(distinct a)” by eliminating rows that have duplicate values for column a. Similarly, we can compute “count(distinct b)” by eliminating rows that have duplicate values for column b. But, given that these two sets of rows are different, how can we compute both at the same time? The answer is we cannot. We must first compute one aggregate result, then the other, and then we must combine the two results into a single output row:

|--Nested Loops(Inner Join)
       |--Compute Scalar(DEFINE:([Expr1004]=CONVERT_IMPLICIT(int,[Expr1010],0)))
       |    |--Stream Aggregate(DEFINE:([Expr1010]=COUNT([t].[a])))
       |         |--Sort(DISTINCT ORDER BY:([t].[a] ASC))
       |              |--Table Scan(OBJECT:([t]))
       |--Compute Scalar(DEFINE:([Expr1005]=CONVERT_IMPLICIT(int,[Expr1011],0)))
            |--Stream Aggregate(DEFINE:([Expr1011]=COUNT([t].[b])))
                 |--Sort(DISTINCT ORDER BY:([t].[b] ASC))
                      |--Table Scan(OBJECT:([t]))

The two inputs to the nested loops join compute the two counts from the original query. One of the inputs removes duplicates for column a while the other input removes duplicates for column b. The nested loops join has no join predicate; it is a cross join. Since both inputs to the nested loops join each produce a single row – they are both scalar aggregates – the result of the cross join is also a single row. The cross join just serves to “glue” the two columns of the result into a single row.

If we have more than two distinct aggregates on different columns, we just use more than one cross join. We can also use this type of plan if we have a mix of non-distinct and distinct aggregates. In that case, one of the cross join inputs aggregates without the sort.

In my next post, I’ll write about aggregates with a GROUP BY clause.