<?xml version="1.0" encoding="UTF-8" ?>
<?xml-stylesheet type="text/xsl" href="http://blogs.msdn.com/utility/FeedStylesheets/rss.xsl" media="screen"?><rss version="2.0" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:slash="http://purl.org/rss/1.0/modules/slash/" xmlns:wfw="http://wellformedweb.org/CommentAPI/"><channel><title>Craig Freedman's SQL Server Blog : Common Table Expressions</title><link>http://blogs.msdn.com/craigfr/archive/tags/Common+Table+Expressions/default.aspx</link><description>Tags: Common Table Expressions</description><dc:language>en</dc:language><generator>CommunityServer 2.1 SP1 (Build: 61025.2)</generator><item><title>Recursive CTEs continued ...</title><link>http://blogs.msdn.com/craigfr/archive/2007/11/07/recursive-ctes-continued.aspx</link><pubDate>Thu, 08 Nov 2007 00:58:00 GMT</pubDate><guid isPermaLink="false">91d46819-8472-40ad-a661-2c78acb4018c:5970474</guid><dc:creator>craigfr</dc:creator><slash:comments>0</slash:comments><comments>http://blogs.msdn.com/craigfr/comments/5970474.aspx</comments><wfw:commentRss>http://blogs.msdn.com/craigfr/commentrss.aspx?PostID=5970474</wfw:commentRss><description>In this post, I will finish the discussion of &lt;A title="Recursive CTEs" href="http://blogs.msdn.com/craigfr/archive/2007/10/25/recursive-ctes.aspx" mce_href="http://blogs.msdn.com/craigfr/archive/2007/10/25/recursive-ctes.aspx"&gt;recursive CTEs&lt;/A&gt; that I began in my last post.&amp;nbsp; I will continue to use the CTE examples from &lt;A title="WITH common_table_expression (Transact-SQL)" href="http://msdn2.microsoft.com/en-us/library/ms175972.aspx" mce_href="http://msdn2.microsoft.com/en-us/library/ms175972.aspx"&gt;Books Online&lt;/A&gt;.&amp;nbsp; To run these examples, you'll need to install &lt;A title="Microsoft SQL Server Product Samples: Database" href="http://www.codeplex.com/MSFTDBProdSamples/Release/ProjectReleases.aspx?ReleaseId=4004" mce_href="http://www.codeplex.com/MSFTDBProdSamples/Release/ProjectReleases.aspx?ReleaseId=4004"&gt;the Adventure Works Cycles OLTP sample database&lt;/A&gt;. 
&lt;P&gt;In my last post, I explained that all recursive queries follow the same pattern of one or more anchor sub-selects and one or more recursive sub-selects combined by a UNION ALL.&amp;nbsp; Similarly, all recursive query plans also follow the same pattern which looks like so:&lt;/P&gt;
&lt;P&gt;&amp;nbsp; |--Index Spool(WITH STACK)&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;|--Concatenation&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |--Compute Scalar(DEFINE:([Expr10XX]=(0)))&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |&amp;nbsp;&amp;nbsp;&amp;nbsp; |--&lt;B&gt;&lt;I&gt;... anchor sub-select plan(s) ...&lt;BR&gt;&lt;/I&gt;&lt;/B&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |--Assert(WHERE:(CASE WHEN [Expr10ZZ]&amp;gt;(100) THEN (0) ELSE NULL END))&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |--Nested Loops(Inner Join, OUTER REFERENCES:([Expr10YY], [Recr10XX], ...))&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |--Compute Scalar(DEFINE:([Expr10ZZ]=[Expr10YY]+(1)))&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |&amp;nbsp;&amp;nbsp;&amp;nbsp; |--Table Spool(WITH STACK)&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |--&lt;B&gt;&lt;I&gt;... recursive sub-select plan(s) ...&lt;/I&gt;&lt;/B&gt; &lt;/P&gt;
&lt;P&gt;Because of this basic plan shape, SQL Server cannot execute recursive queries using parallel plans.&amp;nbsp; There are two basic problems.&amp;nbsp; First, the stack spool does not support parallel execution.&amp;nbsp; Second, the nested loops join does not support &lt;A title="Parallel Nested Loops Join" href="http://blogs.msdn.com/craigfr/archive/2006/11/08/parallel-nested-loops-join.aspx" mce_href="http://blogs.msdn.com/craigfr/archive/2006/11/08/parallel-nested-loops-join.aspx"&gt;parallel execution on its inner input&lt;/A&gt;.&lt;/P&gt;
&lt;P&gt;Finally, let's look at how the placement of a WHERE clause can have a big impact on the performance of a recursive query.&amp;nbsp; In my last post, I dissected a recursive query that returns a list of all employees and their level within the organization.&amp;nbsp; Suppose that we want to run the same query but limit the results to those employees in the first two levels.&amp;nbsp; We could write the following query (which you'll also find in Books Online) with an extra WHERE clause to limit the results:&lt;/P&gt;
&lt;BLOCKQUOTE&gt;
&lt;P&gt;WITH DirectReports(ManagerID, EmployeeID, EmployeeLevel) AS &lt;BR&gt;(&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; SELECT ManagerID, EmployeeID, 0 AS EmployeeLevel&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; FROM HumanResources.Employee&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; WHERE ManagerID IS NULL&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; UNION ALL&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; SELECT e.ManagerID, e.EmployeeID, EmployeeLevel + 1&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; FROM HumanResources.Employee e&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; INNER JOIN DirectReports d&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; ON e.ManagerID = d.EmployeeID &lt;BR&gt;)&lt;BR&gt;SELECT ManagerID, EmployeeID, EmployeeLevel &lt;BR&gt;FROM DirectReports &lt;BR&gt;&lt;B&gt;WHERE&lt;/B&gt;&lt;B&gt; EmployeeLevel &amp;lt;= 2&lt;/B&gt;&lt;/P&gt;&lt;/BLOCKQUOTE&gt;
&lt;P&gt;This query computes uses essentially the same plan as the query without the extra WHERE clause.&amp;nbsp; The WHERE clause simply introduces a filter operator at the root of the plan:&lt;/P&gt;&lt;PRE&gt;Rows   Executes
34     1        |--Filter(WHERE:([Recr1012]&amp;lt;=(2)))
290    1             |--Index Spool(WITH STACK)
290    1                  |--Concatenation
0      0                       |--Compute Scalar(DEFINE:([Expr1013]=(0)))
0      0                       |    |--Compute Scalar(DEFINE:([Expr1003]=(0)))
1      1                       |         |--Index Seek(OBJECT:([Employee].[IX_Employee_ManagerID]), SEEK:([ManagerID]=NULL) ...)
289    1                       |--Assert(WHERE:(CASE WHEN [Expr1015]&amp;gt;(100) THEN (0) ELSE NULL END))
289    1                            |--Nested Loops(Inner Join, OUTER REFERENCES:([Expr1015], [Recr1006], [Recr1007], [Recr1008]))
0      0                                 |--Compute Scalar(DEFINE:([Expr1015]=[Expr1014]+(1)))
290    1                                 |    |--Table Spool(WITH STACK)
0      0                                 |--Compute Scalar(DEFINE:([Expr1009]=[Recr1008]+(1)))
289    290                                    |--Index Seek(OBJECT:([Employee].[IX_Employee_ManagerID]), SEEK:([ManagerID]=[Recr1007]) ...)&lt;/PRE&gt;
&lt;P mce_keep="true"&gt;As you can see from the row counts, this query computes the level of every employee in the organization before discarding the results that we do not want.&amp;nbsp; Now, suppose that we instead move the WHERE clause into the recursive sub-select:&lt;/P&gt;
&lt;BLOCKQUOTE&gt;
&lt;P&gt;WITH DirectReports(ManagerID, EmployeeID, EmployeeLevel) AS &lt;BR&gt;(&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; SELECT ManagerID, EmployeeID, 0 AS EmployeeLevel&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; FROM HumanResources.Employee&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; WHERE ManagerID IS NULL&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; UNION ALL&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; SELECT e.ManagerID, e.EmployeeID, EmployeeLevel + 1&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; FROM HumanResources.Employee e&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; INNER JOIN DirectReports d&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; ON e.ManagerID = d.EmployeeID&amp;nbsp;&lt;BR&gt;&lt;B&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; WHERE EmployeeLevel &amp;lt; 2&lt;BR&gt;&lt;/B&gt;)&lt;BR&gt;SELECT ManagerID, EmployeeID, EmployeeLevel &lt;BR&gt;FROM DirectReports&lt;/P&gt;&lt;/BLOCKQUOTE&gt;
&lt;P&gt;This query is semantically identical to the previous query, but if we look at the plan, we can see that the filter moves into the recursive portion of the plan:&lt;/P&gt;&lt;PRE&gt;Rows   Executes
34     1        |--Index Spool(WITH STACK)
34     1             |--Concatenation
0      0                  |--Compute Scalar(DEFINE:([Expr1013]=(0)))
0      0                  |    |--Compute Scalar(DEFINE:([Expr1003]=(0)))
1      1                  |         |--Index Seek(OBJECT:([Employee].[IX_Employee_ManagerID]), SEEK:([ManagerID]=NULL) ...)
33     1                  |--Assert(WHERE:(CASE WHEN [Expr1015]&amp;gt;(100) THEN (0) ELSE NULL END))
33     1                       |--Nested Loops(Inner Join, OUTER REFERENCES:([Expr1015], [Recr1006], [Recr1007], [Recr1008]))
0      0                            |--Compute Scalar(DEFINE:([Expr1015]=[Expr1014]+(1)))
34     1                            |    |--Table Spool(WITH STACK)
0      0                            |--Compute Scalar(DEFINE:([Expr1009]=[Recr1008]+(1)))
33     34                                |--Nested Loops(Inner Join)
7      34                                     |--Filter(WHERE:(STARTUP EXPR([Recr1008]&amp;lt;(2))))
7      7                                      |    |--Constant Scan
33     7                                      |--Index Seek(OBJECT:([Employee].[IX_Employee_ManagerID]), SEEK:([ManagerID]=[Recr1007]) ...)&lt;/PRE&gt;
&lt;P mce_keep="true"&gt;Notice from the row counts that the new plan computes only the portion of the result set that really interests us and then terminates the recursion.&lt;/P&gt;
&lt;P&gt;You may also observe than the plan now has a startup filter.&amp;nbsp; An ordinary filter reads rows from its input tree and then evaluates a Boolean expression on each row to determine whether to return it.&amp;nbsp; A startup filter evaluates its expression only once per execution.&amp;nbsp; The expression does not depend on the input rows.&amp;nbsp; If the expression evaluates to true, the startup filter returns all of its input rows.&amp;nbsp; If the expression evaluates to false, the startup filter returns no rows.&lt;/P&gt;
&lt;P&gt;In this example, if the startup filter expression evaluates to true, the constant scan returns one row to the nested loops join immediately above the filter and this join then executes the index seek on its inner input.&amp;nbsp; However, if the startup filter expression evaluates to false, the filter and then the nested loops join return no rows.&lt;/P&gt;
&lt;P&gt;While there are many scenarios where a startup filter is genuinely useful, this plan is slightly more complex than is really necessary.&amp;nbsp; The plan could have used an ordinary filter:&lt;/P&gt;
&lt;P&gt;&amp;nbsp; |--Index Spool(WITH STACK)&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |--Concatenation&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |--Compute Scalar(DEFINE:([Expr1013]=(0)))&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |&amp;nbsp;&amp;nbsp;&amp;nbsp; |--Compute Scalar(DEFINE:([Expr1003]=(0)))&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |--Index Seek(OBJECT:([HumanResources].[Employee].[IX_Employee_ManagerID]), SEEK:([HumanResources].[Employee].[ManagerID]=NULL) ORDERED FORWARD)&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |--Assert(WHERE:(CASE WHEN [Expr1015]&amp;gt;(100) THEN (0) ELSE NULL END))&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |--Nested Loops(Inner Join, OUTER REFERENCES:([Expr1015], [Recr1006], [Recr1007], [Recr1008]))&lt;BR&gt;&lt;B&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |--Filter(WHERE:([Recr1008]&amp;lt;(2)))&lt;BR&gt;&lt;/B&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |&amp;nbsp;&amp;nbsp;&amp;nbsp; |--Compute Scalar(DEFINE:([Expr1015]=[Expr1014]+(1)))&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |--Table Spool(WITH STACK)&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |--Compute Scalar(DEFINE:([Expr1009]=[Recr1008]+(1)))&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |--Index Seek(OBJECT:([HumanResources].[Employee].[IX_Employee_ManagerID] AS [e]), SEEK:([e].[ManagerID]=[Recr1007]) ORDERED FORWARD)&lt;/P&gt;Note that this plan is artificial.&amp;nbsp; You cannot get SQL Server to generate it.&lt;img src="http://blogs.msdn.com/aggbug.aspx?PostID=5970474" width="1" height="1"&gt;</description><category domain="http://blogs.msdn.com/craigfr/archive/tags/Common+Table+Expressions/default.aspx">Common Table Expressions</category></item><item><title>Recursive CTEs</title><link>http://blogs.msdn.com/craigfr/archive/2007/10/25/recursive-ctes.aspx</link><pubDate>Thu, 25 Oct 2007 21:57:00 GMT</pubDate><guid isPermaLink="false">91d46819-8472-40ad-a661-2c78acb4018c:5676250</guid><dc:creator>craigfr</dc:creator><slash:comments>1</slash:comments><comments>http://blogs.msdn.com/craigfr/comments/5676250.aspx</comments><wfw:commentRss>http://blogs.msdn.com/craigfr/commentrss.aspx?PostID=5676250</wfw:commentRss><description>One of the most important uses of CTEs is to write recursive queries.&amp;nbsp; In fact, CTEs provide the only means to write recursive queries.&amp;nbsp; As I noted &lt;A title="CTEs (Common Table Expressions)" href="http://blogs.msdn.com/craigfr/archive/2007/10/18/ctes-common-table-expressions.aspx" mce_href="http://blogs.msdn.com/craigfr/archive/2007/10/18/ctes-common-table-expressions.aspx"&gt;last week&lt;/A&gt;, there are several excellent CTE examples, including recursive CTE examples, in &lt;A title="WITH common_table_expression (Transact-SQL)" href="http://msdn2.microsoft.com/en-us/library/ms175972.aspx" mce_href="http://msdn2.microsoft.com/en-us/library/ms175972.aspx"&gt;Books Online&lt;/A&gt;.&amp;nbsp; I'll once again start with the examples from Books Online.&amp;nbsp; To run these examples, you'll need to install &lt;A title="Microsoft SQL Server Product Samples: Database" href="http://www.codeplex.com/MSFTDBProdSamples/Release/ProjectReleases.aspx?ReleaseId=4004" mce_href="http://www.codeplex.com/MSFTDBProdSamples/Release/ProjectReleases.aspx?ReleaseId=4004"&gt;the Adventure Works Cycles OLTP sample database&lt;/A&gt;. 
&lt;P&gt;Recursive CTEs always follow the same pattern.&amp;nbsp; The body of the CTE is a UNION ALL query that combines one or more anchor sub-selects which seed the result set and one or more recursive sub-selects which generate the remainder of the result set.&amp;nbsp; The recursive sub-selects reference the recursive CTE itself.&lt;/P&gt;
&lt;P&gt;Let's look at a simple example which computes a list of all employees and their level within the organization.&amp;nbsp; Computing the level with the organization can only be done using a recursive CTE as it requires counting the number of joins required to reach each employee:&lt;/P&gt;
&lt;BLOCKQUOTE&gt;
&lt;P&gt;WITH DirectReports(ManagerID, EmployeeID, EmployeeLevel) AS &lt;BR&gt;(&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; SELECT ManagerID, EmployeeID, 0 AS EmployeeLevel&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; FROM HumanResources.Employee&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; WHERE ManagerID IS NULL&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; UNION ALL&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; SELECT e.ManagerID, e.EmployeeID, EmployeeLevel + 1&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; FROM HumanResources.Employee e&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; INNER JOIN DirectReports d&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; ON e.ManagerID = d.EmployeeID &lt;BR&gt;)&lt;BR&gt;SELECT ManagerID, EmployeeID, EmployeeLevel &lt;BR&gt;FROM DirectReports&lt;/P&gt;&lt;/BLOCKQUOTE&gt;
&lt;P&gt;This example has a single anchor sub-select and a single recursive sub-select.&amp;nbsp; The anchor sub-select computes a list of top level employees who have no manager (in this case, there is only one such employee) and assigns these employees level 0.&amp;nbsp; The recursive sub-select then searches for employees, who work for each top level manager, and assigns these employees level 1.&amp;nbsp; The process then repeats with the recursive sub-select searching for employees progressively deeper within the organization.&amp;nbsp; The query terminates when it reaches employees who are not managers.&amp;nbsp; At this point the recursive sub-select returns no further employees.&lt;/P&gt;
&lt;P&gt;Now let's look at the plan used by this query:&lt;/P&gt;
&lt;P&gt;&amp;nbsp; |--Index Spool(WITH STACK)&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;|--Concatenation&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |--Compute Scalar(DEFINE:([Expr1013]=(0)))&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |&amp;nbsp;&amp;nbsp;&amp;nbsp; |--Compute Scalar(DEFINE:([Expr1003]=(0)))&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |--Index Seek(OBJECT:([HumanResources].[Employee].[IX_Employee_ManagerID]), SEEK:([HumanResources].[Employee].[ManagerID]=NULL) ORDERED FORWARD)&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |--Assert(WHERE:(CASE WHEN [Expr1015]&amp;gt;(100) THEN (0) ELSE NULL END))&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |--Nested Loops(Inner Join, OUTER REFERENCES:([Expr1015], [Recr1006], [Recr1007], [Recr1008]))&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |--Compute Scalar(DEFINE:([Expr1015]=[Expr1014]+(1)))&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |&amp;nbsp;&amp;nbsp;&amp;nbsp; |--Table Spool(WITH STACK)&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |--Compute Scalar(DEFINE:([Expr1009]=[Recr1008]+(1)))&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |--Index Seek(OBJECT:([HumanResources].[Employee].[IX_Employee_ManagerID] AS [e]), SEEK:([e].[ManagerID]=[Recr1007]) ORDERED FORWARD) &lt;/P&gt;
&lt;P&gt;Like the query itself, this plan has two parts: an anchor part and a recursive part.&amp;nbsp; These two parts are "glued" together using the concatenation operator which implements the UNION ALL.&lt;/P&gt;
&lt;P&gt;The single most interesting operator in this plan is the spool.&amp;nbsp; This is a special spool known as a stack spool.&amp;nbsp; A stack spool is a bit like a common subexpression spool in that it has a primary instance which loads input rows into the worktable and a secondary instance which reads and returns the rows from the worktable.&amp;nbsp; The primary spool has a child since it load input rows while the secondary spool does not have a child since it merely reads and returns the rows loaded by the primary spool.&amp;nbsp; (See my posts on &lt;A title="Optimized Non-clustered Index Maintenance in Per-Index Plans" href="http://blogs.msdn.com/craigfr/archive/2007/08/22/optimized-non-clustered-index-maintenance-in-per-index-plans.aspx" mce_href="http://blogs.msdn.com/craigfr/archive/2007/08/22/optimized-non-clustered-index-maintenance-in-per-index-plans.aspx"&gt;Optimized Non-clustered Index Maintenance in Per-Index Plans&lt;/A&gt; and &lt;A title="Aggregation WITH CUBE" href="http://blogs.msdn.com/craigfr/archive/2007/09/27/aggregation-with-cube.aspx" mce_href="http://blogs.msdn.com/craigfr/archive/2007/09/27/aggregation-with-cube.aspx"&gt;Aggregation WITH CUBE&lt;/A&gt; for examples of common subexpression spools.)&lt;/P&gt;
&lt;P&gt;Unlike ordinary common subexpression spools which eagerly load all rows into the worktable before returning any rows, a stack spool is a lazy spool which means that it begins returning rows immediately.&amp;nbsp; The primary spool (the topmost operator in the plan) is fairly mundane.&amp;nbsp; Like any other lazy spool, it reads each input row, inserts the row into the spool, and returns the row to the client.&amp;nbsp; The secondary spool (below the nested loops join) is where all the interesting action is.&amp;nbsp; Each time the nested loops join requests a row from the secondary spool, the spool reads the &lt;I&gt;last&lt;/I&gt; row from the spool, returns it, and then deletes it to ensure that it is not returned more than once.&amp;nbsp; Note that the term stack spool comes from this last in first out behavior.&amp;nbsp; The primary spool pushes rows onto the stack and the secondary spool pops rows off of the stack.&lt;/P&gt;
&lt;P&gt;Now that we understand how the spool works, let's step through the plan and see how it works.&amp;nbsp; The query processor begins by executing the first input to the concatenation operator which also happens to be the anchor part of the plan.&amp;nbsp; This part of the plan uses an index seek to find all employees who have no manager.&amp;nbsp; The index seek returns the following row which the primary spool inserts into the worktable:&lt;/P&gt;&lt;PRE&gt;ManagerID   EmployeeID  EmployeeLevel
----------- ----------- -------------
NULL        109         0&lt;/PRE&gt;
&lt;P&gt;Once it finishes executing the anchor part of the plan, the query processor continues by executing the second input to the concatenation operator which happens to be the recursive part of the plan.&amp;nbsp; This part of the plan has a nested loops join with the secondary spool as its outer input.&amp;nbsp; The nested loops join requests a row from the spool which so far contains only the one row for employee 109.&amp;nbsp; The spool returns this row and deletes it from the worktable.&amp;nbsp; The join then executes the index seek on its inner input to find all employees who work for employee 109.&amp;nbsp; This index seek returns the following rows which the primary spool inserts into the worktable:&lt;/P&gt;&lt;PRE&gt;ManagerID   EmployeeID  EmployeeLevel
----------- ----------- -------------
109         6           1
109         12          1
109         42          1
109         140         1
109         148         1
109         273         1&lt;/PRE&gt;
&lt;P&gt;Next, the nested loops join requests another row from the secondary spool.&amp;nbsp; This time the worktable contains the six rows shown above, so the spool returns (and deletes) the last row corresponding to employee 273.&amp;nbsp; The nested loops join again executes the index seek on its inner input and returns the following rows:&lt;/P&gt;&lt;PRE&gt;ManagerID   EmployeeID  EmployeeLevel
----------- ----------- -------------
273         268         2
273         284         2
273         288         2&lt;/PRE&gt;
&lt;P&gt;Now the worktable contains the following rows:&lt;/P&gt;&lt;PRE&gt;ManagerID   EmployeeID  EmployeeLevel
----------- ----------- -------------
109         6           1
109         12          1
109         42          1
109         140         1
109         148         1
273         268         2
273         284         2
273         288         2&lt;/PRE&gt;
&lt;P&gt;This process repeats continuing with employee 288.&amp;nbsp; Eventually, the spool returns a row corresponding to an employee who is not a manager.&amp;nbsp; At that point, the index seek on the inner side of the join returns no rows.&amp;nbsp; After the join consumes the final row from the spool, the query terminates.&lt;/P&gt;
&lt;P&gt;There are few other interesting points worth noting about this plan.&amp;nbsp; First, the plan includes two sets of compute scalars to compute the recursion level.&amp;nbsp; One set ([Expr1003] and [Expr1009]) is used to compute EmployeeLevel.&amp;nbsp; The other set ([Expr1013] and [Expr1015]) is generated internally and used by the assert operator to ensure that the query terminates and does not get into an infinite loop.&amp;nbsp; If you repeat this query without EmployeeLevel, you will see that one set of the compute scalars disappears from the plan while the internally generated set remains.&lt;/P&gt;
&lt;P&gt;We can control the maximum recursion level.&amp;nbsp; For example, if you run the query with OPTION (MAXRECURSION 2), you will see that the constant in the assert operator changes from the default of 100 to 2.&amp;nbsp; Since this query requires more than two levels of recursion to complete, if you execute it with this option, it will fail with the following error:&lt;/P&gt;
&lt;BLOCKQUOTE&gt;
&lt;P&gt;Msg 530, Level 16, State 1, Line 1&lt;BR&gt;The statement terminated. The maximum recursion 2 has been exhausted before statement completion.&lt;/P&gt;&lt;/BLOCKQUOTE&gt;In my next post I'll continue this discussion of recursive CTEs.&lt;img src="http://blogs.msdn.com/aggbug.aspx?PostID=5676250" width="1" height="1"&gt;</description><category domain="http://blogs.msdn.com/craigfr/archive/tags/Common+Table+Expressions/default.aspx">Common Table Expressions</category></item><item><title>CTEs (Common Table Expressions)</title><link>http://blogs.msdn.com/craigfr/archive/2007/10/18/ctes-common-table-expressions.aspx</link><pubDate>Thu, 18 Oct 2007 22:59:00 GMT</pubDate><guid isPermaLink="false">91d46819-8472-40ad-a661-2c78acb4018c:5512974</guid><dc:creator>craigfr</dc:creator><slash:comments>3</slash:comments><comments>http://blogs.msdn.com/craigfr/comments/5512974.aspx</comments><wfw:commentRss>http://blogs.msdn.com/craigfr/commentrss.aspx?PostID=5512974</wfw:commentRss><description>&lt;P&gt;CTEs or common table expressions, which are new in SQL Server 2005, provide an easy way to break a complex SQL statement down into smaller more manageable queries.&amp;nbsp; CTEs are is some ways very much like views.&amp;nbsp; Unlike a view which can be created once and used by many SQL statements, a CTE is associated with a single SQL statement.&amp;nbsp; There are several excellent CTE examples, including recursive CTE examples, in &lt;A title="WITH common_table_expression (Transact-SQL)" href="http://msdn2.microsoft.com/en-us/library/ms175972.aspx"&gt;Books Online&lt;/A&gt;.&amp;nbsp; Rather than construct entirely new examples for this post, I've started with the examples from Books Online.&amp;nbsp; To run these examples, you'll need to install &lt;A title="Microsoft SQL Server Product Samples: Database" href="http://www.codeplex.com/MSFTDBProdSamples/Release/ProjectReleases.aspx?ReleaseId=4004"&gt;the Adventure Works Cycles OLTP sample database&lt;/A&gt;.&lt;/P&gt;
&lt;P&gt;Let's start with the following simple example:&lt;/P&gt;
&lt;BLOCKQUOTE&gt;
&lt;P&gt;WITH DirReps(ManagerID, DirectReports) AS &lt;BR&gt;(&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; SELECT ManagerID, COUNT(*)&amp;nbsp;&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; FROM HumanResources.Employee AS e&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; WHERE ManagerID IS NOT NULL&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; GROUP BY ManagerID&lt;BR&gt;)&lt;BR&gt;SELECT ManagerID, DirectReports &lt;BR&gt;FROM DirReps &lt;BR&gt;ORDER BY ManagerID&lt;/P&gt;&lt;/BLOCKQUOTE&gt;
&lt;P&gt;This query is identical to the following simpler query:&lt;/P&gt;
&lt;BLOCKQUOTE&gt;
&lt;P&gt;SELECT ManagerID, COUNT(*) AS DirectReports&lt;BR&gt;FROM HumanResources.Employee AS e&lt;BR&gt;WHERE ManagerID IS NOT NULL&lt;BR&gt;GROUP BY ManagerID&lt;BR&gt;ORDER BY ManagerID&lt;/P&gt;&lt;/BLOCKQUOTE&gt;
&lt;P&gt;SQL Server expands the CTE without materializing it.&amp;nbsp; Thus, both queries produce the same query plan:&lt;/P&gt;
&lt;P&gt;&amp;nbsp; |--Compute Scalar(DEFINE:([Expr1002]=CONVERT_IMPLICIT(int,[Expr1005],0)))&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |--Stream Aggregate(GROUP BY:([e].[ManagerID]) DEFINE:([Expr1005]=Count(*)))&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |--Index Seek(OBJECT:([HumanResources].[Employee].[IX_Employee_ManagerID] AS [e]), SEEK:([e].[ManagerID] IsNotNull) ORDERED FORWARD)&lt;/P&gt;
&lt;P&gt;CTEs are more useful when a query includes multiple instances of the same sub-select.&amp;nbsp; For example, the following query computes the number of orders sold by and the most recent order date for each Adventure Works employee and then computes the same information for each employee's manager.&amp;nbsp; Rather than repeat the sub-select that computes the number of orders and the most recent order date, we can use a CTE to make the query simpler and clearer:&lt;/P&gt;
&lt;BLOCKQUOTE&gt;
&lt;P&gt;WITH Sales_CTE (SalesPersonID, NumberOfOrders, MaxDate)&lt;BR&gt;AS&lt;BR&gt;(&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; SELECT SalesPersonID, COUNT(*), MAX(OrderDate)&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; FROM Sales.SalesOrderHeader&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; GROUP BY SalesPersonID&lt;BR&gt;)&lt;BR&gt;SELECT E.EmployeeID, OS.NumberOfOrders, OS.MaxDate,&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; E.ManagerID, OM.NumberOfOrders, OM.MaxDate&lt;BR&gt;FROM HumanResources.Employee AS E&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; JOIN &lt;B&gt;Sales_CTE&lt;/B&gt; AS OS&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; ON E.EmployeeID = OS.SalesPersonID&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; LEFT OUTER JOIN &lt;B&gt;Sales_CTE&lt;/B&gt; AS OM&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; ON E.ManagerID = OM.SalesPersonID&lt;BR&gt;ORDER BY E.EmployeeID&lt;/P&gt;&lt;/BLOCKQUOTE&gt;
&lt;P&gt;We could write this query without the CTE as:&lt;/P&gt;
&lt;BLOCKQUOTE&gt;
&lt;P&gt;SELECT E.EmployeeID, OS.NumberOfOrders, OS.MaxDate,&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; E.ManagerID, OM.NumberOfOrders, OM.MaxDate&lt;BR&gt;FROM HumanResources.Employee AS E&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; JOIN (&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; SELECT SalesPersonID, COUNT(*) AS NumberOfOrders, MAX(OrderDate) AS MaxDate&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; FROM Sales.SalesOrderHeader&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; GROUP BY SalesPersonID&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; ) AS OS&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; ON E.EmployeeID = OS.SalesPersonID&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; LEFT OUTER JOIN (&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; SELECT SalesPersonID, COUNT(*) AS NumberOfOrders, MAX(OrderDate) AS MaxDate&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; FROM Sales.SalesOrderHeader&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; GROUP BY SalesPersonID&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; ) AS OM&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; ON E.ManagerID = OM.SalesPersonID&lt;BR&gt;ORDER BY E.EmployeeID&lt;/P&gt;&lt;/BLOCKQUOTE&gt;
&lt;P&gt;Alternatively, we could use a view although that requires explicitly creating a view (and dropping it if you do not want to persist it).&amp;nbsp; Views are less convenient and more costly (the DDL is not free) than CTEs.&amp;nbsp; If you do create and drop views on the fly, you must also be careful not to use a name that will not conflict with other sessions that might also create the same view.&lt;/P&gt;
&lt;BLOCKQUOTE&gt;
&lt;P&gt;CREATE VIEW Sales_View (SalesPersonID, NumberOfOrders, MaxDate) AS&lt;BR&gt;SELECT SalesPersonID, COUNT(*), MAX(OrderDate)&lt;BR&gt;FROM Sales.SalesOrderHeader&lt;BR&gt;GROUP BY SalesPersonID&lt;BR&gt;GO&lt;BR&gt;SELECT E.EmployeeID, OS.NumberOfOrders, OS.MaxDate,&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; E.ManagerID, OM.NumberOfOrders, OM.MaxDate&lt;BR&gt;FROM HumanResources.Employee AS E&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; JOIN &lt;B&gt;Sales_View&lt;/B&gt; AS OS&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; ON E.EmployeeID = OS.SalesPersonID&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; LEFT OUTER JOIN &lt;B&gt;Sales_View&lt;/B&gt; AS OM&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; ON E.ManagerID = OM.SalesPersonID&lt;BR&gt;ORDER BY E.EmployeeID;&lt;BR&gt;GO&lt;BR&gt;DROP VIEW Sales_View&lt;BR&gt;GO&lt;/P&gt;&lt;/BLOCKQUOTE&gt;
&lt;P&gt;All three of these variations produce the same query plan:&lt;/P&gt;
&lt;P&gt;&amp;nbsp; |--Nested Loops(Left Outer Join, WHERE:([E].[ManagerID]=[Sales].[SalesOrderHeader].[SalesPersonID]))&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |--Sort(ORDER BY:([E].[EmployeeID] ASC))&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |&amp;nbsp;&amp;nbsp;&amp;nbsp; |--Nested Loops(Inner Join, OUTER REFERENCES:([Sales].[SalesOrderHeader].[SalesPersonID]))&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; &lt;B&gt;|--Compute Scalar(DEFINE:([Expr1005]=CONVERT_IMPLICIT(int,[Expr1021],0)))&lt;BR&gt;&lt;/B&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |&lt;B&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; |--Hash Match(Aggregate, HASH:([Sales].[SalesOrderHeader].[SalesPersonID]), RESIDUAL:(...) DEFINE:([Expr1021]=COUNT(*), [Expr1006]=MAX([Sales].[SalesOrderHeader].[OrderDate])))&lt;BR&gt;&lt;/B&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |&lt;B&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |--Clustered Index Scan(OBJECT:([Sales].[SalesOrderHeader].[PK_SalesOrderHeader_SalesOrderID]))&lt;BR&gt;&lt;/B&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |--Clustered Index Seek(OBJECT:([HumanResources].[Employee].[PK_Employee_EmployeeID] AS [E]), SEEK:([E].[EmployeeID]=[Sales].[SalesOrderHeader].[SalesPersonID]) ORDERED FORWARD)&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |--Table Spool&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |--Compute Scalar(DEFINE:([Expr1010]=[Expr1010], [Expr1011]=[Expr1011]))&lt;BR&gt;&lt;B&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |--Compute Scalar(DEFINE:([Expr1010]=CONVERT_IMPLICIT(int,[Expr1022],0)))&lt;BR&gt;&lt;/B&gt;&lt;B&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |--Hash Match(Aggregate, HASH:([Sales].[SalesOrderHeader].[SalesPersonID]), RESIDUAL:(...) DEFINE:([Expr1022]=COUNT(*), [Expr1011]=MAX([Sales].[SalesOrderHeader].[OrderDate])))&lt;BR&gt;&lt;/B&gt;&lt;B&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |--Clustered Index Scan(OBJECT:([Sales].[SalesOrderHeader].[PK_SalesOrderHeader_SalesOrderID]))&lt;/B&gt;&lt;/P&gt;
&lt;P&gt;Notice that this query plan computes the CTE result twice (once for each reference).&amp;nbsp; We have two scans of the SalesOrderHeader table and two hash aggregates.&amp;nbsp; Even with the CTE (or the view), SQL Server still computes the result twice.&amp;nbsp; Do not be fooled by the table spool.&amp;nbsp; This spool merely caches the CTE result to avoid computing the entire aggregate once per employee.&lt;/P&gt;
&lt;P&gt;We could force SQL Server to compute the CTE result only once by explicitly materializing it using a table variable, a temp table, or an indexed view.&amp;nbsp; Here is an example using a table variable:&lt;/P&gt;
&lt;BLOCKQUOTE&gt;
&lt;P&gt;DECLARE @Sales_Data TABLE (SalesPersonID INT, NumberOfOrders INT, MaxDate DATETIME)&lt;/P&gt;
&lt;P mce_keep="true"&gt;INSERT @Sales_Data&lt;BR&gt;SELECT SalesPersonID, COUNT(*), MAX(OrderDate)&lt;BR&gt;FROM Sales.SalesOrderHeader&lt;BR&gt;GROUP BY SalesPersonID&lt;/P&gt;
&lt;P mce_keep="true"&gt;SELECT E.EmployeeID, OS.NumberOfOrders, OS.MaxDate,&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; E.ManagerID, OM.NumberOfOrders, OM.MaxDate&lt;BR&gt;FROM HumanResources.Employee AS E&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; JOIN @Sales_Data AS OS&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; ON E.EmployeeID = OS.SalesPersonID&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; LEFT OUTER JOIN @Sales_Data AS OM&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; ON E.ManagerID = OM.SalesPersonID&lt;BR&gt;ORDER BY E.EmployeeID&lt;/P&gt;&lt;/BLOCKQUOTE&gt;
&lt;P&gt;This rewrite has two statements and, thus, has two plans.&amp;nbsp; The first plan materializes the CTE result into a table variable while the second plan reads this result twice.&lt;/P&gt;
&lt;P&gt;&amp;nbsp; |--Table Insert(OBJECT:(@Sales_Data), SET:([SalesPersonID] = [Sales].[SalesOrderHeader].[SalesPersonID],[NumberOfOrders] = [Expr1007],[MaxDate] = [Expr1008]))&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |--Top(ROWCOUNT est 0)&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |--Compute Scalar(DEFINE:([Expr1007]=CONVERT_IMPLICIT(int,[Expr1012],0)))&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |--Hash Match(Aggregate, HASH:([Sales].[SalesOrderHeader].[SalesPersonID]), RESIDUAL:(...) DEFINE:([Expr1012]=COUNT(*), [Expr1008]=MAX([Sales].[SalesOrderHeader].[OrderDate])))&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |--Clustered Index Scan(OBJECT:([Sales].[SalesOrderHeader].[PK_SalesOrderHeader_SalesOrderID]))&lt;/P&gt;
&lt;P&gt;&amp;nbsp; |--Nested Loops(Left Outer Join, WHERE:([E].[ManagerID]=[OM].[SalesPersonID]))&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |--Nested Loops(Inner Join, OUTER REFERENCES:([OS].[SalesPersonID]))&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |&amp;nbsp;&amp;nbsp;&amp;nbsp; |--Sort(ORDER BY:([OS].[SalesPersonID] ASC))&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |&amp;nbsp;&amp;nbsp;&amp;nbsp; |&amp;nbsp;&amp;nbsp;&amp;nbsp; &lt;B&gt;|--Table Scan(OBJECT:(@Sales_Data AS [OS]))&lt;BR&gt;&lt;/B&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;|&amp;nbsp;&amp;nbsp;&amp;nbsp; |--Clustered Index Seek(OBJECT:([HumanResources].[Employee].[PK_Employee_EmployeeID] AS [E]), SEEK:([E].[EmployeeID]=[OS].[SalesPersonID]) ORDERED FORWARD)&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; &lt;B&gt;|--Table Scan(OBJECT:(@Sales_Data AS [OM]))&lt;/B&gt;&lt;B&gt;&lt;/B&gt;&lt;/P&gt;
&lt;P&gt;Explicitly materializing a CTE (or view or sub-select) result does not always yield better performance.&amp;nbsp; First, we must consider the cost of creating and writing the temp table.&amp;nbsp; If the cost of computing the CTE is not too great, it may be cheaper simply to compute the CTE result multiple times.&amp;nbsp; Second, in some cases, the query optimizer may be able to choose a better plan and avoid computing the entire CTE result.&amp;nbsp; For example, suppose that we only want to compute the number of orders for a single employee:&lt;/P&gt;
&lt;P&gt;DECLARE @EID INT&lt;BR&gt;SET @EID = 268;&lt;/P&gt;
&lt;P mce_keep="true"&gt;WITH Sales_CTE (SalesPersonID, NumberOfOrders, MaxDate)&lt;BR&gt;AS&lt;BR&gt;(&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; SELECT SalesPersonID, COUNT(*), MAX(OrderDate)&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; FROM Sales.SalesOrderHeader&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; GROUP BY SalesPersonID&lt;BR&gt;)&lt;BR&gt;SELECT E.EmployeeID, OS.NumberOfOrders, OS.MaxDate,&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; E.ManagerID, OM.NumberOfOrders, OM.MaxDate&lt;BR&gt;FROM HumanResources.Employee AS E&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; JOIN Sales_CTE AS OS&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; ON E.EmployeeID = OS.SalesPersonID&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; LEFT OUTER JOIN Sales_CTE AS OM&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp; ON E.ManagerID = OM.SalesPersonID&lt;BR&gt;WHERE E.EmployeeID = @EID&lt;/P&gt;
&lt;P&gt;This query yields a different plan from the original query:&lt;/P&gt;
&lt;P&gt;&amp;nbsp; |--Nested Loops(Left Outer Join, OUTER REFERENCES:([E].[ManagerID]))&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |--Nested Loops(Inner Join)&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |&amp;nbsp;&amp;nbsp;&amp;nbsp; |--Clustered Index Seek(OBJECT:([HumanResources].[Employee].[PK_Employee_EmployeeID] AS [E]), SEEK:([E].[EmployeeID]=[@EID]) ORDERED FORWARD)&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |&amp;nbsp;&amp;nbsp;&amp;nbsp; |--Compute Scalar(DEFINE:([Expr1005]=CONVERT_IMPLICIT(int,[Expr1021],0)))&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |--Stream Aggregate(DEFINE:([Expr1021]=Count(*), [Expr1006]=MAX([Sales].[SalesOrderHeader].[OrderDate])))&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |--Clustered Index Scan(OBJECT:([Sales].[SalesOrderHeader].[PK_SalesOrderHeader_SalesOrderID]), &lt;B&gt;WHERE:([Sales].[SalesOrderHeader].[SalesPersonID]=[@EID])&lt;/B&gt;)&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |--Compute Scalar(DEFINE:([Expr1010]=[Expr1010], [Expr1011]=[Expr1011]))&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |--Compute Scalar(DEFINE:([Expr1010]=CONVERT_IMPLICIT(int,[Expr1022],0)))&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |--Stream Aggregate(DEFINE:([Expr1022]=Count(*), [Expr1011]=MAX([Sales].[SalesOrderHeader].[OrderDate])))&lt;BR&gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp; |--Clustered Index Scan(OBJECT:([Sales].[SalesOrderHeader].[PK_SalesOrderHeader_SalesOrderID]), &lt;B&gt;WHERE:([E].[ManagerID]=[Sales].[SalesOrderHeader].[SalesPersonID])&lt;/B&gt;)&lt;/P&gt;
&lt;P&gt;While the original plan computed the number of orders and the most recent order date for all employees, this plan only computes this information for a single employee and for that employee's manager.&amp;nbsp; Observe the WHERE predicates on the clustered index scans of the SalesOrderHeader table.&amp;nbsp; Moreover, notice that this plan uses scalar stream aggregates instead of the hash aggregates in the original plan.&amp;nbsp; Finally, in this example, the optimizer chose scans, but if the tables involved were larger and the predicates were more selective, it would have chosen index seeks to avoid scanning the entire table.&lt;/P&gt;In my next post I'll take a look at what is perhaps the most important use of CTEs: recursive CTEs.&lt;img src="http://blogs.msdn.com/aggbug.aspx?PostID=5512974" width="1" height="1"&gt;</description><category domain="http://blogs.msdn.com/craigfr/archive/tags/Subqueries/default.aspx">Subqueries</category><category domain="http://blogs.msdn.com/craigfr/archive/tags/Common+Table+Expressions/default.aspx">Common Table Expressions</category></item></channel></rss>