SQL Server Engine Tips : Dynamic Management View

OBJECT_NAME enhancement and OBJECT_SCHEMA_NAME addition in SQL Server 2005 SP2

SQL Server Engine Team — Sat, 24 Mar 2007 02:24:46 GMT

SQL Server 2005 SP2 has an important enhancement to OBJECT_NAME metadata function and a new OBJECT_SCHEMA_NAME metadata function. I will first describe the old functionality to give some context and demonstrate how the new features help a lot.

Please note that the examples uses DMVs that looks at metadata in each database or plan cache so performance of the queries depends on your environment, number of objects in the database, auto open/auto close setting of database, size of buffer pool or plan cache etc. So exercise caution if you run the query on a production database and it might be prudent just to restrict it to a specific database to see the results.

The OBJECT_NAME function is typically used to get the name of an object by specifying the object identifier (object_id value). For example, the query below returns the usage stats of indexes in the AdventureWorks database:

select OBJECT_NAME(i.object_id) as object_name, *
from sys.dm_db_index_usage_stats as i
where i.database_id = db_id('AdventureWorks');

Now, the above query will return the correct object name only if the query was run in the AdventureWorks database. This is because the metadata function OBJECT_NAME is database specific and uses information from the catalog tables in the current database. If you run the query in a different database (say master) then the names will either be wrong (because object_id may match a different object in master database) or NULL. So in order to get the correct object name, the query has to be executed in the context of the database you are interested (AdventureWorks in our example).

This DMV also returns information from multiple databases on the server. Let us now consider a scenario where you want to write a single query that returns say the top 5 entries from each database based on the number of user seeks that were performed on the index. The query will look like below:

select *
from (
select *, DENSE_RANK() OVER(PARTITION by database_id ORDER BY user_seeks DESC) as rnk
from sys.dm_db_index_usage_stats
) as i
where i.rnk <= 5;

To modify the query to return the user friendly object names instead of the identifiers, you will have to use dynamic SQL in older versions of SQL Server. This is because the OBJECT_NAME function always worked within the context of the current database. SQL Server 2005 SP2 removes this restriction and we now have an additional optional parameter that specifies the database id. This allows us to write a single query that can retrieve the OBJECT_NAME given the object identifier and database identifier of object in any database in a server. The modified query that lists the entries along with the object names is shown below:

select OBJECT_NAME(i.object_id, i.database_id) as objname, *
from (
select *, DENSE_RANK() OVER(PARTITION by database_id ORDER BY user_seeks DESC) as rnk
from sys.dm_db_index_usage_stats
) as i
where i.rnk <= 5;

Can you see how simple it is now to write queries that involve looking at object metadata in multiple databases? The enhancement to OBJECT_NAME now allows you to write complex queries that look at metadata in different databases without using dynamic SQL.

Lastly to round off the post, SQL Server 2005 has user schema separation and supports the ANSI SQL model of schemas. So it is always better to return object names along with their schema names. The OBJECT_SCHEMA_NAME metadata function can be used to return the schema name of a schema-scoped object like table or view by specifying the object identifier and optionally database identifier. The modified query below shows how to list the schema name also:

select OBJECT_SCHEMA_NAME(i.object_id, i.database_id) as schname
, OBJECT_NAME(i.object_id, i.database_id) as objname
, *
from (
select *, DENSE_RANK() OVER(PARTITION by database_id ORDER BY user_seeks DESC) as rnk
from sys.dm_db_index_usage_stats
) as i
where i.rnk <= 5;

The final query that shows the full object names along with the database in quoted format is shown below. The query lists the top 5 indexes in each database that had the most number of user_seeks performed.

select QUOTENAME(DB_NAME(i.database_id), '"')
+ N'.'
+ QUOTENAME(OBJECT_SCHEMA_NAME(i.object_id, i.database_id), '"')
+ N'.'
+ QUOTENAME(OBJECT_NAME(i.object_id, i.database_id), '"') as full_obj_name, *
from (
select *, DENSE_RANK() OVER(PARTITION by database_id ORDER BY user_seeks DESC) as rnk
from sys.dm_db_index_usage_stats
) as i
where i.rnk <= 5
order by i.database_id, i.rnk;

Below are some more examples on how to use the new functions.

-- Returns top 5 fragmented indexes in each database:

select QUOTENAME(DB_NAME(i.database_id), '"')
+ N'.'
+ QUOTENAME(OBJECT_SCHEMA_NAME(i.object_id, i.database_id), '"')
+ N'.'
+ QUOTENAME(OBJECT_NAME(i.object_id, i.database_id), '"') as full_obj_name
, *
from (
select *, DENSE_RANK() OVER(PARTITION by database_id ORDER BY avg_fragmentation_in_percent DESC) as rnk
from sys.dm_db_index_physical_stats(default, default, default, default, default)
where avg_fragmentation_in_percent > 0
) as i
where i.rnk <= 5
order by i.database_id, i.rnk;

-- Returns top 10 statements (in sql modules) per database that took the most elapsed time:

select coalesce(QUOTENAME(DB_NAME(t.dbid), '"'), '') /* NULL means resource db*/
+ N'.'
+ QUOTENAME(OBJECT_SCHEMA_NAME(t.objectid, t.dbid), '"')
+ N'.'
+ QUOTENAME(OBJECT_NAME(t.objectid, t.dbid), '"') as full_obj_name
, SUBSTRING(t.text, (t.statement_start_offset/2)+1,
((CASE t.statement_end_offset
WHEN -1 THEN DATALENGTH(t.text)
ELSE t.statement_end_offset
END - t.statement_start_offset)/2) + 1) AS statement_text
, t.*
from (
select *, DENSE_RANK() OVER(PARTITION BY t.dbid ORDER BY qs.total_elapsed_time) as rnk
from sys.dm_exec_query_stats as qs
cross apply sys.dm_exec_sql_text(qs.sql_handle) as t
where t.objectid is not null
) as t
where t.rnk <= 10
order by t.dbid, t.rnk;

Hope you find these enhancements in SQL Server 2005 SP2 useful and productive. Feel free to comment if you need more examples or more use cases.

Umachandar

Find Top N costly query plans in adhoc batches or modules...

SQL Server Engine Team — Thu, 06 Oct 2005 04:03:00 GMT

I encountered a problem about a query not using a particular indexed view in SQL Server 2005. To investigate this issue, I figured that I would go about writing a query using the execution related dynamic dynamic management views to get the cost of the query that should have used the indexed view. This proved to be an interesting adventure that also revealed some inconsistencies in how we expose certain metadata in the DMVs. On the bright side, the final query showcases the powerful new relational, TSQL and XQuery features of SQL Server 2005.

Let's start with brief outline of the DMVs that I planned to use:

sys.dm_exec_query_stats - Returns performance statistics for cached query plans. This contains one row per query plan so if a stored procedure or batch contains two SELECT statements you may get two rows here
sys.dm_exec_sql_text - Returns the text of the sql statement based on the SQL handle
sys.dm_exec_query_plan - Returns the showplan in XML format for a batch or module based on the plan handle

A simple query to get the elapsed time counters and SQL text of the batch/module is shown below. This query uses the new APPLY operator to join with a table-valued function passing the sql_handle value for each row.

select qs.sql_handle, qs.execution_count

, qs.total_elapsed_time, qs.last_elapsed_time

, qs.min_elapsed_time, qs.max_elapsed_time

, qs.total_clr_time, qs.last_clr_time

, qs.min_clr_time, qs.max_clr_time
from sys.dm_exec_query_stats as qs
cross apply sys.dm_exec_sql_text(qs.sql_handle) as st

This query can be modified to return the actual statement for which the stats are returned using the offset columns in sys.dm_exec_query_stats. Please note that the statement_start_offset and statement_end_offset columns are zero based, represents number of bytes and the end offset is -1 for last statement in the batch. In addition the text value returned by sys.dm_exec_sql_text is nvarchar(max) instead of text as documented incorrectly in Books Online. The modified query that accounts for these quirks is shown below:

select qs.sql_handle, qs.execution_count

, qs.total_elapsed_time, qs.last_elapsed_time

, qs.min_elapsed_time, qs.max_elapsed_time

, qs.total_clr_time, qs.last_clr_time

, qs.min_clr_time, qs.max_clr_time
, substring(st.text, (qs.statement_start_offset/2)+1

                        , ((case qs.statement_end_offset
                              when -1 then datalength(st.text)
                              else qs.statement_end_offset
                           end - qs.statement_start_offset)/2) + 1) as statement_text
from sys.dm_exec_query_stats as qs
cross apply sys.dm_exec_sql_text(qs.sql_handle) as st

Next, a similar query that gets the elapsed time counters and the XML showplan output for the batch/module.

select qs.sql_handle, qs.execution_count

, qs.total_elapsed_time, qs.last_elapsed_time

, qs.min_elapsed_time, qs.max_elapsed_time

, qs.total_clr_time, qs.last_clr_time

, qs.min_clr_time, qs.max_clr_time
, qp.query_plan
from sys.dm_exec_query_stats as qs
cross apply sys.dm_exec_query_plan (qs.plan_handle) as qp

Note that as I mentioned above, sys.dm_exec_query_stats returns a row for each query plan in a batch/module. The showplan output on the other hand contains the entire plan in XML format with each statement identified by the StatementID attribute which is simply a counter starting from the first statement in a batch. For example, if a stored procedure contains two queries then there will be two entries in the XML plan - one with StatementID = 1 and another with StatementID = 2. If there are additional variable assignments statements or DDLs or other TSQL statements they are also numbered appropriately in the order of appearance.

Consider the example stored procedure below:

create procedure TestProc
as
begin
declare @spid int
set @spid = @@spid
select count(*) from master.sys.sysprocesses where spid = @spid
select count(*) from master.sys.syslockinfo where req_spid = @spid
end

The showplan output will contain 3 StmtSimple nodes (fragments shown below with StatementText, StatementType and StatementId attributes only):

StatementText="create procedure testproc
as
begin
 declare @spid int
 set @spid = @@spid
 "
StatementId="1"
StatementType="ASSIGN"
StatementText=" select count(*) from master.sys.sysprocesses where spid = @spid
 "
StatementId="2"
StatementType="SELECT"
StatementText=" select count(*) from master.sys.syslockinfo where req_spid = @spid"
StatementId="3"
StatementType="SELECT"

The sys.dm_exec_query_stats will contain two rows one for each query only. So when we join sys.dm_exec_query_stats with sys.dm_exec_query_plan you will get the same showplan for both rows and we need to aggregate information from the plan or identify the appropriate StmtSimple nodes to get the cost of each query. So I generate first artificial counters by sorting and counting the statment_start_offset value from sys.dm_exec_query_stats. The same is done for the StmtSimple nodes in the query plan by selecting those of type "SELECT" and sorting them by StatementId value. Now this relative sequence numbers can be used to join and get the costing attributes for each query.

The final query that contains the logic to match the query stats with the plan nodes is shown below:

declare @top_n_plans int;
set @top_n_plans = 25;
with XMLNAMESPACES ('http://schemas.microsoft.com/sqlserver/2004/07/showplan' as sql)
select top(@top_n_plans)
       qst.text as sql_text, qp.query_plan
     , qst.statement_id
     , qst.statement_text as select_statement
     , qps2.statement_optimization_level
     , qps2.statement_optimization_early_abort_reason
     , (
        select sum(ro.SubTreeCost.value(N'@EstimatedTotalSubtreeCost', 'float'))
          from qp.query_plan.nodes(

N'//sql:Batch/sql:Statements/sql:StmtSimple/sql:QueryPlan[1]/sql:RelOp[1]'

             ) as ro(SubTreeCost)
       ) as Totalcost
     , qps2.statement_sub_tree_cost
     , qst.creation_time, qst.last_execution_time, qst.execution_count
     , qst.total_elapsed_time, qst.last_elapsed_time, qst.min_elapsed_time, qst.max_elapsed_time
     , qst.total_worker_time, qst.last_worker_time, qst.min_worker_time, qst.max_worker_time
     , qst.total_physical_reads, qst.last_physical_reads

, qst.min_physical_reads, qst.max_physical_reads
, qst.total_logical_writes, qst.last_logical_writes

, qst.min_logical_writes, qst.max_logical_writes
, qst.total_logical_reads, qst.last_logical_reads

, qst.min_logical_reads, qst.max_logical_reads

     , qst.total_clr_time, qst.last_clr_time, qst.min_clr_time, qst.max_clr_time
     , qst.sql_handle, qst.plan_handle
from (
        select *
             , substring(st.text, (qs.statement_start_offset/2)+1

                                , ((case qs.statement_end_offset
                                      when -1 then datalength(st.text)
                                      else qs.statement_end_offset
                                   end - qs.statement_start_offset)/2) + 1) as statement_text
             , ROW_NUMBER() OVER(PARTITION BY qs.plan_handle

                                     ORDER BY qs.statement_start_offset) as statement_id
          from sys.dm_exec_query_stats as qs
         cross apply sys.dm_exec_sql_text(qs.sql_handle) as st
     ) as qst
cross apply sys.dm_exec_query_plan (qst.plan_handle) as qp
cross apply (
        -- Since sys.dm_exec_query_stats doesn't have statement id,

-- we just sort the actual statement id from showplan for

        -- SELECT statements and join them with similar sequence number generated based
        -- on the statement start offset in sys.dm_exec_query_stats.
        -- This allows us to match the row from showplan with that of the query stats.
        -- This is a problem for batches containing multiple SELECT statements

        -- and hence this solution.
        select ROW_NUMBER() OVER(ORDER BY qps1.statement_id) as rel_statement_id
             , qps1.statement_optimization_level, qps1.statement_sub_tree_cost

             , qps1.statement_optimization_early_abort_reason
          from (
                select sel.StmtSimple.value('@StatementId', 'int')
                     , sel.StmtSimple.value('@StatementSubTreeCost', 'float')
                     , sel.StmtSimple.value('@StatementOptmLevel' , 'varchar(30)')
                     , sel.StmtSimple.value('@StatementOptmEarlyAbortReason', 'varchar(30)')
                  from qp.query_plan.nodes(

N'//sql:Batch/sql:Statements/sql:StmtSimple[@StatementType = "SELECT"]'

) as sel(StmtSimple)
) as qps1(statement_id, statement_sub_tree_cost

, statement_optimization_level, statement_optimization_early_abort_reason)
) as qps2
where qps2.rel_statement_id = qst.statement_id
-- and qst.text like ... /* can be used to filter only particular statemetns */
order by Totalcost desc, qst.plan_handle, qst.statement_id;

Lastly, let me highlight the new relational, TSQL and Xquery features of SQL Server 2005 as used in the query.

TOP clause - Support for variables or expressions in the TOP clause
APPLY operator - Used to join table against table-valued function in the query
ROW_NUMBER analytic function - To generate sequence numbers based on a particular order and optionally partition the rows into different groups
Xquery nodes method - To get the matching nodes based on a XPath expression for each instance of a query plan
Xquery value method - To get a particular Xml attribute as a typed column
WITH XMLNAMESPACES - To use namespace prefix in XPath expressions easily in a query

So run the query and enjoy the output / richness of metadata that SQL Server 2005 exposes. The Books Online topics and the MSDN links of the various features used in the query is below for convenience sake:

TOP (Transact-SQL) ms-help://MS.SQLCC.v9/MS.SQLSVR.v9.en/tsqlref9/html/da983c0a-06c5-4cf8-a6a4-7f9d66f34f2c.htm
FROM (Transact-SQL) ms-help://MS.SQLCC.v9/MS.SQLSVR.v9.en/tsqlref9/html/36b19e68-94f6-4539-aeb1-79f5312e4263.htm
ROW_NUMBER (Transact-SQL) ms-help://MS.SQLCC.v9/MS.SQLSVR.v9.en/tsqlref9/html/82fa9016-77db-4b42-b4c8-df6095b81906.htm
OVER Clause (Transact-SQL) ms-help://MS.SQLCC.v9/MS.SQLSVR.v9.en/tsqlref9/html/ddcef3a6-0341-43e0-ae73-630484b7b398.htm
sys.dm_exec_query_stats ms-help://MS.SQLCC.v9/MS.SQLSVR.v9.en/tsqlref9/html/eb7b58b8-3508-4114-97c2-d877bcb12964.htm
sys.dm_exec_sql_text ms-help://MS.SQLCC.v9/MS.SQLSVR.v9.en/tsqlref9/html/61b8ad6a-bf80-490c-92db-58dfdff22a24.htm
sys.dm_exec_query_plan ms-help://MS.SQLCC.v9/MS.SQLSVR.v9.en/tsqlref9/html/e26f0867-9be3-4b2e-969e-7f2840230770.htm
XML Showplans ms-help://MS.SQLCC.v9/MS.SQLSVR.v9.en/udb9/html/8f8fa597-1c57-496f-84cc-275c2b80fd8f.htm
Adding Namespaces Using WITH XMLNAMESPACES ms-help://MS.SQLCC.v9/MS.SQLSVR.v9.en/udb9/html/2189cb5e-4460-46c5-a254-20c833ebbfec.htm
WITH XMLNAMESPACES (Transact-SQL) ms-help://MS.SQLCC.v9/MS.SQLSVR.v9.en/tsqlref9/html/3b32662b-566f-454d-b7ca-e247002a9a0b.htm
nodes() Method (xml Data Type) ms-help://MS.SQLCC.v9/MS.SQLSVR.v9.en/udb9/html/7267fe1b-2e34-4213-8bbf-1c953822446c.htm
value() Method (xml Data Type) ms-help://MS.SQLCC.v9/MS.SQLSVR.v9.en/udb9/html/298a7361-dc9a-4902-9b1e-49a093cd831d.htm

Umachandar Jayachandran

Determining optimal MAXDOP setting from TSQL in SQL Server 2005

SQL Server Engine Team — Wed, 14 Sep 2005 22:02:00 GMT

For optimal performance of multi-processor installations, we recommend that the MAXDOP setting remain equal to the number of physical processors that are being used. For example, if the system is configured for two physical processors and four logical processors, MAXDOP should be set to 2. This is documented in the KB article:

http://support.microsoft.com/default.aspx/kb/322385

But there is no easy way to determine the number of physical processors in case of hyper-threaded CPUs for example from TSQL itself. Often you have to resort to using OS utilities or write small program using Win32 API to determine the logical processors in a hyper-threaded configuration or look at the BIOS or processor type.

In SQL Server 2005, there are set of views and table-valued functions that fall under the umbrella of Dynamic Management Views which expose lot of information about SQL Server, memory structures, SQLOS information and so on. More details can be obtained from the "Dynamic Management Views and Functions " topic in Books Online:

ms-help://MS.SQLCC.v9/MS.SQLSVR.v9.en/tsqlref9/html/cf893ecb-0bf6-4cbf-ac00-8a1099e405b1.htm

You can now derive the logical processors information from sys.dm_os_sys_info DMV in SQL Server 2005 easily. For example, the query below gives the optimal MAXDOP setting taking into account number of physical processors.

select case

when cpu_count / hyperthread_ratio > 8 then 8

else cpu_count / hyperthread_ratio

end as optimal_maxdop_setting

from sys.dm_os_sys_info;

This view also contains other interesting information like physical memory of the system, virtual memory available for processes, and number of schedulers. I will post more tips about the various interesting DMVs in SQL Server 2005. If you want to know about any particular area feel free to post a comment and I will post something on that.

[Modified: 20060517]

Recently, I came across a limitation in this system view. The hyperthread_ratio column is > 0 for the multi-core systems too. So there is no way to differentiate between a system with multi-core and hyperthreaded CPUs using the DMV. And since in the case of multi-core processor based systems, the MAXDOP value can be set to the number of CPUs the usefulness of the query is limited. The sure way to know the effect of MAXDOP setting is to test against your workload that involves parallel queries/operations.