SQL Server Engine Tips : General

Differences between ISNULL and COALESCE

SQL Server Engine Team — Fri, 27 Jun 2008 06:55:00 GMT

I came across a question in the SQL Server MVP newsgroup recently about ISNULL and COALESCE usage. COALESCE basically translates to CASE expression and ISNULL is a built-in implemented in the database engine. Both ISNULL and COALESCE can be used to get the same results but there are some differences.

1. Data type determination of the resulting expression - ISNULL uses the first parameter type, COALESCE follows the CASE expression rules and returns type of value with highest precedence

2. The NULLability of result expression is different for ISNULL and COALESCE. ISNULL return value is always considered NOT NULLable (assuming the return value is a non-nullable one) whereas COALESCE is not. So the expressions ISNULL(NULL, 1) and COALESCE(NULL, 1) although equivalent have different NULLability values. This makes a difference if you are using these expressions in computed columns and creating key constraints or making return value of a scalar UDF deterministic so that it can be indexed.

Please note that I am referring to expressions that will alwahys return a non-NULLable value here. Otherwise, you can have ISNULL or COALESCE return NULL value just fine.

3. Validations for ISNULL and COALESCE is also different. For example, NULL value for ISNULL is converted to int whereas for COAELSCE you have to provide a type. Ex:

ISNULL(NULL, NULL) -- is int

COALESCE(NULL, NULL) -- Will throw an error

COALESCE(CAST(NULL as int), NULL) -- it valid and returns int

4. ISNULL takes only 2 parameters whereas COALESCE takes variable number of parameters

5. COALESCE is based on the ANSI SQL standard whereas ISNULL is a proprietary TSQL function

6. You could get different plans for queries using ISNULL & COALESCE if the expressions involve scalar sub-queries. This will make a performance difference and queries with COALESCE often fare worse here. See below repro script:

use tempdb

create table t1 ( i int );

create table t2 ( i int );

create table t3 ( i int );

set showplan_text on;

select isnull((select i from t1 where t1.i = t2.i), (select max(i) from t3))

from t2;

select coalesce((select i from t1 where t1.i = t2.i), (select max(i) from

t3))

from t2;

set showplan_text off;

drop table t1, t2;

DATALENGTH optimizations for LOB data types...

SQL Server Engine Team — Sat, 15 Jul 2006 02:25:00 GMT

DATALENGTH function in TSQL can be used to find the actual length in bytes of the data in a specific value. The value can be any of the data types. It is often used to determine length of LOB data type columns (text, ntext, image, varchar(max), nvarchar(max) and varbinary(max)) in a table. One of the question that comes up regarding use of DATALENGTH on LOB columns is whether it requires reading the entire value. For example, if the text or varchar(max) value is 16 MB in size does SQL Server need to read the entire value to determine the size. In this post, I will describe why SQL Server doesn't need to read the entire value and show you some mechanism to ascertain that easily in SQL Server 2005.

The default storage option for text/ntext/image data type in a table is to store it out of row. So the text/ntext/image value will take up 16-bytes per data row. And the value is stored in separate set of pages. The 16-bytes pointer points to the root node of the tree built of pointers that map the pages into the value fragments. The root structure also contains the size of the text/ntext/image value. Lastly by using the system stored procedure "sp_tableoption" with 'text in row' option, part of the text/ntext/image value can be stored in row for quicker access.

So the DATALENGTH function needs to access only the root structure for BLOB values stored out of row to determine the size and the in-row data if the 'text in row' option is set. This means that if your BLOB data is very large in size there is one additional IO incurred to read the root structure and the entire value need not be accessed to compute the size. This can be demonstrated by using the SET STATISTICS IO command in SQL Server 2005 which exposes new counters for LOB reads (logical, physical and read-ahead). The script below shows the amount of IO performed when only DATALENGTH function is used on a BLOB column vs reading the entire column value.

use tempdb
go
create table blob_t( i int not null primary key, t text not null )
go

--- Insert a row in the table:
insert into blob_t values( 1, replicate('x', 8000 ) )
go

-- Set 'text in row' option so we can create a value that is stored in-row:
exec sp_tableoption blob_t, 'text in row', 7000
go

-- Insert a row with text value that will be stored in-row:
insert into blob_t values( 2, replicate('x', 7000 ) )
go
set statistics io on
go

-- Find length of data stored in row #1
select datalength(t) from blob_t where i = 1

Table 'blob_t'. Scan count 0, logical reads 2, physical reads 0, read-ahead reads 0, lob logical reads 1, lob physical reads 0, lob read-ahead reads 0.
*/

-- Find length of data stored in row #2
select datalength(t) from blob_t where i = 2

Table 'blob_t'. Scan count 0, logical reads 2, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.

-- Read row #1
select * from blob_t where i = 1

Table 'blob_t'. Scan count 0, logical reads 2, physical reads 0, read-ahead reads 0, lob logical reads 5, lob physical reads 0, lob read-ahead reads 0.

-- Read row #2
select * from blob_t where i = 2

Table 'blob_t'. Scan count 0, logical reads 2, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.

*/
go
set statistics io off
go
drop table blob_t
go

As you can observe from the statistics io output, when the DATALENGTH function is used to determine the length of value in the row (i =1) that contains the text value out of row there is one extra read to locate the root structure. This is shown in the log logical reads counter output. Similarly, in case of the row (i=2) that contains the text value in row there is no extra IO to read any of the LOB pages and the size can be determined by reading the in-row data.

The same behavior can be observed for the new LOB data types in SQL Server 2005. For the new data types, the defaults are however different in that the LOB valued is stored in row by default (SQL Server determines the size of the in-row data automatically). The setting can be modified by using "sp_tableoption" with the option "large value types out of row". The script that demonstrates the behavior of DATALENGTH for varchar(max) data type column is shown below:

use tempdb
go
create table blob_t( i int not null primary key, t varchar(max) not null )
go

-- Add new rows. SQL Server will automatically determine size of in-row data
insert into blob_t values( 1, replicate(cast('x' as varchar(max)), 8000 ) )
insert into blob_t values( 2, replicate(cast('x' as varchar(max)), 8000*4 ) )
go

-- Set option large values to be stored out of row:
exec sp_tableoption blob_t, 'large value types out of row', 1
go

-- Insert a new row that will be stored out of row:
insert into blob_t values( 3, replicate(cast('x' as varchar(max)), 8000*8 ) )
go
set statistics io on
go
select datalength(t) from blob_t where i = 1

Table 'blob_t'. Scan count 0, logical reads 2, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.
*/

select datalength(t) from blob_t where i = 2

Table 'blob_t'. Scan count 0, logical reads 2, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.
*/

select datalength(t) from blob_t where i = 3

Table 'blob_t'. Scan count 0, logical reads 2, physical reads 0, read-ahead reads 0, lob logical reads 1, lob physical reads 0, lob read-ahead reads 0.

select * from blob_t where i = 1
/*

Table 'blob_t'. Scan count 0, logical reads 2, physical reads 0, read-ahead reads 0, lob logical reads 0, lob physical reads 0, lob read-ahead reads 0.

select * from blob_t where i = 2
/*

Table 'blob_t'. Scan count 0, logical reads 2, physical reads 0, read-ahead reads 0, lob logical reads 12, lob physical reads 0, lob read-ahead reads 0.

select * from blob_t where i = 3
/*

Table 'blob_t'. Scan count 0, logical reads 2, physical reads 0, read-ahead reads 0, lob logical reads 47, lob physical reads 0, lob read-ahead reads 0.

go
set statistics io off
go
drop table blob_t
go

I hope you found the information useful and also learnt some new techniques on using SET STATISTICS IO output in SQL Server 2005.

To summarize, the use of DATALENGTH function on LOB data type (text, ntext, image, varchar(max), nvarchar(max) and varbinary(max)) column values incurs one additional IO to access the root structure if the value is stored out of row. And the root structure contains the size of the entire value so the value need not be accessed or read in it's entirety to determine the size in case of out of row storage.

Renaming logins in SQL Server 2005...

SQL Server Engine Team — Mon, 10 Oct 2005 21:34:00 GMT

Did you know that SQL Server 2005 supports renaming logins? This can be done via ALTER LOGIN statement. I posted a sample about renaming sa login and disabling it before. But it is probably not obvious if you can do it for all logins. Ex:

ALTER LOGIN sqluser WITH NAME = [newuser];

You can also use this to rename Windows Logins that map to same SID.

See the ALTER LOGIN topic [ms-help://MS.SQLCC.v9/MS.SQLSVR.v9.en/tsqlref9/html/e247b84e-c99e-4af8-8b50-57586e1cb1c5.htm] in Books Online for more details. The topic can also be found in MSDN at http://msdn2.microsoft.com/ms245841.

SQL Server 2005 features that are dependent on Windows Server 2003...

SQL Server Engine Team — Fri, 07 Oct 2005 04:34:00 GMT

The question about what features are supported by SQL Server 2005 running on Windows Server 2003 comes up quite often. So below are some of the features that are depends on the OS and brief description about them.

1. Password policy/expiration check for SQL logins - CREATE LOGIN is a new DDL for creating SQL logins. It has two options called CHECK_POLICY and CHECK_EXPIRATION that can be used to enforce Windows password policies for SQL logins. This leverages the functionality provided by NetValidatePasswordPolicy() API

2. Hot add memory - This is a new feature in Windows Server 2003 that can help SQL Server 2005 indirectly
3. Dynamic AWE - SQK Server 2005 running on Windows Server 2003 uses dynamic AWE memory allocation. This works similar to regular memory management

4. 64-bit versions of SQL Server 2005 runs only on Windows Server 2003 64-bit (IA and X64) - Native 64-bit editions of SQL Server 2005 runs only on corresponding Windows Server 2003 64-bit editions

5. SOAP support - Native XML web services in SQL Server 2005 relies on HTTP API support which is provided by Windows Server 2003 and Windows XP Service Pack 2

6. Fast file initialization - Creating a new database of any size is almost instantaneous in SQL Server 2005 running on Windows Server 2003/XP. This requires the service account to have the SE_MANAGE_VOLUME_NAME permissions. With fast file initialization, disk content is overwritten when new allocation happens

7. SQL Writer service features that work with Volume Shadow Copy Services - Differential backup, differential restore, restore with move, database rename, copy only backup and auto-recovered Snapshots are supported only on Windows Server 2003 with Service Pack 1

In addition to these features that can benefit SQL Server installations on Windows Server 2003 platform, there are other inherent advantages. Windows Server 2003 is more scalaeable and secure than Windows Server 2000. These are just some of the things to keep in mind when choosing a platform for running SQL Server 2005. The link below contains some benefits on using Windows Server 2003 with SQL Server 2005:

http://www.microsoft.com/sql/techinfo/planning/winsvr2003benefits.mspx

Request for topics...

SQL Server Engine Team — Fri, 23 Sep 2005 05:07:00 GMT

Please post requests for topics or features that you would like to know about. This can be any of the SQL/TSQL language features or programmability in general. There are so many things to discuss about wrt SQL Server 2005 so it will be good to gauge interest of database developers/DBAs out there. Thanks.

Umachandar

Determine primary keys and unique keys for all tables in a database...

SQL Server Engine Team — Fri, 16 Sep 2005 21:27:00 GMT

With SQL Server 2005, there are new ways to obtain richer metadata in a database and more efficiently. We have introduced new catalog views that exposes all the metadata that SQL Server uses and can be created by various DDL statements. The older ANSI SQL style INFORMATION_SCHEMA views are also still available if you want to write portable queries. I already posted a tip about finding dependencies http://blogs.msdn.com/sqltips/archive/2005/07/05/435882.aspx between various objects using the new catalog views. Here are two queries that show you how to retrieve primary/unique key details for all tables in a database:

-- ANSI SQL compatible and works from SQL70 onwards:

select kcu.TABLE_SCHEMA, kcu.TABLE_NAME, kcu.CONSTRAINT_NAME, tc.CONSTRAINT_TYPE, kcu.COLUMN_NAME, kcu.ORDINAL_POSITION
from INFORMATION_SCHEMA.TABLE_CONSTRAINTS as tc
join INFORMATION_SCHEMA.KEY_COLUMN_USAGE as kcu
    on kcu.CONSTRAINT_SCHEMA = tc.CONSTRAINT_SCHEMA
   and kcu.CONSTRAINT_NAME = tc.CONSTRAINT_NAME
   and kcu.TABLE_SCHEMA = tc.TABLE_SCHEMA
   and kcu.TABLE_NAME = tc.TABLE_NAME
where tc.CONSTRAINT_TYPE in ( 'PRIMARY KEY', 'UNIQUE' )
order by kcu.TABLE_SCHEMA, kcu.TABLE_NAME, tc.CONSTRAINT_TYPE, kcu.CONSTRAINT_NAME, kcu.ORDINAL_POSITION;

-- SQL Server 2005 specific:

select s.name as TABLE_SCHEMA, t.name as TABLE_NAME

     , k.name as CONSTRAINT_NAME, k.type_desc as CONSTRAINT_TYPE
     , c.name as COLUMN_NAME, ic.key_ordinal AS ORDINAL_POSITION
from sys.key_constraints as k
join sys.tables as t
    on t.object_id = k.parent_object_id
join sys.schemas as s
    on s.schema_id = t.schema_id
join sys.index_columns as ic
    on ic.object_id = t.object_id
   and ic.index_id = k.unique_index_id
join sys.columns as c
    on c.object_id = t.object_id
   and c.column_id = ic.column_id
order by TABLE_SCHEMA, TABLE_NAME, CONSTRAINT_TYPE, CONSTRAINT_NAME, ORDINAL_POSITION;

Ordering guarantees in SQL Server...

SQL Server Engine Team — Wed, 20 Jul 2005 21:20:00 GMT

Ordering guarantees of queries in various context is a common source of confusion. For example, a common workaround to make the results from querying a view ordered is to introduce TOP 100 PERCENT and ORDER BY in the view definition. But this however does not guarantee order in the actual results sent to the client since the query optimizer will re-order operations to find more efficient query plans. Note that even though this topic applies to SQL Server 2005 most of the rules are valid for SQL Server 2000 too.

Here are the scenarios that guarantee ordering:

If you have an ORDER BY in the top-most SELECT block in a query, the presentation order of the results honor that ORDER BY request
If you have a TOP in the same SELECT block as an ORDER BY, any TOP computation is performed with respect to that ORDER BY. For example, if there is a TOP 5 and ORDER BY clause then SQL Server picks the TOP 5 rows within a given sort. Note that this does not guarantee that subsequent operations will somehow retain the sort order of a previous operation. The query optimizer re-orders operations to find more efficient query plans
Cursors over queries containing ORDER BY in the top-most scope will navigate in that order
INSERT queries that use SELECT with ORDER BY to populate rows guarantees how identity values are computed but not the order in which the rows are inserted
SQL Server 2005 supports a number of new "sequence functions" like RANK(), ROW_NUMBER() that can be performed in a given order using a OVER clause with ORDER BY
For backwards compatibility reasons, SQL Server provides support for assignments of type SELECT @p = @p + 1 ... ORDER BY at the top-most scope.

Conor Cunningham

Query Optimizer Development Lead