SQL Server Storage Engine : VLDB Maintenance

TechEd session video available...

Paul Randal - MSFT — Thu, 28 Jun 2007 03:16:00 GMT

The session I gave at TechEd this year on 'Secrets of Fast Detection and Recovery from Database Corruptions' was videotaped as part of the Its Showtime! TechEd program. The video is now available to watch at http://www.microsoft.com/emea/itsshowtime/sessionh.aspx?videoid=549. This is the same session I've been delivering to user groups around the world for the past few months but this time Kimberly joined me and played demo-monkey :-)

Enjoy!

What happens to non-clustered indexes when the table structure is changed?

Paul Randal - MSFT — Thu, 07 Jun 2007 19:52:00 GMT

Here’s a topic that’s cropped up several times during Q&As at TechEd this year – what happens to non-clustered indexes when changes are made to the underlying table? Are they always rebuilt or not?

Before we get into that discussion, I’ll give you a little background.

One way to describe a non-clustered index is whether it is a covering index or not. A covering index is one which has all the table columns necessary to satisfy a query, and so there is no need to go back to the underlying table (which I’ll call the base table) to fetch additional columns. Most non-clustered indexes are not covering indexes and so it is necessary for each non-clustered index record to include a link back to the corresponding base table record. This link takes various forms, depending on the structure of the base table:

Heap: A heap does not impose any logical ordering on the records within it, so the only way to find a particular record within a heap is to do a full table scan or to know the physical location of the record (i.e. which record slot on which data page) – called the physical record ID or physical RID. Clearly doing a full table scan every time a query needs to go from a non-clustered index record back to a heap is not feasible, so each non-clustered index record stores the physical RID of the corresponding heap record.
Clustered index: A clustered index does impose a logical ordering on the records within it and the ordering exactly matches the definition of the clustered index keys – known as the cluster key. The fastest way to find a particular record in a clustered index is to use the cluster key to navigate through the clustered index b-tree to find the exact record. So, non-clustered indexes over a clustered index include the cluster key of the base table record in each non-clustered index record. This is known as the logical RID. There is a twist though, depending on whether the clustered index is unique or not.

Non-unique clustered index: Every record in a clustered index HAS to be unique, otherwise there would be no way to deterministically navigate to a particular record using the index b-tree. In a non-unique clustered index, SQL Server has to add a special column, called the uniquifier, to each record, so that if multiple records have the same cluster key values, the uniquifier column will be the tie-breaker. This uniquifier column is added as part of the cluster key, and so it is also present in all non-clustered index records as part of the logical RID.
Unique clustered index: Every record in a unique clustered index is already unique and so no extra column is required in the cluster key.

Ok - background out of the way. Now let's discuss what happens to non-clustered indexes when various actions are performed to the base table.

SQL Server 2000

Going from a heap to a clustered index: Changing the fundamental structure of a table from a heap to a clustered index means you're changing the way that records can be looked up in the table, from physical to logical. This means that all the physical RIDs that are stored in the non-clustered index records need to be changed to logical RIDs. So - in this case, all non-clustered indexes are rebuilt after the clustered index has been built.
Going from a clustered index to a heap: This is the opposite of the case above - you're changing the way records are looked up from logical to physical. This means that all the logical RIDs that are stored in the non-clustered index records need to be changed to physical RIDs. So - in this case, all non-clustered indexes are rebuilt after the clustered index has been dropped.
Rebuilding a unique clustered index: This operation has no effect on the cluster key, so there is no need for the non-clustered indexes to be rebuilt. Various early Service Packs of SQL Server 2000 had bugs that caused this behavior to be broken - and this is the cause of much of the confusion around this behavior.
Rebuilding a non-unique clustered index: In SQL Server 2000, when a non-unique clustered index (which contains uniquifier columns) is rebuilt, all the uniquifier values are regenerated. This essentially means that the cluster keys have changed and so all the non-clustered indexes are rebuilt after the clustered index rebuild has completed.
Changing the clustered index schema: This is simple - any change to the clustered index schema that changes the cluster keys will cause all the non-clustered indexes to be rebuilt.

SQL Server 2005

Going from a heap to a clustered index: This is the same as SQL Server 2000 - all non-clustered indexes are rebuilt, regardless of whether the clustered index was created online or offline.
Going from a clustered index to a heap: Again, all non-clustered indexes are rebuilt, regardless of whether the clustered index is dropped online or offline.
Rebuilding a unique clustered index: This is the same as SQL Server 2000 - the cluster keys aren't changing and so the non-clustered indexes are not rebuilt.
Rebuilding a non-unique clustered index: Aha! This is different from SQL Server 2000. SQL Server 2005 will RE-USE the old uniquifier values so the cluster keys don't change. This means that non-clustered indexes are NOT rebuilt in this case - that's very cool!
Changing the clustered index schema:

Changing the cluster key: This behavior has to be the same as SQL Server 2000 - any change to the cluster keys forces all the non-clustered indexes to be rebuilt after the new clustered index has been created.
Changing anything else (e.g. partitioning it or moving it): This is one of the cases that confuses people in SQL Server 2005. Applying a partitioning scheme to a clustered index, or moving it to a different filegroup, doesn't change the cluster key at all - so there's no need to rebuild the non-clustered indexes.

So the rule of thumb to remember is: if the physical or logical RID contained in the non-clustered index records needs to change then all the non-clustered indexes need to be rebuilt. Its as simple as that.

Hopefully this will help to clear up some confusion - let me know of any questions!

Quick list of VLDB maintenance best practices

Paul Randal - MSFT — Mon, 30 Apr 2007 18:44:00 GMT

One evening last week I sat down with Kimberly for 5 minutes to come up with a top-of-our heads list of VLDB maintenance concerns for a company migrating a multi-TB database to SQL Server 2005. This isn't in any way based on the VLDB survey I've been doing (see previous posts) but is a common-sense list of things that everyone should do. People really liked the list so I'm posting it here. Maybe we should turn this into a book???

Hope this helps.

Have page checksums turned on
Make sure auto-stats update is turned on
Pay attention to index fragmentation

Logical fragmentation only affects read-ahead performance
Only rebuild/defrag indexes that have this will help
Low page density affects IO throughput and memory usage
Low page density could be a sign of page-splits, so investigate the cause
If you defrag instead of rebuild, make sure you manually update stats
Be wary of doing large index maintenance jobs if you use log shipping or DBM

They contribute to large log backups
Index rebuilds are always full-logged when DBM is present

Make sure all indexes are actually needed

See http://blogs.msdn.com/sqlserverstorageengine/archive/2007/04/20/how-can-you-tell-if-an-index-is-being-used.aspx
Run adequate consistency checks

Even a VVVVLDB can be checked for corruptions

See http://blogs.msdn.com/sqlserverstorageengine/archive/2006/10/20/consistency-checking-options-for-a-vldb.aspx
Have a disaster recovery plan

Test it before you have to use it
Make sure the most junior DBA can follow it
See http://blogs.msdn.com/sqlserverstorageengine/archive/2007/04/17/example-corrupt-database-to-play-with.aspx

Understand your SLAs

See http://blogs.msdn.com/sqlserverstorageengine/archive/tags/Service+Level+Agreements/default.aspx

Have a backup strategy that allows you meet your SLAs

E.g. Weekly full backups with no HA solution only won’t allow you to meet zero data-loss SLAs

Make sure tempdb is optimized for the workload
Make sure you’re managing the transaction log correctly

E.g. being in full recovery mode with no log backups means you’ll eventually run out of space on the log drive
Don’t have multiple log files – no need
Don’t let auto-grow go nuts – causes VLF fragmentation which leads to horrible perf

Don’t run database shrink

See http://blogs.msdn.com/sqlserverstorageengine/archive/2007/04/15/how-to-avoid-using-shrink-in-sql-server-2005.aspx
Don’t rely on auto-grow
Pro-actively manage file growth but have auto-grow on as a safeguard

Consider turning on Instant Initialization to speed up file growth and restores

See http://www.sqlskills.com/blogs/kimberly/2007/03/04/InstantInitializationWhatWhyAndHow.aspx

Can you mix-n-match backup devices?

Paul Randal - MSFT — Mon, 23 Apr 2007 03:15:00 GMT

I was asked this question yesterday and didn't know the answer so thought it would be good for a quick post. Can you use devices from your main and mirrored backup media sets together to perform a restore?

The code below creates a single-device backup and then examines it.

BACKUP DATABASE AdventureWorks

TO DISK = N'c:\mediaset1device1.bck'

WITH FORMAT, STATS;

RESTORE HEADERONLY FROM DISK=N'c:\mediaset1device1.bck';

The BackupSize in the output is 172,044,800 bytes and the on-disk size of the file is 164MB. Now let's use two devices and see what happens.

BACKUP DATABASE AdventureWorks

TO DISK=N'c:\mediaset1device1.bck', DISK=N'c:\mediaset1device2.bck'

MIRROR TO DISK=N'c:\mediaset2device1.bck', DISK=N'c:\mediaset2device2.bck'

WITH FORMAT, STATS;

RESTORE HEADERONLY FROM DISK=N'c:\mediaset1device1.bck';

The BackupSize is now 344,113,152 bytes (roughly double what it was previously because we've mirrored the whole backup) and the on-disk size of each file is 82MB (half of what it was for a single-device). This proves that the backup has been split over the two devices. Now let's try to mix devices from the two backup media sets and see if it's possible:

RESTORE DATABASE AdventureWorks

FROM DISK=N'c:\mediaset1device1.bck', DISK=N'c:\mediaset2device2.bck'

WITH REPLACE, STATS;

And it works fine - excellent! That's the whole point of having mirrored backups.

The second part of the question was about whether backup device types can differ between media sets in the same backup. The answer to this is no - as documented in Books Online. All the backup devices involved in a single backup, regardless of whether they're part of a mirror media set or not, must be of the same type and have similar characteristics.

Next up - can it hurt to have thousands of filegroups?

How can you tell if an index is being used?

Paul Randal - MSFT — Fri, 20 Apr 2007 19:25:00 GMT

Whenever I’m discussing index maintenance, and specifically fragmentation, I always make a point of saying ‘Make sure the index is being used before doing anything about fragmentation’.

If an index isn’t being used very much, but has very low page density (lots of free space in the index pages), then it will be occupying a lot more disk space than it could do and it may be worth compacting (with a rebuild or a defrag) to get that disk space back. However, usually there’s not much point spending resources to remove any kind of fragmentation when an index isn’t being used. This is especially true of those people who rebuild all indexes every night or every week.

You could even go so far as to say if a non-clustered index isn’t being used, why is it there at all? Extra non-clustered indexes drag down performance in a number of ways. Consider a non-clustered index called IX_MyNCIndex on the table MyTable:

Any time a record is inserted into MyTable, a matching record is inserted into IX_MyNCIndex. This is a bunch of extra IOs, plus maybe even a page-split.
Any time a record is deleted from MyTable, the matching record in IX_MyNCIndex must be deleted. Extra IOs again.
Any time a record in MyTable is updated:

If MyTable has a clustered index, and the clustered index key value changes, then the matching record in IX_MyNCIndex must be updated. Extra IOs again.
If any of the non-clustered index key values changes, or any of the INCLUDEd column values changes, then the matching record in IX_MyNCIndex must be updated. Extra IOs again.

If a clustered index is created on MyTable, then IX_MyNCIndex has to be rebuilt to include the logical RIDs rather than the physical heap RIDs. Lot of extra IOs.

That’s a significant amount of extra IOs to maintain each extraneous non-clustered index.

So, how can you tell if an index is being used? There are a few different ways in SQL Server 2005 – the one I want to discuss in this post is using the sys.dm_db_index_usage_stats DMV.

This DMV exposes the information that is tracked about index usage (as the name suggests). It does not generate any information itself; it just returns info from a cache inside SQL Server. This cache is empty when the server instance starts, and is not persisted across instance restarts. All cache entries for indexes in a database are removed when that database is closed. So, the cache tracks usage information about indexes since the database they are part of was last opened (either manually or as part of instance start-up).

The cache tracks the following info for each index (for user queries and system queries):

· The number of times it was used in a seek operation (either looking up a single row, or doing a range scan) along with the time of the last seek.

· The number of times it was used in a scan operation (e.g. a select * operation) along with the time of the last scan

· The number of times it was used in a lookup operation (this means a bookmark lookup – where a non-clustered index does not fully cover a query and additional columns must be retrieved from the base table row) along with the time of the last lookup.

· The number of times it was used in an update operation (this counts inserts, updates, and deletes) along with the time of the last update.

Let’s have a look at its use.

SELECT * FROM sys.dm_db_index_usage_stats;

Unless you've just re-started your instance, you'll see a bunch of output from this, representing all index activity since the instance/databases started. If you're interested in whether an index is being used, you can filter the output. Let's focus in on a particular table - AdventureWorks.Person.Address.

SELECT * FROM sys.dm_db_index_usage_stats

WHERE database_id = DB_ID('AdventureWorks')

and object_id = OBJECT_ID('AdventureWorks.Person.Address');

You'll probably see nothing in the output, unless you've been playing around with that table. Let's force the clustered index on that table to be used, and look at the DMV output again.

SELECT * FROM AdventureWorks.Person.Address;

SELECT * FROM sys.dm_db_index_usage_stats

WHERE database_id = DB_ID('AdventureWorks')

and object_id = OBJECT_ID('AdventureWorks.Person.Address');

Now there's a single row, showing a scan on the clustered index. Let's do something else.

SELECT StateProvinceID FROM AdventureWorks.Person.Address

WHERE StateProvinceID > 4 AND StateProvinceId < 15;

SELECT * FROM sys.dm_db_index_usage_stats

WHERE database_id = DB_ID('AdventureWorks')

and object_id = OBJECT_ID('AdventureWorks.Person.Address');

And there's another row, showing a seek in one of the table's non-clustered indexes.

So, its easy to look at the index usage for particular tables and indexes. But how can you monitor this over time? This is easy too - let's see how.

First we need to create our own table to store snapshots of the DMV output.

IF OBJECTPROPERTY(object_id(N'master.dbo.MyIndexUsageStats'), 'IsUserTable') = 1

DROP TABLE dbo.MyIndexUsageStats;

SELECT GETDATE () AS ExecutionTime, *

INTO master.dbo.MyIndexUsageStats

FROM sys.dm_db_index_usage_stats WHERE database_id=0;

Next we need to take a baseline snapshot of the DMV output.

INSERT master.dbo.MyIndexUsageStats

SELECT getdate (), * FROM sys.dm_db_index_usage_stats;

And now simulate a few operations and take another snapshot of the DMV:

SELECT * FROM AdventureWorks.Person.Address;

SELECT StateProvinceID FROM AdventureWorks.Person.Address

WHERE StateProvinceID > 4 AND StateProvinceId < 15;

INSERT master.dbo.MyIndexUsageStats

SELECT getdate (), * FROM sys.dm_db_index_usage_stats;

And look at the filtered contents of our snapshot table:

SELECT * FROM master.dbo.MyIndexUsageStats

WHERE database_id = DB_ID('AdventureWorks')

and object_id = OBJECT_ID('AdventureWorks.Person.Address');

You should see four rows - two from the baseline snapshot and two from the final snapshot. If you ran just the statements above, you'll see that the user_scans count for the clustered index has increased by two, and the user_seeks count for the non-clustered index has increased by one.

So this is a pretty simple example of how you can track index usage. By putting something like this into a regularly run script you can tell which indexes aren't being used and could be candidates for less-regular index maintenance or removal altogether.

Let me know how you get on.

(Btw - still looking for more people to fill out my VLDB Maintenance survey...)

How to avoid using shrink in SQL Server 2005?

Paul Randal - MSFT — Mon, 16 Apr 2007 09:45:00 GMT

Late night blog post to round out spring-break vacation...

A number of customers I’ve spoken to in the last few weeks have been making use of database or file shrink in situations where they don’t really need to. There are few recurring scenarios I’ve seen:

Deleting a lot of data and then taking a backup – how to make the backup smaller without running shrink?
Emptying a file before removing it
Moving a file/filegroup/database to read-only
Running ALTER INDEX … REBUILD and then running shrink to reclaim the space used for the rebuild.

The last scenario is particularly nasty because shrink effectively reverses the effect of an index rebuild or reorganize (see this post for more on the shrink algorithm).

So, with these scenarios demanding some way to reduce space usage, how can you do it without using shrink? The answer depends on whether your base tables are clustered indexes or heaps.

Clustered Indexes

The trick here is to rebuild the clustered indexes into new filegroups, using the CREATE INDEX … WITH DROP_EXISTING statement. This does the same as a regular index rebuild, but you can alter the location of the index to a new filegroup (or to start using a partitioning scheme that defines a filegroup layout) and you can do it online (in Enterprise Edition).

If you’re moving the filegroup or database to be read-only, then you should set the FILLFACTOR to be 100% when you rebuild the indexes.

If you’re not changing the index schema, you don’t need to worry about the non-clustered indexes, as they will not be automatically rebuilt. You will want to manually rebuild them into the new filegroup(s) in the same, or just drop and recreate them (which could be tricky if they’re enforcing constraints).

Once you’ve moved everything out of the old filegroup, you can just remove it, and you’re done without having to execute shrink at all.

Heaps

The first thing to preferably do is drop all the non-clustered indexes, or at least those that aren’t enforcing constraints that would be in danger of being violated while an enforcing non-clustered index was dropped. This is because when shrink moves a heap page, all related non-clustered index rows have to be updated; because heaps have physical row identifiers (see my previous post on shrink for more details).

Once you’re rid of the non-clustered indexes, you’re free to run shrink. Heaps are not ordered, so cannot become logically fragmented – so the problem of shrink causing fragmentation doesn’t affect heaps. Then you have two choices:

Run shrink with EMPTYFILE into a new filegroup and then recreate the non-clustered indexes; or
Shrink the existing filegroup with NOTRUNCATE (to create a bunch of contiguous free space in the database files), then recreate the non-clustered indexes in the contiguous space, and then, if you’re making the filegroup or database read-only, run shrink with TRUNCATEONLY (to give back extra space to the operating system)

Summary

With a little thought, shrink can be avoided. I know of several customers who have tried this approach successfully, and saved themselves lots of resource usage in the process.

Let me know how you get on.

SP2 Maintenance Plan bugs fixed

Paul Randal - MSFT — Fri, 13 Apr 2007 03:31:00 GMT

From the tools team here in Redmond:

We recently posted updates to address an issue in two SP2 Maintenance Plan
tasks. The KB article describes the symptoms as follows:

FIX: The Check Database Integrity task and the Execute T-SQL Statement task
in a maintenance plan may lose database context in certain circumstances in
SQL Server 2005

In Microsoft SQL Server 2005 Service Pack 2 (SP2), the following tasks in a
maintenance plan may lose database context in certain circumstances:
. Check Database Integrity
. Execute T-SQL Statement
For example, if a Check Database Integrity task is configured to check
integrity on multiple databases, the task repeatedly checks the default
database.

For the Execute T-SQL Statement task, this problem occurs if the following
conditions are true:
. The Execute T-SQL Statement task contains multiple batches.
. The batches are separated by the GO statement.
. No batches use the USE statement to specify the database.
. The statements in each batch do not contain the database name.
In this scenario, those statements in the second batch and in succeeding
batches will be executed against the default database.

This fix is shipped in two different packages:
. The GDR release is available via http://support.microsoft.com/kb/934458
. The QFE release is available via http://support.microsoft.com/kb/934459

What is the difference between a GDR and QFE release? GDRs are a new means
of providing support. GDR fixes are reserved for those issues identified by
SQL Server support as important enough to install on every instance of SQL
Server. QFEs are used for the majority of fixes where the effects of the
problem are not widespread or severe enough to warrant a GDR.

GDRs and QFEs are associated with certain build numbers. Build numbers 3043
through 3149 are reserved for GDR releases. Build numbers 3150 and up are
reserved for QFEs. By reserving lower build numbers for GDR releases, users
can continue to take advantage of GDR fixes, without having to install QFEs,
which aggregate all the fixes shipped since the most recent service pack.
Once a QFE is installed, GDRs will no longer update your system, because the
build number is too low, and you have to install QFEs to pick up new fixes.

Why is this fix a GDR? This fix was promoted to GDR because users that
relied on the Check Database Integrity task to validate their databases were
not informed of database integrity issues that they expected to be detected.
The fix also addresses issues in the Execute T-SQL task, but, since these
issues typically lead to script failure, duly noted by SQL Server, the T-SQL
issues alone were not enough to justify the GDR.

Those users who now, or in the future, rely on the Check Database Integrity
Task or the Execute T-SQL task in a maintenance plan should install the GDR.
Others may skip the GDR.

VLDB Maintenance - quick addendum

Paul Randal - MSFT — Tue, 10 Apr 2007 20:19:00 GMT

So it seems that Microsoft's email system barfs when some people try to send email with 'survey' in. And I was unclear how big a database I'm interested in. To be honest, anything over 100 GB or so starts to require careful maintenance planning.

So, if you tried to send email and it bounced, or were unclear how big a DB I'm interested in, please send me mail - prandal@microsoft.com.

Thanks

VLDB Maintenance Practices and Problems?

Paul Randal - MSFT — Mon, 09 Apr 2007 04:11:00 GMT

It’s survey time again. I’ll be doing lots around VLDB maintenance in the coming months so I’d like to get more info from all of you about what’s happening in the field. I’ll be working on Katmai features, strategizing about Katmai+, blogging on maintenance topics, planning lectures, including a session for TechEd – VLDB Maintenance Q&A – and much more.

I have questions about your maintenance practices (e.g. backups, disaster recovery, fragmentation, file growth/shrink, stats, and so on). Additionally, I’d like to know about problems you have with VLDB maintenance, for databases around 100GB or larger. If you’re interested, you can do the following:

1. Just let me know what maintenance problems you have, or if you have time;

2. Let me know you want to fill in the survey and I’ll send it to you

What about incentives? By doing #1, you’ll be providing valuable feedback to the SQL dev team and letting me know what the most pressing issues are to discuss in lectures and the blog. By doing #2, if you fill in the survey, then in return for your time I’ll send you the latest copy of the Always-On DVD (by post). The DVD has 16 hours worth of hands-on labs from which to learn and has been very popular with those who’ve received it. In either case, I (or my team) should be able to help you with your problems – through email, the blog or maybe a conference call with you and your team.

So, if you’re interested in doing the survey, let me know – prandal@microsoft.com – otherwise just send me email describing your maintenance problems. [Edit - don't put 'survey' in the title or body of the message - seems that our email servers are filtering those out as spam]

In a few weeks I’ll summarize the results and start blogging answers etc

Look forward to hearing from you!