SQL Server Storage Engine : Shrink

How to avoid using shrink in SQL Server 2005?

Paul Randal - MSFT — Mon, 16 Apr 2007 09:45:00 GMT

Late night blog post to round out spring-break vacation...

A number of customers I’ve spoken to in the last few weeks have been making use of database or file shrink in situations where they don’t really need to. There are few recurring scenarios I’ve seen:

Deleting a lot of data and then taking a backup – how to make the backup smaller without running shrink?
Emptying a file before removing it
Moving a file/filegroup/database to read-only
Running ALTER INDEX … REBUILD and then running shrink to reclaim the space used for the rebuild.

The last scenario is particularly nasty because shrink effectively reverses the effect of an index rebuild or reorganize (see this post for more on the shrink algorithm).

So, with these scenarios demanding some way to reduce space usage, how can you do it without using shrink? The answer depends on whether your base tables are clustered indexes or heaps.

Clustered Indexes

The trick here is to rebuild the clustered indexes into new filegroups, using the CREATE INDEX … WITH DROP_EXISTING statement. This does the same as a regular index rebuild, but you can alter the location of the index to a new filegroup (or to start using a partitioning scheme that defines a filegroup layout) and you can do it online (in Enterprise Edition).

If you’re moving the filegroup or database to be read-only, then you should set the FILLFACTOR to be 100% when you rebuild the indexes.

If you’re not changing the index schema, you don’t need to worry about the non-clustered indexes, as they will not be automatically rebuilt. You will want to manually rebuild them into the new filegroup(s) in the same, or just drop and recreate them (which could be tricky if they’re enforcing constraints).

Once you’ve moved everything out of the old filegroup, you can just remove it, and you’re done without having to execute shrink at all.

Heaps

The first thing to preferably do is drop all the non-clustered indexes, or at least those that aren’t enforcing constraints that would be in danger of being violated while an enforcing non-clustered index was dropped. This is because when shrink moves a heap page, all related non-clustered index rows have to be updated; because heaps have physical row identifiers (see my previous post on shrink for more details).

Once you’re rid of the non-clustered indexes, you’re free to run shrink. Heaps are not ordered, so cannot become logically fragmented – so the problem of shrink causing fragmentation doesn’t affect heaps. Then you have two choices:

Run shrink with EMPTYFILE into a new filegroup and then recreate the non-clustered indexes; or
Shrink the existing filegroup with NOTRUNCATE (to create a bunch of contiguous free space in the database files), then recreate the non-clustered indexes in the contiguous space, and then, if you’re making the filegroup or database read-only, run shrink with TRUNCATEONLY (to give back extra space to the operating system)

Summary

With a little thought, shrink can be avoided. I know of several customers who have tried this approach successfully, and saved themselves lots of resource usage in the process.

Let me know how you get on.

How does your schema affect your shrink run-time?

Paul Randal - MSFT — Thu, 29 Mar 2007 20:58:00 GMT

For part two of my short series on data file shrinking, I want to look at how elements of your schemas can cause shrink to take much longer to run. In SQL Server 2005, three things in your schema will drastically affect the run-time of data file shrink.

LOB data

By LOB data, I mean any large-value data-type (e.g. text, image, varchar(max), XML). The problem here is with the way that LOB values are stored. They’re usually stored off-row, which means they’re stored in separate text pages from the table or index rows that they’re part of. Each row has a physical link to the LOB value(s) on the different text page(s).

The big catch is that the link is one-way only – from the data/index row to the LOB value – there is no backwards link from the LOB value back to the ‘owning’ data/index row. So, if a LOB value needs to be moved to a new page as part of the shrink process, how can the owning data/index row be updated with the new physical location?

Not easily is the answer. Because there’s no backwards link from the LOB value back to the owning row, shrink has to scan all rows that could own the text value (i.e. all records in the partition, index, or base table that the text page is part of). As you can imagine, this is a slow process and it has to happen for every LOB value that is moved.

Note: This was the same in SQL Server 2000 as well.

Row-overflow data

This is quite non-obvious. Row-overflow data is where you have a schema that allows row sizes larger than can fit in a page (see here for more details and here for how to examine the on-disk structure of row-overflow data values). The key point is that row-overflow data is stored in exactly the same way as LOB values, and so the same algorithm kicks in during shrink when a page with one or more of these values needs to be moved.

Heaps with non-clustered indexes

Non-clustered index rows have to have a link back to the matching base table (heap or clustered index) record, so that queries that have used non-covering non-clustered indexes can retrieve the desired columns from the base table record. For tables with a clustered index, this link is a logical link – the clustering key of the matching base table record. For tables without a clustered index (i.e. just a heap), this link is a physical link – the actual physical location of the matching base table record.

So, if a clustered index data-page moves, then nothing needs to change in the non-clustered indexes, because the clustering keys of the records on that page haven’t changed – only their location. However, if a heap data-page needs to be moved, than all the matching non-clustered index records need to be updated with the new location of the base table records!

On SQL Server 2000, this isn’t as bad as it seems because the Storage Engine could do this manually itself. However, we made a bunch of architectural changes in SQL Server 2005 that means that the Storage Engine no longer does this. Now the Query Processor is the only part of the SQL Server Engine that can update non-clustered index records, so when the Storage Engine wants to move a heap page in SQL Server 2005, it needs to ask the Query Processor to update the non-clustered index records for it. It turns out that this is a lot slower than the SQL 2000 method.

Summary

More reasons not to use shrink. Next up – how to avoid using shrink in SQL Server 2005.

Turn AUTO_SHRINK off!!

Paul Randal - MSFT — Wed, 28 Mar 2007 15:13:00 GMT

This week's topic is data file shrinking. I've seen lots of mis-information in the last few weeks and I've had a bunch of questions about it. First up is auto-shrink.

In my opinion, this feature causes way more problems than it solves (in fact, I can't think of a single problem it solves) and should be removed from the product. Remember I'm talking about auto-shrink, not manual shrink. Post a comment or drop me a line if you think there's a scenario where it's required.

So why?

The algorithms that shrink use are basically brute force. It starts at the end of the data file, picks up allocated pages and moves them as near to the front of the data file as it can, fixing up all the links as it goes. This is fine as long as there are no indexes involved - if there are, every time shrink moves a non-clustered index leaf-page, or a clustered index data-page, it's causing fragmentation. Yes, any shrink (apart from a TRUNCATEONLY) can cause fragmentation - this is not widely known, although every opportunity I get to explain this I do. I updated the Books Online for DBCC SHRINKDATABASE in SQL Server 2005 to call this out as a reason not to run shrink in general. So, although this affects all shrink operations, its still a very good reason not to run shrink automatically.
You can't control when it kicks in. Although it doesn't have any effect like long-term blocking, it does take up a lot of resources, both IO and CPU. It also moves a lot of data through the buffer pool and so can cause hot pages to be pushed out to disk, slowing things down further. If the server is already pushing the limits of the IO subsystem, running shrink may push it over, causing long disk queue lengths and possibly IO timeouts.
You can't control when it kicks in - it will start up every 30 minutes and try to shrink one of the databases that has AUTO_SHRINK turned on.
You're likely to get into a death-spiral of auto-grow then auto-shrink then auto-grow then auto-shrink... (in my experience, if someone is using auto-shrink, they're most likely using and relying on auto-grow too). An active database usually requires free space for normal operations - so if you take that free space away then the database just has to grow again. This is bad for several reasons:

Repeatedly shrinking and growing the data files will cause file-system level fragmentation, which can slow down performance
It wastes a huge amount of resources, basically running the shrink algorithm for no reason
Auto-grow itself can be bad, especially if you're using SQL Server 2000 (or don't have Instant File Initialization turned on - see this recent post from Kimberly's blog) where all allocations to the file being grown are blocked while the new portion of the file is being zero-initialized.

So - if that hasn't convinced you to make sure its turned off, I don't know what will. For databases created on SQL Server 2005, the AUTO_SHRINK option is off by default. You can use ALTER DATABASE yourdb SET AUTO_SHRINK OFF for all others, especially the MODEL database, so new databases don't inherit the setting unwittingly.

Next up - a look inside the algorithm for 2005 and how you may see the run-time of shrink increase unexpectedly, depending on your schema...