Notes from SQL Server Premier Field Engineer

Automatically refreshing your Excel reports

Yuriy — Fri, 23 Oct 2009 22:40:00 GMT

Many organization have lots of Excel reports and pivot tables whose data comes from SQL Server. As data on the backend database changes you would need to repopulate your excel books with updated information. Doing it by hand can be very tedious manual process for the reporting analyst and there are ways to automate it. The simplest way is to create a SQL Agent job which refreshes your books on a scheduled basis. VBScript below does exactly that. Just add a step to your refresh job for every report you want to refresh substituting "<path to your excel file>" with location of your excel file on some file share.

Set oWorkBook = oExcel.Workbooks.Open("<path to your excel file>")

Now you saved yourself a whole lot time and can start working on more interesting things!

Cheers, -Yuriy

Script below

Function Main()
Dim oExcel
Dim oWorkBook
Dim oWorksheet
Dim oPivotTable
Dim IStartedExcel

On Error Resume Next
Set oExcel = GetObject("Excel.Application") 'will attach to an existing instance if it is there.
oExcel.Quit

'Instantiate excel object; disable alert messages
Set oExcel = CreateObject("Excel.Application")
oExcel.DisplayAlerts = false

'when debugging set visible = true
oExcel.visible = true

'open workbook we wish to refresh
Set oWorkBook = oExcel.Workbooks.Open("<path to your excel file>")
oWorkbook.EnableConnections
oWorkbook.RefreshAll
oExcel.CalculateUntilAsyncQueriesDone
oWorkbook.Save
oWorkBook.Close

oExcel.Quit
Set oExcel = Nothing

'Main = DTSTaskExecResult_Success

End Function

Interpreting SQL Server execution plans

Yuriy — Tue, 06 Oct 2009 01:08:00 GMT

As many of you may know, most of the time performance problems with SQL server come from bad queries/execution plans. That is why it is important to know how to interpret them. Most of you are likely peeked on graphical execution plans in SQL Server once or twice and familiar with physical operators for index seeks, table scans and joins but there are much more operators. In case you ever wondered or looked for a complete list of operators SQL Server implements here it is from deep inside of SQL Server documentation - books online. I personally find it very handy as it helps me to understand what SQL server does to return results and how to improve my query...

Hope you find it useful too

Indexing fact tables

Yuriy — Tue, 25 Aug 2009 23:17:00 GMT

Recently I was working on datawarehouse project during which i had to automate generation of indexes for fact tables. Tuning SQL Server relational warehouse is a complex task. As you may already know fact tables have lots of foreign keys into dimension tables in star schema in relational warehouse. These foreign keys should be your first candidates for indexes as most of the queries in a relational warehouse are joins on a surrogate keys of fact tables with dimensional tables.

Script below will help you to automate generation of indexes on fact tables from foreign keys. Just change <fact table name> to an actual fact table name.

Hope it saves you some time in tuning your datawarehouse

select 'if exists (select name from sys.indexes where name=''IX_<fact table name>_'+c.name+

''') drop index <fact table name>.IX_<fact table name>_'+c.name

from sys.foreign_key_columns fkc

inner join sys.foreign_keys fk on fk.object_id=fkc.constraint_object_id

inner join sys.columns c on c.column_id=fkc.parent_column_id

where fk.parent_object_id=object_id('<fact table name>')

and c.object_id=object_id('<fact table name>)

select 'create nonclustered index IX_<fact table name>_'+c.name+ ' on <fact table name>('+c.name+')' from sys.foreign_key_columns fkc

inner join sys.foreign_keys fk on fk.object_id=fkc.constraint_object_id

inner join sys.columns c on c.column_id=fkc.parent_column_id

where fk.parent_object_id=object_id('<fact table name>')

and c.object_id=object_id('<fact table name>'')

Enjoy it and until next time...

SQL Server 2008 failover clustering whitepaper

Yuriy — Mon, 13 Jul 2009 21:02:00 GMT

Microsoft finally published SQL Server 2008 failover clustering whitepaper which can be found here

Microsoft SQL Server 2008 Virtual Labs

Yuriy — Sun, 26 Apr 2009 02:21:00 GMT

Hi guys!

While travelling around Australia and helping Microsoft customers to get work done with SQL Server, I was quite surprized to realize how many customers do not know about Microsoft SQL Server 2008 Virtual Labs . There you can remotely test drive latest and greatest innovations in Microsoft data platform without leaving your own desk (some registraton may be requird though). How cool is that?

Give it a try, fellows, and enjoy!

-Yuriy

SQL Server 2008 SP1 is out

Yuriy — Thu, 09 Apr 2009 01:33:00 GMT

and can be downloaded here

SQL Server 2005/2008 database tuning - picking low hanging fruits - part I

Yuriy — Tue, 31 Mar 2009 03:25:00 GMT

Hello fellow readers!

It has been a while since I posted. Sorry about that - have been busy travelling around customers in Australia.

Let me write a few words how you can easily increase performance of your database application without doing much hard work. Sounds cheezy you might think. Not at all. I would like to cover two easy options you have to collect those low hanging fruits of optimizing SQL Server performance. First shows the power of dynamic management views (DMVs) and second uses tools which have been shipping with SQL Server for a long time - Database Tuning Advisor and SQL Profiler. Let's cover these approaches one at a time.

1. Dynamic management views

SQL Server 2005/2008 has a set of dynamic management views maintained by query optimizer which tell you if there are any indexes missing. These DMVs are populated based on queries optimizer has seen so far since last time SQL Server booted up. They are: sys.dm_db_missing_index_details, sys.dm_db_missing_index_groups, sys.dm_db_missing_index_group_stats.

Simly open SQL Server Management Studio of your production server and type:

select * from sys.dm_db_missing_index_details

select * from sys.dm_db_missing_index_groups

select * from sys.dm_db_missing_index_group_stats

and look at the output. You maybe surprised how much entries are there - I suggested one customer to do a quick check and we saw ~400 entries in sys.dm_db_missing_index_details. This SQL Server environment had multiple databases and was shared between multiple applications, so don't be totally scared with a big number. Still, it shows that there is a big room for improvement to optimize performance of SQL Server installation. I am sure, you will see some in your environment as well. It is especially interesting to see what output is if you just recently migrated your databases from SQL Server 2000 to 2005 or 2008.

Lets have a quick look at the columns of the sys.dm_db_missing_index_details DMV. Our best friend SQL Server books online has this to say about them:

Column name	Data type	Description
index_handle	int	Identifies a particular missing index. The identifier is unique across the server. index_handle is the key of this table.
database_id	smallint	Identifies the database where the table with the missing index resides.
object_id	int	Identifies the table where the index is missing.
equality_columns	nvarchar(4000)	Comma-separated list of columns that contribute to equality predicates of the form: table.column = constant_value
inequality_columns	nvarchar(4000)	Comma-separated list of columns that contribute to inequality predicates, for example, predicates of the form: table.column > constant_value Any comparison operator other than "=" expresses inequality. For a complete list of comparison operators, see Comparison Operators (Database Engine).
included_columns	nvarchar(4000)	Comma-separated list of columns needed as covering columns for the query. For more information about covering or included columns, see Creating Indexes with Included Columns.
statement	nvarchar(4000)	Name of the table where the index is missing.

As one can see it is easy to build a CREATE INDEX statement based on the information in this DMV. Unfortunately, you have to do it by hand. I am not yet aware of simple tool which will autoscript all the entries into a SQL script. You may not necessarily want to create all indexes logged in the 'missing indexes DMV'. Try to correlate its entries with frequency of certain queries which can be collected through profiler or looked in the SQL Server plan cache DMVs like sys.dm_exec_query_stats or other sys.dm_exec_* DMVs. At the very least, discuss this output to your developers and implement some of them in your Dev environment and see if it helps your performance.

This is all for now. Please collect your juicy fruits!

Next time I will cover another method of using SQL Server profiler and Database Tuning Advisor.

SQL Server 2008 books online update is available

Yuriy — Wed, 04 Feb 2009 09:12:00 GMT

It can be downloaded here

SQL Server User Group in Sydney CBD

Yuriy — Tue, 27 Jan 2009 09:37:00 GMT

Victor Isakov (MCT, MCITP, MCM, SQL Ranger) is starting a new SQL Server User Group Meeting during lunchtimes in the Sydney CBD. It will run on the 1st Wednesday of every month, from 12:30 to 14:30, at Westpac Place.

This new user group will focus more on best practices, optimization, design and implementation - areas specifically geared towards the corporate sector. The meetings will be of high quality that you would typically see at conferences such as Microsoft Tech Ed, SQL PASS or SQL Connections. A number of industry experts and Microsoft personnel have committed to presenting at this new user group in 2009.

The meetings of this new user group will rotate through 3 core themes:

Database Administrator (DBA) Focuses on the responsibilities of the Database Administrator, covering areas such as disaster recovery, management, performance tuning, security and operational excellence.
Database Developer (DBD) Focuses on the responsibilities of the Database Developer, covering areas such as query writing, query tuning, indexing strategies, locking architecture and database design.
Database Architect (ARC) Focuses on the responsibilities of the Database Architect, covering areas such as high-availability, storage, consolidation / virtualization and scalability.

The details for the new SQL Server User Group are as follows:

Time	12:30 to 14:30
Agenda	12:30 - 13:00: Registration / Networking 13:00 - 13:15: News / Case Studies 13:15 - 14:30: Presentation
Location	Westpac Place 275 Kent St, Sydney, NSW 2000‎
Dates	4th Feb, 4th Mar, 1st Apr, 6th May, 3rd Jun, 1st Jul, 5th Aug, 2nd Sep, 7th Oct, 4th Nov, 2nd Dec
Contact	victor@sqlserversolutions.com.au

It is important to pre-register for the SQL Server User Group Meetings to help facilitate entry into Westpac Place.

You can come on the day without pre-registering, and are encouraged to "bring a friend" but it will take longer to gain entry.

So please email victor@sqlserversolutions.com.au if you plan to attend.

Otherwise please pass on this information to your colleagues and / or managers so as to encourage attendance. We would like to see the corporate sector encourage attendance and participation in this new SQL Server User Group.

Just tell your manager these meetings represent "free quality training", something that they will surely love in the current economic climate. J

The first three meetings for 2009 are as follows:

Topic: DBA: Best Practices for All DBAs to Follow

Date: 4th February

Description: As the popularity of SQL Server continues to grow, so is the demand for new SQL Server DBAs. Unfortunately, you can't “go to school” to learn how to become a DBA. Most DBAs learn from books, seminars, short classes, and trial and error. This session is designed to encourage a cross-pollination of ideas and best practices between DBAs from different organizations. It includes best practices from the trenches which can only be obtained through experience.

Topic: DBD: A Comprehensive Guide to Indexing in SQL Server

Date: 4th March

Description: With the release of SQL Server 2008 the database developer has more choices in their arsenal for optimally indexing database solutions. In this session we will explore the different indexing technologies that are available in SQL Server 2005 and SQL Server 2008, such as included, composite and filtered indexes. We will discuss the primary design considerations and tradeoffs made when implementing an indexing strategy and present more "exotic" scenarios such as how to implement custom hash indexes, how to combine filtered indexes with sparse columns, partitioned indexes and how to implement partitioned views with different underlying table indexes.

Topic: ARCH: Reducing Your Storage Cost in SQL Server 2008

Date: 1st April

Description: Are increasing storage costs hurting your bottom line as data sizes keep growing throughout your organization? In this session, we explore how SQL Server can help you lower your storage costs. We start with Microsoft SQL Server 2005 partitioning to transparently migrate older data to lower-cost storage tiers over time. We then discuss new storage cost-saving functionality in Microsoft SQL Server 2008 such as data compression, backup compression, and sparse columns, filtered indexes, file stream which can get you to 50% or more reduction in storage requirements for typical applications. Throughout the session, we share experiences, insights, and best practices from SQL Server customers and Microsoft-internal deployments. Learn how to make good use of these technologies in combination.

Free SQL Server 2005 High Availability EBook

Yuriy — Fri, 09 Jan 2009 07:40:00 GMT

Here is a cool perk from Red-Gate: a free EBook by Alan Hirt ‘Pro SQL Server 2005 High Availability’ http://downloads.red-gate.com/ebooks/DBA_ebook_pack.zip.

Enjoy!

-Yuriy

'Priority boost' sp_configure option and SQL Server failover clustering

Yuriy — Fri, 09 Jan 2009 07:05:00 GMT

Hello again !

This is yet another post on SQL Server failover clustering. Hope you don't mind :).

I was working the other day on the case and customer had a SQL Server 2005 cluster. As I was looking at output of sp_configure I noticed that it had 'priority boost' option set to 1 on a cluster. Microsoft discourages set 'priority boost' option on a cluster. This fact is documented in the following article. The reason for this is that it may cause failovers on a cluster system due to OS resource starvation since SQL Server runs at a higher priority.

-Yuriy

Renaming SQL Server cluster resource group

Yuriy — Fri, 09 Jan 2009 05:08:00 GMT

I was recently asked by a customer who just installed SQL Server 2005 with SP2 on 3 node failover cluster if he can rename SQL Server resource group after SQL Server is installed. The answer is Yes it is possible and does not affect functionality of SQL Server failover cluster. Though it is better to follow method described in this KB article method 3. Sometimes you may want to do this so that your cluster resource group name is meaningful eg reflects the name of the application which is hosted on SQL Server.

Hope this helps,

-Yuriy

Tools to troubleshoot SQL Server - part 1

Yuriy — Sun, 04 Jan 2009 08:29:00 GMT

Good day, everyone!

With this post I will start a series of posts describing tools which everyone of you can use to troubleshoot SQL Server problems. They are freely available and either come standard with SQL Server distribution or freely downlodable from world wide web. SQL Server Premier Field Engineers at Microsoft worldwide are using these to resolve issues.

Here they are:

1. Perfom (perfmon.exe) - Windows veteran executable to collect performance counters

2. SQL traces (server side) and SQL Profiler (located at <drive>:\Program Files\Microsoft SQL Server\<sqlversion>\Tools\Binn\PROFILER.EXE) to collect and analyse SQL server traces and correlate them with performance counters collected via perfmon.

3. Performance analysis of logs (PAL) a wonderful tool to use to analyze SQL Server performance counter logs against predefined thresholds defined for each Microsoft product by support engineers.

4. SQLDiag.exe - SQL traces and performance counters collection utility. Allows both to be collected together. Comes standard with SQL Server distribution and located at <drive>:\Program Files\Microsoft SQL Server\<sqlversion>\Tools\binn.

5. SQLNexus - a visualizer for output produced by SQLDiag.exe. Shows top N most expensive queries consuming CPU, IO and other resources.

6. Database Engine Tuning Advisor (DTA) - excellent tool to tune your database against predefined workload. Comes standard with SQL Server distribution and located in <drive>:\Program Files\Microsoft SQL Server\<sqlversion>\Tools\Binn\DTASHELL.EXE

7. Microsoft Best Practices Analyzer for SQL Server (BPA) - gathers data from Microsoft Windows and SQL Server configuration settings. BPA uses a predefined list of SQL Server 2005 recommendations and best practices to determine if there are potential issues in the database environment.

8. SQL Server Database Management Views - virtual tables in SQL Server to provide internal information on the state of the database engine

This is a list only for now. In next posts I will go in more details on them. Obviously, there are 3d-party tools out there but you have to pay for them.

I encourage you to explore these tools and see how you can apply them to your database environment.

This is all for now. Until next time.

Cheers,

-Yuriy

Beware of special characters in cluster resource/group names

Yuriy — Sat, 03 Jan 2009 05:36:00 GMT

Hello my fellow readers!

I want to bring to your attention one small thing which will likely save you time and relieve from lots of headaches when installing SQL Server on the cluster. I am talking about special characters like & < > " etc in cluster resource and group names. Sometimes it is tempting to name cluster resource groups for SQL Server under name of the application which SQL Server will be hosting. Say, application is Microsoft Office Communication Server (MS OCS) then you are likely to call this resource group OCS. If you intend to host multiple applications on SQL Server cluster (say OCS and Sharepoint) you may choose use & as a separator for a cluster resource name to reflect application names eg OCS & Sharepoint. Though it may seem as a good idea, you may not to do so as stated on MSDN:

"When naming a cluster group for your failover cluster installation, you must not use any of the following characters in the cluster group name:

Less than operator (<)
Greater than operator (>)
Double quote (")
Single quote (')
Ampersand (&)

Also verify that existing cluster group names do not contain unsupported characters."

So, before you proceed installing SQL Server on the cluster, please go ahead and check for special and unsupported characters in names of cluster resources and groups. It well might be that these groups and resources were created by somebody else for you because at times people who are configuring Windows clustering and SQL Server failover cluster are different and not aware of this issue.

And of course, you should all read MSDN or SQL Server Books Online for things to do before installing SQL failover clustering

SQL Server 2005 SP3 available

Yuriy — Sat, 03 Jan 2009 04:20:00 GMT

Micrsoft SQL Server 2005 SP3 is now available for download here . Fixes and changes made in SP3 are documented in KB article 955706.