Microsoft SQL Server Support Blog : SQL Best Practices

PART II - Data security enhancements in SQL Server 2005

sqlblog — Fri, 16 Feb 2007 19:50:00 GMT

In the previous edition (Data security enhancements in SQL Server 2005) , we talked about data encryption/decryption in general.

In this installment, let’s talk about key maintenance. It is important to ensure that we can decrypt our data, even in case of user mistake or corruption.

For this, we need to protect our keys in our database. We have couple of options to do this.

Master key and certificate

You can use the following steps to backup Master key and Certificates

· Backing Master key

BACKUP MASTER KEY TO FILE = 'path_to_file'

ENCRYPTION BY PASSWORD = 'password'

RESTORE MASTER KEY FROM FILE = 'path_to_file'

DECRYPTION BY PASSWORD = 'password'

ENCRYPTION BY PASSWORD = 'password'

[ FORCE ]

More information on the syntax is available here.

Backing Certificate

BACKUP CERTIFICATE certname TO FILE = 'path_to_file'

[ WITH PRIVATE KEY

(

FILE = 'path_to_private_key_file' ,

ENCRYPTION BY PASSWORD = 'encryption_password'

[ , DECRYPTION BY PASSWORD = 'decryption_password' ]

)

]

To restore a backed up certificate, use the CREATE CERTIFICATE statement. More information is available here.

Symmetric key

In order to maintain symmetric key properly and have the ability to restore it, you will have to use the same values for the following during key creation:

· algorithm,

· key_source,

· identity_value

Key can be protected by password or certificate, but values above should exactly be the same. You may also use different name of symmetric key.

Let’s take an example to better understand this.

SELECT * INTO Employee_symm FROM AdventureWorks.HumanResources.Employee

ALTER TABLE Employee_symm

ADD EncryptedNationalIDNumber varbinary(128);

--- Lets create symmetric key with protection by password first

CREATE SYMMETRIC KEY symm_key_combo

WITH

ALGORITHM = triple_des,

IDENTITY_VALUE = 'Example of encryption',

KEY_SOURCE = 'Put here some unique and long enough phrase.'

ENCRYPTION BY PASSWORD = 'Avc#ptNO$cf@o!';

--- Encrypt data

OPEN SYMMETRIC KEY symm_key_combo

DECRYPTION BY PASSWORD = 'Avc#ptNO$cf@o!'

UPDATE Employee_symm

SET EncryptedNationalIDNumber = EncryptByKey(Key_GUID('symm_key_combo'), NationalIDNumber);

--- Now let’s drop symmetric key

DROP SYMMETRIC KEY symm_key_combo

--- Re-create symmetric key with protection by certificate.

--- Also let’s use different name for symmetric key

CREATE SYMMETRIC KEY symm_key_combo_new

WITH

-- Values here should be exactly the same

ALGORITHM = triple_des,

IDENTITY_VALUE = 'Example of encryption',

KEY_SOURCE = 'Put here some unique and long enough phrase.'

--- Protection can be done by certificate now though

ENCRYPTION BY CERTIFICATE HumanResources037;

--- Now let’s decrypt here using new symmetric key

OPEN SYMMETRIC KEY symm_key_combo_new

DECRYPTION BY CERTIFICATE HumanResources037;

SELECT NationalIDNumber, EncryptedNationalIDNumber

AS 'Encrypted ID Number',

CONVERT(nvarchar, DecryptByKey(EncryptedNationalIDNumber))

AS 'Decrypted ID Number'

FROM Employee_symm;

CLOSE SYMMETRIC KEY symm_key_combo_new

DROP SYMMETRIC KEY symm_key_combo_new

DROP TABLE Employee_symm

So in order to be able to restore symmetric key following 3 values need to be set

ALGORITHM = triple_des,

IDENTITY_VALUE = 'Example of encryption',

KEY_SOURCE = 'Put here some unique and long enough phrase.'

Use the same values to re-create the symmetric key in future.

Hope this helps!

Posted By: Sergey Pustovit

Tools of the Trade: Part I – Universal Data Link Files

sqlblog — Fri, 16 Feb 2007 19:21:00 GMT

The other day I was working on an issue with one of our customers.

I had been having him run several different tests using a few of the tools that we use on the SQL Developer team on a daily basis. After about the third or fourth diagnostic we ran, he asked me something along the lines of: “how are we supposed to know about all these different tools?”

I didn’t have a good answer. Although many of the tools we use are well known, unless a person spends their days troubleshooting, they are unlikely to be familiar with all of them. I was not aware of most of them before I began working for Microsoft’s Product Support Services, and in the cases where I was familiar with the tool, I did not always know how to get the most mileage out of it.

In this series of posts, we’re going to take a look at some of the tools we commonly use, how to use them, and the types of issues they can be used to resolve.

Universal Data Link Files

One of the first things we try to do when troubleshooting a database connectivity issue is to establish just how “broken” things are.

In many situations, database connectivity is failing only when used by a specific user, for a specific application, or after taking a specific series of steps. People often are not aware of which is true for them; there are lots of systems out there running only a single program that utilizes a data back-end and if that’s not working properly, how do we know if the failure is system wide or application specific?

Enter the Universal Data Link (UDL) file. UDL files offer a very simple way to test the connectivity of a given Ole DB provider to a data source. The very simple instructions for creating and configuring a UDL file can be found here.

Once you’ve created and opened the UDL, you should start out on the “Providers” tab, and select the provider appropriate for you data source (e.g. “Microsoft Ole DB Provider for SQL Server” or “SQL Native Client” for SQL Server databases). The Connection tab allows you to set provider-specific details for connecting, such as the name of the server, the authentication mechanism, and the initial catalog.

If necessary, you can use the All tab to set values for provider specific properties (for example, you can set the Failover_partner for SQL Native Client here to test a Database Mirroring scenario). Once that’s complete, you can click the “Test Connection” button and observe the results.

There are a lot of different deductions you can make based on the result of your test. If the connection is successful, you know at least you are able to connect and authenticate to the datasource from that machine.

At this point, you’ll want to take a look at the differences between what your application is doing, and what the datasource is doing. One possibility is that your application is using a different data provider (An ODBC driver? A .Net Data Provider? JDBC? A different Ole DB provider?). Another difference is that often your application runs under the context of a different user account with different permissions (very common for connections made from services such as IIS).

If the connection failed, you’re probably running into a more system-wide issue. You should get a specific error, which will guide your troubleshooting from here. If you see something along the lines of “SQL Server does not exist or access is denied” then you might be a good time to use some other tools to diagnose possible network connectivity issues, like SQL Profiler to audit failed logins, or Netmon to take a look at what traffic is being sent across the wire (there will be posts to come discussing some of these tools).

Other errors will suggest different courses of action, for example checking out possible local permissions issues (Filemon, Regmon, ProcessMonitor). The good news is, now you’ve got a quick and easy way to reproduce your issue, without the additional complexity of an entire application.

UDLs are also very useful for constructing Ole DB connection strings. A UDL file is essentially a text file that holds a connection string. If you configure the UDL using the dialog to connect to your data source, you can then open the .udl file using Notepad and copy the connection string for use in your applications.

So, that’s an overview of how we use UDL files. Just about any time you have a database connectivity issue, your best first step is to see if you can connect using a UDL file. This will allow you to better define your problem, which really helps in deciding what your next troubleshooting steps should be. In future articles, we’ll look at some of the tools that you might use to further define your problem, and get closer to a solution.

Posted By : Justin Trobec

SQL Server 2000/2005 and DST changes

sqlblog — Mon, 29 Jan 2007 18:39:00 GMT

Background

In August of 2005 the United States Congress passed the Energy Policy Act, which changes the dates of both the start and end of daylight saving time (DST).

When this law goes into effect in 2007, DST will start three weeks earlier (2:00 A.M. on the second Sunday in March) and will end one week later (2:00 A.M. on the first Sunday in November) than what had traditionally occurred.

MS products impacted by this change

You can view a list of all Microsoft products that are impacted by this change here.

Impact on SQL Server

At the time of writing this article, only Notification Services is impacted by this change.

Microsoft has published a KB to help you ease the transition. It highlights the steps needed to be performed in Notification Services to accommodate this change.

Other than NS, please make sure your OS is patched as per the instructions here.

IMPORTANT

Please check this page at least weekly to ensure you have the latest information available.

Posted by : Sanchan Saxena

How can I limit the number of reboots when applying hotfixes and the like to SQL Server and OS?

sqlblog — Wed, 06 Dec 2006 20:16:00 GMT

Avoiding reboots after applying a Service Pack or a hotfix to SQL is a big concern to our customers.

In SQL 2005, we have made several enhancements to help you avoid a reboot after applying Service Packs and hotfixes.

To learn about these enhancements, read our MSDN article here.

Happy reading!

Posted By: Sanchan Saxena

MSDTC and SQL Server on a cluster

sqlblog — Fri, 06 Oct 2006 19:46:00 GMT

Through this blog, we will try to address commonly asked questions about MSDTC and SQL Server on a cluster. All this information is available in BOL and KB articles, but we will summarize it here for your convenience.

Do I need to install MSDTC on my SQL Server 2005 cluster?
Before installing SQL Server 2005 on a failover cluster, Microsoft strongly recommends that you install and configure Microsoft Distributed Transaction Coordinator (MS DTC) on Windows 2000 and Windows Server 2003 operating systems.

Failure to cluster MS DTC will not block SQL Server 2005 Setup, but SQL Server 2005 application functionality may be affected if MS DTC is not properly configured.
What is MSDTC used for?
SQL Server 2005 requires MS DTC in the cluster for distributed queries and two-phase commit transactions, as well as for some replication functionality.
Should MSDTC be a clustered resource?
Microsoft only supports running MSDTC on cluster nodes as a clustered resource. We do not recommend or support running MSDTC in stand-alone mode on a cluster. Using MSDTC as a non-clustered resource on a Windows cluster is problematic. This configuration is problematic because transactions could be orphaned and you may experience data corruption if a cluster failover occurs.
Should MSDTC be in its own resource group?
To help ensure availability between multiple clustered applications, Microsoft highly recommends that the MS DTC have its own resource group and resources. If MS DTC cannot be configured to have its own resource group, the recommended alternate choice is to use the Cluster group and Quorum drive.

Other Resources
Before Installing Failover Clustering in SQL 2005

How to configure Microsoft Distributed Transaction Coordinator on a Windows Server 2003 cluster

How to configure MSDTC in a Windows 2000 cluster environment

How to rebuild or move a MSDTC installation to be used with a SQL failover cluster

How to re-install MSDTC on Windows XP?

Hope this helps!

POSTED BY : Sanchan Saxena

SQL Server and Network Attached Storage (NAS)

sqlblog — Wed, 04 Oct 2006 04:54:00 GMT

Through this blog, we will address some of the commonly asked customer questions on SQL Server and Network Attached Storage (NAS).

Support for NAS database files with SQL Server
You can configure SQL Server to store a database on a networked server or NAS storage server. You need to enable trace flag 1807 before you can achieve this. For more information on setting up NAS with SQL Server, refer to KB 304261.
Are there any issues with NAS that I need to be aware of?
Before you decide to run your database files on NAS, make yourself aware of the performance and database consistency considerations for using NAS. These considerations are documented in detail at KB 304261.
What does Microsoft recommend – SAN or NAS?
Because of the risks of network errors compromising database integrity, together with possible performance implications that may result from the use of network file shares to store databases, Microsoft recommends that you store database files either on local disk subsystems or on Storage Area Networks (SANs).

Refer to KB 304261 for more information on this.
Does Microsoft certify any hardware that works with SQL Server?
Microsoft does not certify or validate that third-party products will work correctly with Microsoft SQL Server. Additionally, Microsoft does not provide any warranty, guaranty, or statement of any third-party product's fitness for use with SQL Server. This policy applies to hardware and software products. It is up to the third-party vendor of the product to provide any such warranty, guaranty, or statement of the product's fitness for use.

For more information, please refer to KB 913945.
If Microsoft doesn’t certify any third party hardware, how do I feel confident that I am running on good hardware?
SQL Server 2005 software works on the hardware certified for use with the Microsoft Windows operating system.

Any such hardware that is certified to work with Windows is posted at Products Designed for Microsoft Windows - Windows Catalog and HCL. You can use these hardware products from third party vendors with SQL Server.

Hope this helps!

POSTED BY : Sanchan Saxena

SQL Server databases are not supported on compressed volumes

sqlblog — Tue, 03 Oct 2006 03:53:00 GMT

It is important to understand that SQL Server database on compressed volumes are not supported. This blog will help you understand the differences between SQL 2000 and SQL 2005 w.r.t compressed drives.

SQL 2000

In SQL Server 2000, you could create a new database with its .mdf and .ldf files located on NTFS or FAT compressed drives. The interface doesn’t restrict you from doing this.

However, it is NOT a good idea and is NOT supported. For more details, refer to KB 231347.

SQL 2005

As opposed to SQL 2000, SQL 2005 doesn’t even allow you to place data and log files of Read/Write databases on compressed volumes. Let’s examine two scenarios.

· Let’s create a database called Compressed_DB with .mdf and .ldf files located on a compressed file system (E drive in my case here). You will get an error:

Create failed for Database 'Compressed_DB'. (Microsoft.SqlServer.Smo)

An exception occurred while executing a Transact-SQL statement or batch. (Microsoft.SqlServer.ConnectionInfo)

The file "E:\Compressed_DB.mdf" is compressed but does not reside in a read-only database or filegroup. The file must be decompressed.

CREATE DATABASE failed. Some file names listed could not be created. Check related errors. (Microsoft SQL Server, Error: 5118)

This is because a read/write database’s primary .mdf and .ldf files cannot be created on a compressed volumes in SQL 2005.

Also, if you try to restore a database backup (.bak) file of a read/write database with its .mdf or .ldf or both pointed on a compressed drive, you will again get an error as follows:

Restore failed for Server '<servername>'. (Microsoft.SqlServer.Smo)

System.Data.SqlClient.SqlError: The file "e:\Db.mdf" is compressed but does not reside in a read-only database or filegroup. The file must be decompressed. (Microsoft.SqlServer.Smo)

Log files should never be placed on compressed file systems. System database (master, msdb, model and tempdm) cannot be placed on compressed drives.

A word about Read-Only file-groups/Read-Only databases in SQL 2005

However, there are special scenarios in which SQL 2005 allows you to store the data and log files on compressed drives.

If your Read/Write database has read-only secondary file group, then you can host the secondary data files (.ndf) on a compressed drive in SQL 2005.

This functionality allows you to move historical data between your primary data file (.mdf) over to a secondary data file (.ndf) located on a compressed drive in a read-only file group, thereby saving disk space.
If you mark a database read-only, then you can place your primary data file (.mdf) also on a compressed drive.
Similarly, restoring a read-only database to a compressed volume is allowed.

For more information on read-only filegroups, refer to the BOL here.

In Conclusion

Because of performance overhead and database recovery issues (as described in KB 231347), it is not a good idea to run SQL Server databases on compressed drives.

Moreover, such installations are NOT supported.

Hope this helps!

POSTED BY : Sanchan Saxena

Why is a SPID blocking itself in sysprocesses? How is it possible?

sqlblog — Fri, 22 Sep 2006 05:42:00 GMT

After you apply SQL 2000 Service Pack 4, you might observe that a SPID is blocking itself, as reported in the output of sysprocesses.

If your custom scripts/applications monitor blocking by relying on the output of sysprocesses column, they will start reporting extensive blocking on your server.

However, there is more to it than meets the eye.

Reason

SQL 2000 SP4 reports more diagnostic information than earlier versions. In addition to lock waits, SQL 2000 SP4 now reports latch waits as well.

It is this new monitoring which could lead to the situation described above – a SPID blocking itself.

In majority of the cases, you can ignore this condition if either the value in the waittime column is low OR the waittype of the SPID is a latch waittype.

For more information on this, please refer to the KB article 906344.

POSTED BY : Sanchan Saxena

FAQ - Configuring Log Shipping in SQL Server

sqlblog — Wed, 20 Sep 2006 06:20:00 GMT

Through this blog, we will try to address common customer questions on log shipping in SQL 2005.

Can log shipping be configured between SQL 2000 and SQL 2005?

You cannot configure log shipping between SQL 2005 (as primary/secondary) and SQL 2000 (secondary/primary).|

To demonstrate this, let’s try to configure log shipping as below:

o SQL Server 2000 as the source/primary server and SQL Server 2005 as the secondary/destination server.

Use the Database Maintenance Plan Wizard to configure Log Shipping. During this wizard, you will reach the “Specify the Log Shipping Destinations” where you can click on ADD button to add the servers.

Under the Server Name drop down, you will ONLY see those servers that are already registered in your Enterprise Manager. Since you cannot register a SQL 2005 server under Enterprise Manager, it will never pop-up in the drop down menu, thereby not giving you any option to setup log shipping in this scenario.

Now, if you try using SQL Server 2005 Management Studio, you still cannot configure log shipping. Register a SQL 2000 server under Management Studio and right click on the SQL 2000 database.

Here you won’t find the option “Ship Transaction Logs” under TASKS. This is by design.

o SQL Server 2005 as the source server and SQL 2000 as the standby destination server.

In your SQL Server 2005 Management Studio, right click on a SQL 2005 database and go to TASKSà Ship Transaction Logs. When you enable the database for log shipping, you can add Secondary servers. If you try to add a secondary server that is SQL 2000, it will throw the following error message:

“servername” is not a valid secondary server instance because it is not the correct version. Secondary server instances must be servers running SQL Server 2005 or later.

By design, both these scenarios are not possible out of the box in SQL Server.

However, you can always implement your own custom solutions.

How do I upgrade my existing SQL 2000 log shipping configuration to SQL 2005?

You cannot directly upgrade a SQL Server 2000 log shipping configuration to SQL Server 2005. The Database Maintenance Plan Wizard, which was integral to log shipping in SQL Server 2000, is not used as part of the log shipping configuration in SQL Server 2005. As a result, log shipping stops functioning when you upgrade a server to SQL Server 2005.

However, you can migrate a SQL Server 2000 log shipping configuration while maintaining synchronization between your primary and secondary databases. This can be done by the methods described in the BOL topic Upgrading a SQL Server 2000 Log Shipping Configuration.
Can I run queries against the secondary/destination server so as to reduce the load on my primary server?

Yes, you can reduce the load on your primary server by using a secondary server for read-only query processing. To do this, the secondary database must be in STANDBY mode. You will not be able to run queries if the database is in NORECOVERY mode.

For more information on this, refer to the BOL topic Using Secondary Servers for Query Processing.

Hope this helps.

UPDATE on 9th Oct, 2006

Log shipping between the SQL 2005 x86 editions and corresponding 64bit editions (x64 and IA64) is supported.

POSTED BY : Sanchan Saxena

My database file is marked with automatic growth of XXXX% (e.g. 12800%). What’s going on?

sqlblog — Mon, 18 Sep 2006 21:06:00 GMT

You might run into a situation wherein the value of the auto-growth increment of your database files (.mdf, .ndf or .ldf) shows a very large value – e.g. 12800%

This happens if you:

Create the database by using a percentage value for the automatic growth increment of the database file (e.g 10%).
Then change the value for the automatic growth increment to be a fixed size value (e.g. change it to 50 MB).
Perform one of the following operations

Restart the SQL Server service.
Take the database offline, and then bring the database online.
Detach the database, and then attach the database.

If you are running:

SQL 2005 RTM (9.00.1399) à install the hotfix mentioned in KB 917887
SQL 2005 SP1 (9.00.1047) à install the hotfix mentioned in KB 919611

Hope this helps!

POSTED BY : Sanchan Saxena

Why is my transaction log file growing rapidly? How do I stop it from eating up all my disk space?

sqlblog — Mon, 18 Sep 2006 20:32:00 GMT

This is a very common scenario and generates quite a few calls for us. SCENARIO Transaction log file is growing rapidly and has consumed all the disk space on your drive. You don’t have any free disk space left and can no longer perform data modification...(read more)

Installing SQL 2005 using the 2 CDs

sqlblog — Mon, 18 Sep 2006 16:44:00 GMT

Another area where we get quite a few support calls. Lets address this today. SQL 2005 Media When you order SQL 2005 media, you receive either a DVD or 2 CDs. Installing from DVD Installing from DVD is pretty straightforward. Pop in the DVD in your drive...(read more)

How to change the startup accounts for SQL Server services?

sqlblog — Sun, 17 Sep 2006 23:40:00 GMT

Typically, you would set the startup accounts for SQL Server services (SQL Server, SQL Agent etc) during the startup. However, there are times when you might want to change it after installation. Many DBAs are so used to Services.msc console that they...(read more)