Welcome to MSDN Blogs Sign in | Join | Help

An unwanted surprise when changing the SQL Server 2005 Service Account

When changing the Microsoft SQL Server 2005 Service Account you might get back this rather alarming error message box:

---------------------------
WMI Provider Error
---------------------------
The process terminated unexpectedly. [0x8007042b]
---------------------------
OK  
---------------------------

This is related to the problem documented in Books On Line on the page titled: 'Start and Restarting Services' (accessible in the index as 'starting SQL Server').  I quote from the paragraph that describes the problem:

"If SQL Server 2005 is shut down with open shared memory or a named pipe connection and is then restarted with the credentials of a different user, it may fail to start. This is normal behavior for connection handles controlled by the operating system. To restart SQL Server, end all SQL Server client processes manually or by restarting the computer. Possible errors include "Server shared memory provider failed to initialize. Error: 0x5" and "Server named pipe provider failed to listen on [ \\.\pipe\sql\query ]. Error: 0x5" (The pipe name may vary.)"

The problem described in the paragraph above manifests itself as the 'The process terminated unexpectedly. [0x8007042b]' error message box when using the SQL Configuration Manager to change the SQL Server (Engine) Service Account.  The workaround, as stated, is to close all client processes (connecting to the named pipe) or reboot the server. 

The problem is caused by an Operating System limitation which has been removed in Windows Vista.  In Windows Vista a change has been made to allow the tear down of a named pipe listener when only the server side connections are closed.  In Windows 2003 and earlier, there is a requirement to close down the client side connections as well, and this requirement prevents sqlservr.exe from creating a new properly ACLed named piper listener when the service is restarted with new Service Account credentials.

I hope you will find this post using your favorite search engine should you be panicked by this error message in the wild.  I will also push to get a KB article written with this information in it.

Posted by stuartpa | 7 Comments

A strange problem with 'ADD SIGNATURE' and the new SQLCMD command line utility

This one is a little odd:  But, when using the new 'ADD SIGNATURE' DDL in SQL Server 2005, if you are using SQLOLEDB through the SQLNCLI (SQL Native Client) protocol stack, i.e. the stack the new command line utility SQLCMD uses, you will get the following 'Incorrect syntax' error when trying to issue an 'ADD SIGNATURE' command:

C:\>sqlcmd -S .
1> ADD SIGNATURE TO object_signature BY CERTIFICATE my_certificate WITH PASSWORD =
'sd#@@$FDajdlksaj'
2> go
Msg 156, Level 15, State 1, Server <server>, Line 1
Incorrect syntax near the keyword 'ADD'.

No, you are not going mad, the syntax is fine; but, under the covers SQLNCLI is thinking the keyword 'ADD' is a user created stored procedure and is automatically prepending 'exec ' to the command.  You can see this with your own eyes by turning on profiler, you will see the following batch getting executed (notice the prepended 'exec '):

exec ADD SIGNATURE TO object_signature BY CERTIFICATE my_certificate WITH PASSWORD = 'sd#@@$FDajdlksaj'

This is a mistake in our SQLOLEDB protocol stack which we hope to fix in SQL Server 2005 Service Pack 1.  In the mean time, the workaround is simple, if not a little awkward.  You can prepend a ';' to your ADD SIGNATURE DDL i.e.:

C:\>sqlcmd -S .
1> ;ADD SIGNATURE TO object_signature BY CERTIFICATE my_certificate WITH PASSWORD =
'sd#@@$FDajdlksaj'
2> go

The ';' will confuse the parsing going on in SQLOLEDB so that it does not add the 'exec ', so everything will work fine.  We are not aware of any other DDL that if affected by this, but if you find any please let me know.

Again, this is only a problem with SQLOLEDB in SQLNCLI (SQL Native Client). OSQL is okay, because this uses ODBC, so is the old Query Analyzer.  The new SQL Management Studio is also fine, because this uses the new SQLClient managed stack.

Posted by stuartpa | 7 Comments

An error starting a SQL Server 2005 User Instance when using Remote Desktop

Late last week, I was using the new 'User Instance' support in SQL Express 2005.  I was connecting from home using Remote Desktop to my main office machine, and when trying to connect to SQL Express, I was becoming increasingly frustrated with the following error:

Error: 'Failed to generate a user instance of SQL Server due to a failure in starting the process for the user instance. The connection will be closed.'

It turns out there is a bug in Windows XP Service Pack 2 (on my office machine) which prevents connecting to User Instances.  It is documented in this knowledge base article (896613), although the article doesn't give the exact connection error message so I found it hard to find.  The article also doesn't mention this is specific to 'user instances', i.e. connecting to the parent instance is fine.  It is also SQL Express specific, regardless of whether the version of Express came with Visual Studio or you obtained it from somewhere else (i.e. the SQL Express download page here).

The problem exists if the session you are using was started using Remote Desktop.  i.e. you rebooted you office computer from home and logged back on to it (from home) using Remote Desktop.  The problem would exist at home over Remote Desktop and it would also still exist in the office if you just 'unlocked' the session which was started from home.  You would need to logoff and log back on again on your office machine.  Luckily there is a hotfix available for the bug (as described in the article), and it should be included in Windows XP Service Pack 3.

 

Posted by stuartpa | 8 Comments

Protecting the SQL Server 2005 'Service Master Key', the root of all encryption

Microsoft SQL Server 2005 introduces a new feature set, known internally as 'Secret Storage'.  Secret Storage is the overall umbrella technology for the data encryption facilities in the SQL Server database engine, including the key management story.  The key management of a SQL Server 2005 instance is rooted in the Service Master Key (SMK), a 128 bit 3DES key which is encrypted using the DPAPI and the service account credentials.  All other keys are protected in someway by the Service Master Key.  There is a diagram here of the key management hierarchy, and an excellent post with more details form Laurentiu here.

There is also a second encryption of the Service Master Key encrypted using the 'machine key'.  Both encryptions can be viewed using the sys.key_encryptions view in the 'master' database:

SELECT * FROM sys.key_encryptions WHERE key_id = 102

key_id thumbprint    crypt_type crypt_type_desc             crypt_property
102    0x01          ESKM       ENCRYPTION BY MASTER KEY    0xFFFFFFFFEC00000...
102    0x03          ESKM       ENCRYPTION BY MASTER KEY    0xFFFFFFFFEC00000...

The key internally recognized with the 0x01 thumbprint is an encryption of the Service Master Key using the (SQL Engine) service account credentials.  The key internally recognized with the 0x03 thumbprint is an encryption of the Service Master Key using the machine key credentials.

The Service Master Key is loaded into memory at database start up time.  The Service Master Key is first decrypted using the 0x01 key and service account credentials, if that fails, it is decrypted using the 0x03 key and local machine key credentials, and in this failure case, we then replace the original encryption of the Service Master Key using the service account credentials with a new encryption using the new service account credentials.

The second encryption of the Service Master Key is a recent addition because service account changes (of the SQL Engine service) might have led to loss of data.  Originally (and this is the behavior in CTP16 and previous) we were attempting to de-crypt and re-encrypt the SMK when the service account was changed using the SQL Configuration Manager.  However, there were occasions when the user could be surprised:

  • If a user uses the Windows Services Control Manager (SCM) to change the service account (instead of SQL Configuration Manager), the decryption and re-encryption would not happen (Windows SCM does not have any hooks that would allow us to do this), and if the old account was deleted before realizing this, access to the Service Master Key could be lost forever (along with all the data it encrypted).
  • If a Local User is used as the service account, and the password for that account is reset, access to the Service Master Key is lost (again, along with all the data it encrypted).
  • The code we had that decrypted and re-encrypted the SMK during Service Account change (using SQL Configuration Manager) might fail, and because Service Account change is not an atomic action (there is no rollback functionality) we could leave the SQL instance in a state where it has a new service account, but the Service Master Key was still encrypted using the old service account credentials.  Again, leading to loss of access to everything the SMK is protecting.

To avoid these situations we added the second encryption of the Service Master Key using the machine key.  Specifically, the machine key encryption uses DPAPI and passes the flag CRYPTPROTECT_LOCAL_MACHINE.  It also uses 256 bytes of 'Entropy' (randomly generated data), which is stored in the registry under:

   HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\<Instance Name>\Security\Entropy

This registry key is ACL'ed so only Box Administrators and the SQLServer2005MSSQLUser$<ServerName>$<InstanceName> group have access.  The Entropy, while not increasing the overall strength of the encryption, does increase the difficulty for another application, running under the same service account as the SQL Server service, to compromise the encryption key.

(NOTE: In SQL Express, when using a User Instance, the Entropy is stored in a file call Entropy.bin, in the user Local Settings\Application Data\... directory).

We have only just checked this feature in, it will be available in the RTM bits, however it did not make it into the RTM Books Online, (however, everything happens seamlessly in the background, so there are no configuration options to worry about).

We take protection of the Service Master Key very seriously, and while we have done our best to make sure you never get into a situation where you cannot decrypt it, we also strongly encourage you to backup the SMK using the 'BACKUP SERVICE MASTER KEY' DDL.

 

Posted by stuartpa | 4 Comments

SQL Server 2005 password change without server restart, small issue with 32 bit installs on x64 OS

In an earlier post, I said it will be possible to change the Microsoft SQL Server 2005 service account passwords without requiring a service restart.  Today we found a corner case.  It turns out, when you install SQL Server 2005 32 bit editions on an x64 Operating System, running in WOW64, and you don't install a native 64 bit side-by-side SQL installation as well, there is a bug in the operating system (in the API LsaCallAuthenticationPackage) that will require a service restart.

In this specific case, we will catch the specific error code from Windows (ERROR_NOT_SUPPORTED) and pop a dialog with the following text:

The requested operation is successful. Changes will not be effective until the service is restarted.

As the message says, you will need to restart the service so the new password can be used.  All other configurations are fine, and when the OS bug is fixed (I would hope the next service pack for Windows 2003/Windows XP), this dialog message will not be displayed.  Again, this is only for SQL Server 2005 32 bit installations, running on an x64 operating system (in WOW64) with no side-by-side native SQL Server 2005 64 bit install.

Posted by stuartpa | 0 Comments

Using SQL Server Management Studio (and Profiler) Tools with SQL Express 'User Instances'

In the SQL Server 2005 Express Edition, available for free here, there is new connection string option: 'user instance=true'.  When this connection string option is specified, the SQL Express instance you are connecting to (we call this the parent instance), spins of another instance of the sqlservr.exe process, as a client process (not a server service), running in the security context of the currently logged on user.  The connection object passed back to the client is to this new user instance (we also call this the 'child instance'), running as a client process in the user's security context. 

Internally (in the SQL Engine Group), we call this feature RANU, 'Run As Normal User', it enables each user to have their own sqlservr.exe process, sandboxed to their own security context.

For troubleshootig purposes, it maybe useful to use your SQL Server Management Studio and/or SQL Server Profiler tools (obtained when purchasing a Workgroup/Standard/Enterprise edition) to connect to the 'user instance'.

To do this, in the directory:

C:\Documents and Settings\<username>\Local Settings\Application Data\Microsoft\Microsoft SQL Server Data\<instance name>

There will be a file called 'error.log'.  Note: This file is called 'ERRORLOG' in all other editions of SQL Server.

Load up the 'error.log' file in notepad, and look for the line:

2005-09-04 20:55:00.42 Server      Server local connection provider is ready to accept connection on [ \\.\pipe\37B74C8E-45AE-4B\tsql\query ].

You can use the string: \\.\pipe\37B74C8E-45AE-4B\tsql\query, (the hex numbers after the \pipe\ will be different on your machines), in the 'Connect' dialog boxes for SQL Server Management Studio and SQL Profiler. 

I can't say we have fully tested it, but I have used these tools extensively with SQL Express 'User Instances' without problem.  I have found it useful for troubleshooting issues specific to 'user instances'.

Posted by stuartpa | 1 Comments

Change the SQL Server Service Account password without Server Restart

A couple of weeks ago, we checked-in a feature to Microsoft SQL Server 2005, that allows the various Service Account passwords to be changed, using the SQL Server Configuration Manager, without having to restart the respective services.

NOTE: It is going to be very important that you never use the plain old Windows Service Control Manager (SCM) to manipulate SQL Services.  The SQL Server Configuration Manager does a lot more work in the background to keep security consistent across the installation. 

Previous to this feature, when changing the service account password, if you did not restart the service, after a while (i.e. after the cached authentication tickets have expired), you would notice the following error message when the sqlservr.exe process tried to access remote resources:

Logon failure: unknown user name or bad password.

The above error message requires you to stop and restart the SQL service so the new Service Account password can be read in.  With this recent check-in, you will no longer see the 'Logon failure' error message, the new password will be used, without having to stop and restart the service.

NOTE: SQL Server Configuration Manager does not change the password with the Windows Domain.  That still has to be done using the 'Windows Security' dialog (i.e. Ctrl+Alt+Del).

This has been an often requested feature, that will increase service up time.  It will be in CTP16 (which we release very shortly).

Posted by stuartpa | 0 Comments

Setup Secure SQL Server Service Broker across databases in less than 10 seconds!

 

One of the most interesting new features in SQL Server 2005 is Service Broker.  If you need more details on what it is and why you would want to use it, I will let the expert tell you.  But, if you want to get it setup and configured on your machine now then look no further. 

 

Below is a single T/SQL script, which you can cut and paste into a Query Window, that will have you sending messages from one database to the other in less time than it takes to say, “give me Secure-SQL-Server-Service-Broker now” (try and say that 5 times in quick succession).

 

It's free   The reason why I think SQL Service Broker is going to be so interesting is because it enables cheap, easy, and secure deployment of hub-spoke applications.  Why?  Service Broker comes free with the SQL Server Express edition, and by using certificate based dialog- and transport-encryption, you can setup secure message based communications with many (potentially 1,000s, if you use the message forwarding capabilities to fan in/out) branch locations that report into a central hub. 

 

By using the new public key/private key certificate based encryption and identity in SQL Server 2005, you don’t have to set up complicated cross branch/company/organization infrastructures (Active Directory, SSL, Kerberos, SQL Authentication etc.).  NOTE: The hub does have to run a full version of SQL Server, e.g. Standard or Enterprise (The SQL Server Express Service Broker can only send through/to a full SQL Server edition).  I have a feeling there are a lot of problems that can be easily solved with the platform we have built here, it will be interesting to see how people capitalize on this. 

 

I couldn’t find a sample anywhere on the web of how to set up Service Broker with Certificates and CREATE USER WITHOUT LOGIN IN.   So, in this quick sample, we use CERTIFICATEs, AUTHORIZATION on objects, The Database MASTER KEY, USERs WITHOUT LOGINs, and QUEUEs.  Because of this reliance on other features, setting up Service Broker is not quite as simple as doing your first CREATE TABLE.  In fact, it is about as easy as the first time you tried to transitively close a four-way join!  Still, once you have done it a few times, it seems to make sense.

 

Service Broker borrows from the world of Service Orientated Architecture (SOA), the concepts of Initiators and Targets (for the Client/Server among us, you might say the Initiator is the Client and the Target is the Server, but that would make the SOA folks cringe, so we won’t say it). 

 

5 stages to glory   The 5 Stages to cross-database Secure Service Broker glory using certificate based dialog security:

 

1. TARGET - Set up the target database, export a certificate (with public key)

2. INITIATOR - Set up the initiator database, export a certificate (with public key) and import the target certificate

3. TARGET - Set up the target database, import the certificate of the initiator user

4. INITIATOR - Create a stored proc to send messages, and send your first message

5. TARGET - Check the message got to the Target database

 

If you cut and paste all 5 stages into a single Query window (running against an edition other than SQL Server Express), and then Execute them in one go, you will have set-up, sent, and received your first secure SSB message across databases.

 

-- STAGE 1: SETUP TARGET DATABASE

 

      USE master;

 

      /* Create Target Database */

 

            IF EXISTS (SELECT * FROM sys.databases WHERE name = 'Target') DROP DATABASE Target;

            CREATE DATABASE Target

            GO

            -- Activate the Broker in this database (can only be done with ALTER DATABASE)

            ALTER DATABASE Target SET ENABLE_BROKER

 

      USE Target;            

 

            -- Create the database master key

            CREATE MASTER KEY ENCRYPTION BY PASSWORD = '<Specify a password here>'

            GO

 

      /* Set up Target User, and place certificate (without private key) in a file */

 

            -- Create Target User without login

            IF EXISTS (SELECT * FROM sys.sysusers WHERE name = 'TargetUser') DROP USER TargetUser;

            CREATE USER TargetUser WITHOUT LOGIN

 

            -- Create a Cert for the Initiator user

            IF EXISTS (SELECT * FROM sys.certificates WHERE name = 'TargetCert') DROP CERTIFICATE TargetCert;

            CREATE CERTIFICATE TargetCert

                  AUTHORIZATION TargetUser

                        WITH SUBJECT = 'Target Certificate', EXPIRY_DATE = '10/31/2009';

 

            -- Backup the cert up so the Target database can import the cert (public key)

            BACKUP CERTIFICATE TargetCert TO FILE = 'c:\TargetCert.cer';

            GO

 

      /* Set up SSB */

 

            -- First, we need to create a message type. This message type is very simple and allows any type of content

            -- (Drop contract because it binds to message)

            IF EXISTS (SELECT * FROM sys.service_contracts WHERE name = 'SubmissionContract') DROP CONTRACT SubmissionContract;

            IF EXISTS (SELECT * FROM sys.service_message_types WHERE name = 'Message') DROP MESSAGE TYPE Message;

            CREATE MESSAGE TYPE Message VALIDATION = NONE

           

            -- Now create a contract that specifies what type types of messages

            CREATE CONTRACT SubmissionContract (Message SENT BY INITIATOR)

 

            -- Set up Target receive queue to hold messages

            -- (Need to drop service before queue, due to binding)

            IF EXISTS (SELECT * FROM sys.services WHERE name = 'TargetService') DROP SERVICE TargetService;

            IF EXISTS (SELECT * FROM sys.service_queues WHERE name = 'TargetQueue') DROP QUEUE TargetQueue;

            CREATE QUEUE TargetQueue

 

            -- Create the required service and bind to be above created queue

            CREATE SERVICE TargetService

            AUTHORIZATION TargetUser

            ON QUEUE TargetQueue (SubmissionContract)

 

            -- Create a Local Route for the destination TargetService

            IF EXISTS (SELECT * FROM sys.routes WHERE name = 'InitiatorRoute') DROP ROUTE InitiatorRoute;

            CREATE ROUTE InitiatorRoute WITH SERVICE_NAME = 'InitiatorService',     ADDRESS = 'LOCAL'

            GO

 

-- STAGE 2: SETUP INITIATOR DATABASE

 

      USE master;

 

      /* Create Initiator Database */

           

            IF EXISTS (SELECT * FROM sys.databases WHERE name = 'Initiator') DROP DATABASE Initiator;

            CREATE DATABASE Initiator;

            GO

            -- Activate the Broker in this database

            ALTER DATABASE Initiator SET ENABLE_BROKER;

 

      USE Initiator;

 

            -- Create the database master key

            CREATE MASTER KEY ENCRYPTION BY PASSWORD = '<Specify a password here>';

            GO

 

      /* Set up Initiator User */

 

            -- Create Initiator User without Login

            IF EXISTS (SELECT * FROM sys.sysusers WHERE name = 'InitiatorUser01') DROP USER InitiatorUser01;

            CREATE USER InitiatorUser01 WITHOUT LOGIN;

 

            -- Create a Cert with InitiatorUser AUTHORIZATION

            -- This links the Cert to the User so SSB uses the CERT for Dialog security

            IF EXISTS (SELECT * FROM sys.certificates WHERE name = 'InitiatorCert01') DROP CERTIFICATE InitiatorCert01;

            CREATE CERTIFICATE InitiatorCert01

                  AUTHORIZATION InitiatorUser01

                        WITH SUBJECT = 'Initiator Certificate 01', EXPIRY_DATE = '10/31/2009';

 

            -- Backup the cert up so the Target can import the cert (public key)

            BACKUP CERTIFICATE InitiatorCert01 TO FILE = 'c:\InitiatorCert01.cer';

            GO

 

      /* Set up SSB */

 

            -- First, we need to create a message type. Note that our message type is

            -- very simple and allows any type of content

            -- (Need to drop contract before message, due to binding)

            IF EXISTS (SELECT * FROM sys.service_contracts WHERE name = 'SubmissionContract') DROP CONTRACT SubmissionContract;

            IF EXISTS (SELECT * FROM sys.service_message_types WHERE name = 'Message') DROP MESSAGE TYPE Message;

            CREATE MESSAGE TYPE Message VALIDATION = NONE;

 

            -- Once the message type has been created, we need to create a contract