Laurentiu Cristofor's blog @microsoft.com : SQL Server

Finding information about which account xp_cmdshell is running as

lcris — Tue, 27 Oct 2009 21:24:00 GMT

If you ever needed to debug a permission related issue when using xp_cmdshell, you have probably realized that a crucial piece of information is about what particular account xp_cmdshell is executing under. If you are the administrator of the database, you already know the context used by xp_cmdshell, but otherwise you may not have that information. Here are some tips on how to find more.

First, if you have a command line tool that displays the current context, like whoami or a utility for dumping the security contex, you can just execute that under xp_cmdshell. That's pretty easy. But what if there is no such tool available? One thing you can try in this case is to loop back into SQL (assuming the xp_cmdshell account has access to the database - it usually does) and just ask SQL for more information with queries like the following:

EXEC xp_cmdshell 'osql -E -Q"select suser_sname()"'

EXEC xp_cmdshell 'osql -E -Q"select * from sys.login_token"'

Don't forget to pass in the server/instance name using the -S option, if you are not dealing with a default instance. These should give you plenty of information about the xp_cmdshell context and should help you figure out any access permission issue.

Given that I am on the topic of xp_cmdshell, here's how the command can be enabled using T-SQL:

sp_configure 'show advanced options', 1
reconfigure

sp_configure 'xp_cmdshell', 1
reconfigure

SQL Injection watch blog

lcris — Sat, 15 Aug 2009 00:34:00 GMT

I was looking for information on a new SQL injection attack when I stumbled upon this very useful blog: http://s3cwatch.wordpress.com/. It's worth a look from time to time, to get an idea of what attacks are going on in the wild.

Basic SQL Server Security concepts: ownership, CONTROL, TAKE OWNERSHIP

lcris — Wed, 12 Aug 2009 01:39:00 GMT

I realized today that while I have discussed earlier object permissions, I have not gone into the details of object ownership. I want to cover the following here: ownership of objects, how it can be changed, and the relatively new permission CONTROL (introduced in SQL Server 2005).

Ownership: This should be pretty clear - the owner of an object is usually its creator, but a different owner can be specified at creation time using the AUTHORIZATION clause. The owner of an object has all possible permissions on that object and, most importantly, cannot be denied those permissions while he continues to be an owner.

CONTROL permission: The CONTROL permission can be used to easily grant all permissions on an entity to some principal. It's the next best thing after ownership of the entity, but it's not quite as powerful as ownership. The main difference is that a grantee of CONTROL can still be denied some other permissions on the entity. For example, I can be granted CONTROL on a table, while at the same time I can be denied SELECT on that table, preventing me from selecting from it - this can never happen to the owner, because the owner cannot be granted or denied permissions.

Changing ownership: To change the owner of an entity, we can use the ALTER AUTHORIZATION statement. Changing ownership of an object creates some interesting scenarios in terms of what should be allowed and what should not be. Basically, owners should not be able to "dump" their objects to other users who may not want them and users should not be able to "steal" objects from those that own them. To accomplish this, two checks are needed: one controls to whom ownership can be given to - we verify that the performer of the change has some right on the new owner (we check for IMPERSONATE for users and logins, ALTER or membership for roles, etc); the other check controls from whom ownership can be taken away: we require the performer of the change to have TAKE OWNERSHIP permission on the object.

Note as a corollary that a grantee of CONTROL permission on an object has the ability to take ownership of the object because CONTROL implies the TAKE OWNERSHIP permission as well - the way to prevent this would be to grant CONTROL and deny TAKE OWNERSHIP.

TAKE OWNERSHIP is thus used to selectively grant someone the ability to willingly become the owner of an object or to facilitate the change of ownership of that object.

Side effect of ownership change: A potentially surprising side effect of changing ownership of an object is that all permissions granted on that object will be lost. As always, it's a good idea to script all permissions granted on an entity before changing its ownership, so that the grants can be re-executed by the new owner, if appropriate.

SQL Server: Windows Groups, default schemas, and other properties

lcris — Sat, 23 Aug 2008 02:15:00 GMT

Exceptions are dangerous because people like to simplify their thinking process using rules, so exceptions always carry the risk of being overlooked. In security, exceptions are a bad thing because they make the model more complex and complex systems can break in more ways than simple systems, thus being harder to analyze and secure.

Windows Groups are an exception in the SQL Server security model. You can quickly infer my opinion about them from the above paragraph. At Windows OS level they behave similarly to how roles behave in SQL Server - they are simply containers of permissions and privileges. But bringing them in the SQL Server world creates a hybrid - an entity that can not only be granted permissions, but can also own objects as well - a mix of roles and logins/users. In SQL Server, Windows Groups are a secondary identity with additional capabilities that are traditionally reserved only for primary identities. Suddenly, Windows Groups require handling not seen in any other security system.

From a usability point of view, it is convenient to get access to a system via membership to a Windows Group, but from a security perspective, the actual user identity needs to be always available and must not be allowed to disappear behind the group anonymity. Imagine having to explain who deleted an important table - "Builtin\Users did it!" This leads to tradeoffs that are either not noticed or that surprise those that pay attention. For example, even though groups can own objects, an object created by someone that connected via a group membership will be owned by a login or user created under the cover - this is called implicit login/user creation and is done so that the identity of the object creator is not lost.

As far as groups work like they work in the Windows OS or like roles work in SQL Server, their behavior is easy to understand. This is because the semantics of permissions allows them to be added together and still make sense. So, if I belong to two different groups and I inherit from them two different sets of permissions, there is no confusion about what permissions I have - I simply have all the GRANT's and DENY's that those groups provide me with.

But the situation gets trickier when we want to endow groups with properties similar to what "real" principals have. The earliest situation consisted of the default language and default database settings that were added to logins. Because groups were supposed to act as logins, a solution was needed for them too. I was not around at the time these options were introduced, but I can see the result: the options were simply added to Windows Groups as well. However, the problem with having these options available for groups is that they really reflect properties and properties, unlike permissions, are not additive. If I am a member of two groups and each has a different default database, which one do I end up connecting to? Having these properties for groups was a bad choice, but now this cannot be fixed because of backward compatibility.

A new problem these days is that schemas were introduced in SQL Server 2005 and with them, a new property was added to users: DEFAULT_SCHEMA. However, the bad choice made earlier was not repeated this time and users mapped to Windows Groups were not allowed to have this property set. This causes problems with the use of Windows Groups as it is indeed desirable to be able to specify default schemas for Windows Groups, but this must be done in an unambiguous way and there is no mechanism allowing this today. Addressing this problem is currently an open issue.

Some take-offs from this discussion:

Windows Groups can simplify management but due to their hybrid nature, they come with some restrictions. If you want to use them to avoid managing a large set of logins and users, then make sure you don't provide them with permissions to create objects - object creation will trigger implicit login/user creation, defeating the initial reason to use them. If you avoid the object creation, there is still the issue of not being able to control the default schema - until a solution is provided, access to data via Windows Group membership needs to be done using schema qualified object names, Finally, Windows Groups are useful today because they provide a way to authenticate to SQL Server and can carry permissions, but they are not yet ready to be used as carriers of properties.

Additional links:

This topic in the SQL Server Security Forum
The customer feedback item requesting the capability of setting a default schema for Windows Groups - you can use this to rate the importance of fixing the default schema issue.

SQL Server 2005: How to debug login failures (18456, anyone?)

lcris — Fri, 22 Feb 2008 03:05:00 GMT

In my series of new posts on old topics, I decided to gather today several pieces of information that I think will help in debugging SQL Server login failures. Although most information should remain useful for future versions as well, some of it may become outdated, so I tagged this article as 2005 specific.

Login failures can be broadly divided in two categories: failures to connect to SQL Server and failures to authenticate with SQL Server - I will refer to the first as protocol failures and to the second as security failures. Security failures can only occur after successfully establishing connection to the database server. Most security failures result in the 18456 error, which will always be logged by SQL Server. This means that to determine if you are likely hitting a security error, you should check the current ERRORLOG file to see if an 18456 error was logged there - the absence of such an error usually indicates the problem is elsewhere - you either failed connecting or there was some application error that was reported as a login error.

The ERRORLOG file is a text file located in the LOG folder, usually situated in the same place as the DATA folder containing the system databases. You can always find the actual location by checking the following registry key: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\MSSQL.OOO\CPE\ErrorDumpDir, where OOO should be replaced by the proper database instance number.

For protocol errors, I recommend two resources:

MSDN Forum for the SQL Server Protocols Team
SQL Server Protocols Team Blog

In what follows, I will focus more on discussing the 18456 error. This error can be raised for a variety of reasons and the most important information in debugging it is to know the state value for the error. This is not obvious because the error state is always displayed to clients with a value of 1, to prevent information disclosure to unauthorized parties. The real error state value, however, is always logged in the ERRORLOG file, as in the examples below:

2008-01-28 10:34:19.02 Logon       Error: 18456, Severity: 14, State: 8.
2008-01-28 10:34:19.02 Logon       Login failed for user 'sa'. [CLIENT: <local machine>]
...
2008-02-14 18:23:10.10 Logon       Error: 18456, Severity: 14, State: 11.
2008-02-14 18:23:10.10 Logon       Login failed for user 'Domain\User'. [CLIENT: <local machine>]

Some of the most obvious states are described in this classic 18456 post from the Protocols Team Blog, which pretty much already covered this topic. The comments to the post uncover a few more states. With this knowledge, the first failure above is easily explained as being due to the use of an incorrect password, while the second failure is due to the account not having been granted access to the server.

Knowledge of the error state is crucial in finding out the possible reason of an 18456 failure, so it should be the first piece of information you gather before reporting the problem.

For security issues, the main resources are:

MSDN Forum for SQL Server Security Team
SQL Server Security Team Blog

So, when dealing with some sort of login failure, try to determine where it happened: in the application, in establishing a connection to the server, or in authenticating with the server. Checking the SQL Server error log can help you determine if you reached the server or not, and it can also provide you with additional information about the error, which is not available elsewhere. If you are hitting an 18456 error and the state is described in the Protocols post, then you should have a good idea of what the problem is; if the state is not listed, then you should post the state when you are reporting the error - I suggest to try the SQL Server Security Forum first.

How to request features in Microsoft products

lcris — Fri, 08 Feb 2008 00:49:00 GMT

I want to address the topic of requesting feature changes in Microsoft products, to point to some tools that can help, and to describe ways to use those tools more effectively. This post is based on my experience working on customer requests while being a member of the Microsoft SQL Server team, but you may find the information I provide here useful for other Microsoft products than SQL Server.

So, basically, the problem I am discussing here is that you are working on some project using Microsoft technology and you find out that your work might be much easier if only some feature would be available. How can you ask Microsoft to add that feature to their product? One way to do this is to contact technical support, but there are other ways that I want to address here.

Before I start, I should mention that if the problem you are confronting appears to be a security vulnerability, then there is a dedicated site on which you can provide a description of the problem to the Microsoft Security Response Center. Needless to say, security vulnerabilities are addressed with higher priority than the addition of new features and the rest of this post deals with the latter.

Also, while I am talking mainly about requesting features, the steps I present would be similar if you what you need is a bug fix.

Step 1 - Information gathering. Before a feature should be requested, it is important to do a bit of research to find answers to the following questions:

- Does the feature allow you to achieve something that is worth achieving?
- Is the feature absolutely needed or are there existing features that can be used to achieve the same goal with approximately the same effort?
- Has the feature been requested already?

The easiest way to find the answers to these questions is to ask around, and the places you can ask these questions online can be divided in two categories:

- public forums, such as the microsoft.public.* forums or forums hosted by various sites focusing on the use of a specific Microsoft product
- Microsoft forums, such as the MSDN hosted forums at http://social.msdn.microsoft.com/Forums/en-US/categories/

It is worth keeping in mind the differences between these categories of forums. Each offers specific advantages and posting on the right forum is a great step toward getting a quick and appropriate answer. An advantage of public forums is that they can be used to ask a wider array of questions - for example, you can ask questions related to the interactions between Microsoft products and other products. Microsoft forums have the advantage that they are monitored by members of the product units, but they are usually more focused on discussing specific features of a Microsoft product.

As usual, before posting on any forum, you should take the time to search for existing threads that may already answer your questions. This is good forum etiquette that is expected on most forums and that their frequent contributors will often suggest, because it reduces the redundancy of information in the forum and thus increases its quality.

Once you have the answers to the above questions, several things can happen:

- you may find that the request is not necessary either because it does not really help solving a problem (in security it is often the case that a mechanism can be perceived as improving security when it really does not do so) or because there are other alternatives of solving the problem
- you may find that a request already exists and you may be provided with a way to track its progress or with a status update about the plans for developing and releasing it
- you may find that you may be the first customer to request the feature

If you are in the latter case, the next step is to ask formally for the feature.

Step 2 - Filing a request. Depending on the product, there may be various ways to ask for features. One way could be to just ask a member of the product team that you contacted online or at a professional conference to open an item for the request. But some products allow you to directly open requests online using the MSDN Product Feedback Center. The Feedback Center is a very powerful tool, because the item that you open is directly viewable by the product team. When opening a new feedback item, you should include all the information needed to describe the problem you are facing. You should also update the forum thread that you started at the previous step, to include a link to the feedback report that you opened. This will help you by helping others notice your report so they can vote for it - the more votes your proposal gathers, the higher the chance that it will be assigned a higher priority that will allow it to go ahead of other requests.

While in the SQL Server team, I have been on the receiving end of such requests and I have solved many of these (solving them meaning that I made the required changes to the SQL Server code); I also saw requests for existing features or for features that wouldn't accomplish anything useful, which is why I am emphasizing the importance of the information gathering first step. Some of the features I added were important enough to ship in the next service pack (like some of the features mentioned here), while others were postponed for the next release, but the important thing to note is that in all cases the requests got directly to the development team. In most cases, I also provided feedback back to the customer about the progress made and the plans for releasing the changes. My point here is that this process works better than you may expect. As far as I know, no other company provides a way for customers to directly contact the development team.

What next after filing a request?

Step 3 - Monitoring your request. After you made your request, the next thing to do is to monitor it, to provide additional information if needed, and to check for any developments. Feedback items have a great feature in that they allow a two way communication with the product team. As long as the state of your request does not change, there isn't much that you can do other than making sure that other people that ask for the same thing know about your request and vote for it. Each product will have its own schedule for releasing new features. Service packs usually contain high priority fixes so if your issue is not high priority, you can only expect it to show up in a future release. Bugs have a better chance of being resolved than new features have to be added, because they usually involve less code changes. Also, bugs usually get fixed all the time, while features tend to compete against other features for a larger set of resources and they only get worked on during specific periods in the development cycle. Finally, a problem for which a workaround exists will get a lesser priority than a problem with no workaround.

This is basically it - I hope this information helps you figure out how to make use of the resources I listed. I'll end this post with a summary of all these steps:

- Search to see if your problem has already been discussed. If yes, then join the existing discussion.
- Open a new request if your search has not uncovered one. Provide all information someone would need to understand your problem.
- Track your request, to provide additional information, if needed, and to check for status updates.

Finally, keep in mind that forums are great sources of information, but they don't help that much with tracking feature requests - the best way to do that is offered by sites like the Microsoft Product Feedback Center. So once you have a feedback item opened, make sure that other people having the same problem will vote for your feedback item.

SQL Server 2005: How to debug errors in code that does encryption

lcris — Thu, 31 Jan 2008 23:36:00 GMT

Encryption builtin functions in SQL Server have no known issues and, if used properly, they will produce the expected results. However, if they are used incorrectly, it can be hard to figure out what exactly is the problem, so in this post I am going to collect some hints about what to look for in the malfunctioning code.

There are two main types of encryption code related issues that I have seen popping up in forums; they usually are described as "encryption errors", which is not what they really are.

Issue 1: Corruption of encrypted value

This issue happens when the output of encryption is corrupted when stored or passed around by code. This happens because the encryption blob is truncated during a conversion or because the column in which it is stored is shorter or has the ANSI_PADDING option set to OFF. For determining the size of a column that should hold encrypted data, see this post.

Symptom: Decryption of corrupted encryption blob will return NULL.

How to debug this? Print the output of the encryption function, before it is processed in any way - this is the original encrypted blob value. Then also print the blob after each conversion or assignment; if it is inserted into a column, select back the inserted value. Any difference from the original blob value will indicate where the error happened. Most often, you don't even need to do a byte by byte comparison and you can instead just compare the lengths of the blob at different stages - you can get the length using the datalength() builtin.

Issue 2: Incorrect conversion of the decrypted value

This issue happens when encrypting a Unicode string and when later, after decryption, we try to convert the binary decryption output to a non-Unicode value. Or vice-versa. There is nothing wrong with the encryption or decryption in this case, which can be determined by following the debugging steps for Issue 1. It is easy to get this issue because the difference between nvarchar (Unicode) and varchar (ASCII) is the single letter n, which you may fail to type in.

Symptom: Decrypted data is displayed as garbage.

How to debug this? Compare the type of the data you compress and verify that the decompressed blob is appropriately converted back to that type.

SQL Server 2005: A great post by Aaron Morton about using MARS to access opened keys

lcris — Fri, 18 Jan 2008 09:20:00 GMT

Aaron Morton has a very interesting post and demo that show how MARS can be used to access keys temporarily opened by a procedure. This is a must-read for anyone that is interested in implementing custom restrictions around the use of encryption keys. Some time ago, I wrote a post about restricting the use of an encryption key just for encryption. Aaron's demo goes further and demonstrates a pitfall if one would attempt to restrict the use of an encryption key for specific data decryption. The issue happens because MARS allows interleaving around SELECT statements, and a way to prevent it is to use auto-decryption routines, which open the key in the context of the transaction rather than the session context.

SQL Server 2005: Why you should not encrypt data with certificates

lcris — Fri, 11 Jan 2008 23:44:00 GMT

I often recommended to only encrypt data in SQL Server using symmetric keys and to reserve the use of asymmetric encryption for protection of symmetric keys and for signing. In this post, I will go in more detail about why asymmetric encryption is not appropriate for protecting data.

There are three reasons why asymmetric encryption is not suitable for encrypting data:

(1) Asymmetric encryption is much slower than symmetric encryption.

(2) For small data like SSNs and CCNs, asymmetric encryption will result in a blob that will be significantly larger than what you would get from symmetric encryption - about 2 times larger or more.

(3) While for symmetric encryption, the largest data that can be compressed is around 8000 bytes, because of limitations in the SQL Server encryption interfaces, for asymmetric key encryption, the limit is much smaller and is due to limitations of the technology itself. The limits are: a 512 bit RSA key can encrypt up to 53 bytes, a 1024 bit key can encrypt up to 117 bytes, and a 2048 bit key can encrypt up to 245 bytes (reminder: in SQL Server, both certificates and asymmetric keys are wrappers over RSA keys).

So, by using asymmetric encryption to encrypt data, you pay additional time and space costs, and on top of that you are limited about how large a piece of data you can encrypt.

All these points are unrelated to the actual implementation of the algorithms; they are simply derived from the properties of RSA encryption, which is the only form of asymmetric encryption supported in SQL Server 2005.

Related to (3), also see this post and this post, which contain information that can help you determine the limits for RSA keys of different sizes than I enumerated above (or you can just determine those limits by trial and error).

Also related is this post about the use of certificates in SQL Server and this post about the difference between certificates and asymmetric keys.

SQL Server 2005: How to determine the size of a column that will hold encrypted data

lcris — Fri, 11 Jan 2008 22:05:00 GMT

This issue has been addressed before on forums, but with the heavy traffic, it can be hard to find the proper post. So, I'll provide some explanations here as well.

Note: This article is written with symmetric encryption in mind, but the actual technique would work for asymmetric encryption too.

When encrypting a piece of data, the encrypted version will always be larger than the original data. This happens because encryption doesn't compress the data and, in addition to that, SQL Server prefixes the encrypted blob with some data, the most significant being the GUID of the symmetric key used for encryption. It is possible to compute in advance what will be the length of the encryption for a given piece of data, but rather than working with a formula, a safer and simpler method is to just perform the encryption using EncryptByKey and pass the result to the datalength() builtin function. While the contents of the encrypted blob are non deterministic due to the use of a random IV, the length of the blob will be constant if the same method is used for encryption; in fact, the length of the encryption doesn't even depend on the data that is encrypted - it only depends on the length of that data.

So, when trying to decide the size of a column that is supposed to hold encrypted data, you only need to determine, as you would normally do in the absence of encryption, what is the maximum size of unencrypted data that you are willing to store; once you know this, you can just generate a piece of data of that size, encrypt it, and get the length of the result using datalength() - that will be the maximum length that you should define for the encrypted column.

Although I remember several posts on this topic, I could only find one with a quick search. Fortunately, it's a very good post from Raul:

http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=908990&SiteID=1

Update (later same day): This is the original post from Raul that explains in more detail the formula you could use to determine the length of an encrypted blob without actually encrypting. The blog on which it is posted has been discontinued, but there are some good posts made to it that are worth checking out. And if you are wondering about the blog name, Yukondoit was one of the slogans used on T-shirts around the launch of SQL Server 2005, whose code name was "Yukon".

SQL Server 2005: Restoring the backup of a database that uses encryption

lcris — Fri, 16 Nov 2007 22:41:00 GMT

I have addressed this topic in previous threads and comments (here, here, and here, for example), both on this blog and on various forums, but it looks like when you need the answer, it can be hard to dig out. So I'm hoping that by placing these steps in a dedicated post, they will become easier to find.

When you restore a database that uses encryption features, there is only one thing you need to take care off - if the database master key (DbMK) needs a service master key (SMK) encryption, you need to regenerate this encryption. Note that this encryption is made by default when you create the DbMK, but it may be intentionally dropped, if you want tighter control of access to the encrypted data. Anyway, if you did have such SMK encryption for the DbMK, the steps to regenerate it are the following:

OPEN MASTER KEY DECRYPTION BY PASSWORD = 'password'
ALTER MASTER KEY ADD ENCRYPTION BY SERVICE MASTER KEY
CLOSE MASTER KEY

That's it - the database encryption features should now work as when the backup was taken. Also note that it doesn't matter if you restore the database on the server where the backup was taken or elsewhere. The only thing that matters for this procedure is that you know one of the passwords protecting the DbMK (yes, there can be more than one - see this post).

SQL Server 2005: How to recover when the service master key (SMK) is not accessible

lcris — Thu, 15 Nov 2007 00:55:00 GMT

I wrote earlier today a reply on this topic on the public forums, but now that I checked, the reply appears to have got lost, although I still entertain the hope it may only have got delayed and will appear there in 24 hours. Anyway, this is the reason why I prefer to write longer posts on this blog rather than on forums - they don't get lost as easily, either due to forum bugs or to threads going old and forgotten.

So, the problem I want to discuss is about what can be done if the SMK, for some reason or another, becomes inaccessible. I had previously touched on this in a previous article, but this time I want to go in more details.

First, let's look into why such a thing could happen. For the SMK to become inaccessible, something out of the ordinary must happen - either a disk corruption, which would most likely impact more data than just the SMK, or a change in the system configuration that would invalidate the DPAPI encryption of the SMK, or some other problem that I don't know yet about - maybe some obscure software error we have yet to see. In the years since SQL Server 2005 has shipped, I have seen only one case where the SMK became inaccessible and the reason for that was that the master database had been restored on a different system, such that the DPAPI encryptions of the SMK were invalid - even this issue had happened in an internal testing environment and not in a real production scenario. So, there is a way you can realistically end up with an inaccessible SMK, but it is not an ordinary scenario.

The following discussion assumes that something happened to the SMK and just to the SMK - I don't make any assumption on what happened, I just assume that this SMK problem is not compounded by the existence of other problems affecting other areas than the SMK. Also, before launching yourself into a recovery operation, make sure that you can at least recover from that - you should backup your database files such that you don't get in a worse situation than when you started.

Let's review the encryption hierarchy around the SMK, because this helps us understand what we need to fix. On one hand, the SMK has two DPAPI encryptions: using the service account and the machine account credentials, so for the SMK to become inaccessible, the system must be incapable of decrypting both these encryptions (also see this article). On the other hand, the SMK is used for encrypting three different classes of entities: credentials, linked server passwords, and database master keys (also see this article). This information allows us to determine what must have happened for the SMK to become inaccessible - both DPAPI encryptions must be undecryptable, and what is the effect of the SMK becoming inaccessible - credentials, linked server passwords, and database master keys become undecryptable by the system. (If you're reading this article while working with future versions of SQL Server, keep in mind that the list of entities encrypted by the SMK may change.) We can also use this information to verify that the SMK is indeed inaccessible; to do this, we could attempt to use the SMK to encrypt a new entity, such as a new database master key (in a database created just for the purpose of this test) or a credential secret - if such operations fail, a final test can be made by attempting to regenerate the SMK - if this fails as well, it becomes pretty clear that the problem we're dealing with is related to the SMK.

So, the best fix for the problem would be to fix the DPAPI encryptions of the SMK. We can do this in two ways:

(A) If we have a backup of the SMK (this is why such backups are recommended), then we can restore that backup. We will need to use the FORCE option because the current SMK cannot be decrypted. This should fix our SMK and we can check this using the tests I mentioned above.

(B) If we don't have a backup of the SMK and if the issue happened because we moved the databases from a machine to another and if that machine is still available, then all is not lost. We can either make a backup of the SMK on the original machine (if it still has the database system on it) and then apply it according to the (A) solution, or we can copy the database files back to the original machine, to make such a backup. Either way, the purpose would be that we want to make a backup of the SMK and then restore it on the machine that has the SMK problem.

If neither of these apply - we don't have a backup of the SMK and there is no original system where we can make one, then we're in some trouble. Not a lot, but we'll have some non-trivial cleanup to do. So let's assess the situation in this case: we don't have a SMK backup and we can't access the SMK, so let's face it: we've lost the SMK; this means we've lost the credentials and linked server passwords encrypted by the SMK - we'll need to regenerate all of these with help from whoever created them. The good part is that we didn't loose the database master keys - unless we managed to also forget the passwords protecting them - that would be quite unfortunate. Because DbMKs are always encrypted by a password, besides a default SMK encryption, the latter can always be fixed as long as we know the DbMK password. So, let's enumerate the steps we need to follow to cleanup the situation:

(a) We need to regenerate the SMK, because the current one is now unrecoverable. So we need to use REGENERATE and the FORCE option to create a new and valid SMK.

(b) For each DbMK that had a SMK encryption, we need to open the DbMK using its password encryption and then we need to re-encrypt the DbMK using the SMK. This is the same kind of step that we would need to do when moving a database from one system to another. Here is the TSQL for it:

OPEN MASTER KEY DECRYPTION BY PASSWORD = 'password'
ALTER MASTER KEY ADD ENCRYPTION BY SERVICE MASTER KEY
CLOSE MASTER KEY

The steps above should bring our database system back to normal operational state as far as the SMK is concerned. It's going to be painful to do (c), but that's the price to pay because we didn't have a SMK backup. (b) and (c) need to be done after (a) because they need a valid SMK, but their order can be swapped, i.e. we can do (c) before (b).

SQL Server undocumented password hashing builtins: pwdcompare and pwdencrypt

lcris — Wed, 31 Oct 2007 21:08:00 GMT

First, I must say that I don't know why these exist in an undocumented form. They have been around for a long time and a search on their names gets me back pages of hits. Being undocumented means that their actual implementation may change slightly from one version of SQL Server to another, mainly because the password hash format could change - and it has changed for the last three major versions of SQL Server. They could also disappear entirely in a future release. So, you shouldn't write applications relying on these functions, but they may come in handy for ad-hoc queries, when you know they're available.

So here's a short description of the two functions:

pwdencrypt: takes a varchar value as parameter and returns a varbin value that is the SQL Server password hash of the input value. I have seen people talking about the use of this function for hashing passwords when implementing custom application authentication. Do not use this! Starting with SQL Server 2005, you can use instead the HashBytes function, to implement any custom hashing scheme that you want. HashBytes provides you with direct access to several hashing algorithms, so there is no point in using pwdencrypt. In fact, I can't come up with any useful need for pwdencrypt and the only reason I included it here is to warn you against using it.

pwdcompare: takes two arguments - a varchar value that is a cleartext password and a varbin value that is a SQL Server password hash, and returns 1 if they match and 0 if they don't. There is a third optional parameter that can be set to 1 if the second parameter represents a pre-SQL Server 2000 password hash value. This third parameter will most likely be dropped in a future SQL Server version.

pwdcompare is useful if you are an administrator looking for accounts that have weak passwords. For example, to query for logins that have an empty password, the following queries can be used (first one will work on SQL Server 2000 and the second one uses the SQL Server 2005 new catalogs):

select name from sys.syslogins where pwdcompare('', password) = 1
select name from sys.sql_logins where pwdcompare('', password_hash) = 1

Obviously, these queries can be used to search for other weak passwords than empty ones. Does this constitute a threat against the strength of password hashes by allowing a TSQL brute-force attack? Not really, because such an approach would be very slow - it would be much more efficient to attempt such an attack using a compiled program, and even such approach would only have a good chance of success against a short or weak password. So, the pwdcompare function doesn't make an attacker's job easier. Other than helping administrators with checking for weak passwords, I can't think of another use for this function.

Basic SQL Server Security concepts: SIDs, orphaned users, and loginless users

lcris — Fri, 26 Oct 2007 03:42:00 GMT

I am grouping here two topics (orphaned users and loginless users) that are actually very different, but I have often seen confusion between them, so I am covering them together in an attempt to dispel that confusion.

In a previous discussion of logins and users, I pointed out that the way a login gets mapped to users in databases is via its SID value. This mapping is a one to many mapping - a login maps to several users (each one in a different database), but a user maps back to a single login. When a Windows login is created, its SID value is taken to be the SID of the corresponding Windows principal, but for SQL Server specific logins the SID values are generated at creation time, unless they are explicitly provided via the SID clause of CREATE LOGIN. The SID value represents the identity of the login and is invariant for the life of the login - this is why the SID clause is not available for ALTER LOGIN.

The SID link between a login and a user can be severed when databases are moved from one server to another - this is because on the new server, there may be no login having the same SID as that user. When such a situation happens, we say that the user was orphaned - it becomes an orphan user. Orphan users can also result from the deletion of a login - in previous versions, SQL Server attempted to warn about the existence of corresponding users, but even that check would fail to detect users in databases that were offline, so SQL Server 2005 no longer performs it (it was also an expensive check, as it required accessing each database). The bottom line is that orphaned users can appear in various ways and we have to either fix them when they occur or we have to prevent them from occurring at all. The classic method for restoring the SID link was to use the sp_change_users_login procedure, but starting with SQL Server 2005 SP2, a new LOGIN clause of ALTER USER has also been made available, which helps fix Windows users in addition to SQL users - a feature that sp_change_users_login was lacking. These methods will fix the SID of a user to match that of a login, effectively remapping the user to match the login. For the preventive approach, the main tool we have available is to create logins with an explicit SID value - this will help when moving a database between two or more servers, by allowing us to create in advance SQL Server logins that not only have the same name, but the same SID value as well. This will also help with recreating a login that was inadvertently dropped.

Some people seem to think that having the possibility to change a login's SID to match that of a user would be a great feature. That is not true. Changing a login's SID would be wrong because it would attack the problem on the wrong side of the login-user relationship. When fixing the user SID, only that SID needs to be changed and this would be a database scoped operation, but when fixing a login SID, all corresponding user SIDs would have to be changed as well, otherwise we would generate more orphaned users instead of fixing them - and fixing all users mapped to a login isn't a task that can be easily performed, because it would require the availability of all databases in which the login is mapped - something that cannot be guaranteed. There is a good reason why this issue is referred to as orphaned users and not as childless logins.

So, orphaned users are an issue that normally requires some fixing, when it occurs. Loginless users, on the other hand, are an intentional construct. They are created using the WITHOUT LOGIN clause of CREATE USER and they represent standalone users that cannot map to a login (they have special SID values preventing this), so you cannot connect to a database as a loginless user, although you can impersonate one via EXECUTE AS. A loginless user is valuable in many situations where an application needs to have users that don't necessarily correspond to a human user. For example, an application may be in need to create objects from time to time and to designate an owner for them - it could do this in the caller's context, but that would mean the caller would end up owning objects he may not be aware of - it is better for the application to designate a loginless user as the owner of those objects. One of my favorite uses of loginless users is to break ownership chains (see previous discussion) by assigning different loginless users as owners. Raul has a lengthier post on the topic of loginless users that offers further examples of use, including some TSQL sample code.

SQL Server 2005: A note about the use of certificates

lcris — Fri, 05 Oct 2007 06:59:00 GMT

To avoid any confusion, this post is not about the use of certificates for securing the communication between a client machine and the server; instead, this refers to the use of certificates created via the CREATE CERTIFICATE DDL.

I am prompted in writing this post by a recent question I just saw, which I know I answered many times before, but I now realized I never wrote about it here. The question is simply around the differences between how certificates are used elsewhere and how certificates are used in SQL Server. I'll answer this question here and I'll add some additional information and references, which I hope you will find useful.

The main use of certificates that we are familiar with from daily browsing activities is about the signing of code on the Internet. This signing is done as a way to prove the authenticity of a piece of software, because a signature can help validate the manufacturer of a piece of software. I say "can help", because not everyone checks carefully the details of a digital signature to ensure it is indeed trustworthy. For more details around this type of use scenario, you can look into PKI - Public Key Infrastructure.

Certificates can be used in many ways, because asymmetric key encryption is a very powerful tool, so, when certificates were introduced in SQL Server, one of the design goals was to build a generic store for encryption keys, without putting any unnecessary restrictions on their use. Because of this, expiry settings or certificate revocation are not enforced. SQL Server just regards the certificate as a key container and keeps track of all of its settings, but it doesn't restrict its use - it is up to you to decide what to do with it. The main use scenarios within SQL Server do not require any restrictions, so the only thing that is done is to give some warnings at the time the certificate is imported - if it was expired, for example. I should also note that features within SQL Server may do additional checks on a certificates's settings: for example, the Service Broker feature does check for expiration of its certificates. If your intent is to plug into PKI, that is also possible through CLR.

So, what are the main use scenarios for having certificates in SQL Server? Here they are:

1. You can use certificates to sign T-SQL code.
2. You can use certificates to encrypt symmetric encryption keys or small pieces of data.

Let's discuss these a little. Signing T-SQL code is the most powerful security feature shipped with SQL Server 2005 (my opinion, of course, as is any other subjective evaluation you may read on this blog). But, really, think about it: signing allows you to grant permissions to code instead of granting them to principals calling the code - this opens a world of possibilities in terms of how you can manage permissions and how you can restrict access to objects. So that you don't get any wrong idea at my enthusiasm for this feature, I will add that I made no significant contributions to it, so I am not trying to push forward my work ;) For a simple example of the things you can do with signing, you can have a look at this example. Raul's blog has additional examples and information on this feature.

You can also use special builtin functions (SignByCert) to sign your own data, but unfortunately, due to a bug, their implementation uses the MD5 algorithm instead of the SHA1 algorithm used in T-SQL signing, so they are not interoperable - you cannot use these functions to simulate the internal code signing, so that you could manually compute the signature of a function, for example; instead, you would need to actually create the function and sign it via ADD SIGNATURE, then read the signature blob from the system catalogs.

The second use of certificates is quite natural for an encryption key - we would definitely like to encrypt something with it. The reason why certificates are more convenient to use for protecting symmetric keys than other methods is simply because they can be backed up individually. This also gives them an edge over their cryptographic sibling - the asymmetric keys. For a discussion of why there are two objects encapsulating RSA keys in SQL Server, see this explanation.

Some people think that certificate encryption is inherently safer than symmetric key encryption - it is not. There is no significant gain in strength from securing your data with an RSA key instead of using an AES one, as long as you are using equivalent key lengths - not equal, but equivalent - more on this below. The RSA encryption is slower than any symmertric key encryption and its larger key length is simply due to it being a fundamentally different technique and requiring longer key lengths than symmetric key algorithms, to offer the same strength of encryption. For a discussion of key lengths and a comparison of key lengths across symmetric and asymmetric algorithms, you can check this article or a cryptography book. This means that certificates are not appropriate for data encryption and their use should be restricted to the encryption of symmetric keys (which are small enough that the encryption performance is not a concern) and to the application of digital signatures (when the key is only used to encrypt what is basically a hash of the signed data, hence, again, performance is not a concern given that the length of a SHA1 hash value is 20 bytes/160 bits).