Laurentiu Cristofor's blog @microsoft.com : SQL Server - cryptography

New attack on AES-256

lcris — Thu, 15 Oct 2009 00:05:00 GMT

A new attack improves significantly on previous attacks against AES-256, see: http://schneier.com/crypto-gram-0908.html#8. This doesn't mean that AES-256 is broken yet, but the surprising bit here is that AES-128 is not susceptible to this particular attack. Don't panic if you are using AES-256 and read Bruce Schneier's commentary carefully - for example, note that the attack is against a 10 round AES-256, while standard implementations use 14 rounds.

It's interesting to see that the strength of AES-256 is eroded by new attacks, but AES-128 doesn't suffer from them yet. It's another example that larger (or more expensive) isn't necessarily also better.

Can encryption make you more vulnerable?

lcris — Tue, 12 Feb 2008 03:57:00 GMT

A recent article brings up this question and argues that encrypting data at rest can open the door to a new range of security and usability problems. Speaking only of the security aspects, I both agree and disagree, so I'd like to add a few comments on this topic.

I think that the article makes a very good point that represents a great truth about security: the features you build to protect against attackers could end up being used by the attackers against you - this is the double-edged aspect of security. Once a system is compromised and the attacker gets to control it, he can use its defenses against the lawful users. This is not a new idea, but it is one that can be easily forgotten when adding new security features.

That being said, I don't think that the scenario presented in the article about ransoming data is a new or very interesting scenario. Here are the reasons why:

- An attacker can use his own encryption routines to encrypt the data - it doesn't matter if the compromised system had any encryption capabilities. Malware with the capability to encrypt data has existed for years and cyber-criminals are known to be proficient at using encryption to protect their data. Thus, ransoming is not really a new threat - it just appears new to those that also see encryption as a new technology.
- Ransoming could only work in a database system that lacks a disaster recovery procedure. Data loss is an ever-present threat for databases - even if ransoming would be new, it would be addressed by the same measures that address an old threat - data/key backups.
- The weakest link in ransoming is the ransom collection - that's where the attacker exposes himself - this makes ransoming not very tempting.
- Ransoming is also a bad idea because it tells the system owner that his system has been breached and gives him a chance to close that breach.
- If an attacker can get access to data worth ransoming, it is more easy for him to make money from selling it and keeping his access secret than by attempting to ransom it.

However, I do think that encryption can make you more vulnerable - it can do that by giving people a false sense of security if they think that just by having data encrypted, it becomes secure. I went in a bit more detail about this idea here, but the point is that deploying encryption is not only about encrypting data but about carefully considering how the data will be accessed and about securing that access - sadly, the latter part is where the security of most applications fails. Encryption addresses well certain scenarios such as stealing data at rest, but those scenarios do not necessarily represent how most data is getting compromised today.

SQL Server 2005: How to debug errors in code that does encryption

lcris — Thu, 31 Jan 2008 23:36:00 GMT

Encryption builtin functions in SQL Server have no known issues and, if used properly, they will produce the expected results. However, if they are used incorrectly, it can be hard to figure out what exactly is the problem, so in this post I am going to collect some hints about what to look for in the malfunctioning code.

There are two main types of encryption code related issues that I have seen popping up in forums; they usually are described as "encryption errors", which is not what they really are.

Issue 1: Corruption of encrypted value

This issue happens when the output of encryption is corrupted when stored or passed around by code. This happens because the encryption blob is truncated during a conversion or because the column in which it is stored is shorter or has the ANSI_PADDING option set to OFF. For determining the size of a column that should hold encrypted data, see this post.

Symptom: Decryption of corrupted encryption blob will return NULL.

How to debug this? Print the output of the encryption function, before it is processed in any way - this is the original encrypted blob value. Then also print the blob after each conversion or assignment; if it is inserted into a column, select back the inserted value. Any difference from the original blob value will indicate where the error happened. Most often, you don't even need to do a byte by byte comparison and you can instead just compare the lengths of the blob at different stages - you can get the length using the datalength() builtin.

Issue 2: Incorrect conversion of the decrypted value

This issue happens when encrypting a Unicode string and when later, after decryption, we try to convert the binary decryption output to a non-Unicode value. Or vice-versa. There is nothing wrong with the encryption or decryption in this case, which can be determined by following the debugging steps for Issue 1. It is easy to get this issue because the difference between nvarchar (Unicode) and varchar (ASCII) is the single letter n, which you may fail to type in.

Symptom: Decrypted data is displayed as garbage.

How to debug this? Compare the type of the data you compress and verify that the decompressed blob is appropriately converted back to that type.

SQL Server 2005: A great post by Aaron Morton about using MARS to access opened keys

lcris — Fri, 18 Jan 2008 09:20:00 GMT

Aaron Morton has a very interesting post and demo that show how MARS can be used to access keys temporarily opened by a procedure. This is a must-read for anyone that is interested in implementing custom restrictions around the use of encryption keys. Some time ago, I wrote a post about restricting the use of an encryption key just for encryption. Aaron's demo goes further and demonstrates a pitfall if one would attempt to restrict the use of an encryption key for specific data decryption. The issue happens because MARS allows interleaving around SELECT statements, and a way to prevent it is to use auto-decryption routines, which open the key in the context of the transaction rather than the session context.

SQL Server 2005: Why you should not encrypt data with certificates

lcris — Fri, 11 Jan 2008 23:44:00 GMT

I often recommended to only encrypt data in SQL Server using symmetric keys and to reserve the use of asymmetric encryption for protection of symmetric keys and for signing. In this post, I will go in more detail about why asymmetric encryption is not appropriate for protecting data.

There are three reasons why asymmetric encryption is not suitable for encrypting data:

(1) Asymmetric encryption is much slower than symmetric encryption.

(2) For small data like SSNs and CCNs, asymmetric encryption will result in a blob that will be significantly larger than what you would get from symmetric encryption - about 2 times larger or more.

(3) While for symmetric encryption, the largest data that can be compressed is around 8000 bytes, because of limitations in the SQL Server encryption interfaces, for asymmetric key encryption, the limit is much smaller and is due to limitations of the technology itself. The limits are: a 512 bit RSA key can encrypt up to 53 bytes, a 1024 bit key can encrypt up to 117 bytes, and a 2048 bit key can encrypt up to 245 bytes (reminder: in SQL Server, both certificates and asymmetric keys are wrappers over RSA keys).

So, by using asymmetric encryption to encrypt data, you pay additional time and space costs, and on top of that you are limited about how large a piece of data you can encrypt.

All these points are unrelated to the actual implementation of the algorithms; they are simply derived from the properties of RSA encryption, which is the only form of asymmetric encryption supported in SQL Server 2005.

Related to (3), also see this post and this post, which contain information that can help you determine the limits for RSA keys of different sizes than I enumerated above (or you can just determine those limits by trial and error).

Also related is this post about the use of certificates in SQL Server and this post about the difference between certificates and asymmetric keys.

SQL Server 2005: How to determine the size of a column that will hold encrypted data

lcris — Fri, 11 Jan 2008 22:05:00 GMT

This issue has been addressed before on forums, but with the heavy traffic, it can be hard to find the proper post. So, I'll provide some explanations here as well.

Note: This article is written with symmetric encryption in mind, but the actual technique would work for asymmetric encryption too.

When encrypting a piece of data, the encrypted version will always be larger than the original data. This happens because encryption doesn't compress the data and, in addition to that, SQL Server prefixes the encrypted blob with some data, the most significant being the GUID of the symmetric key used for encryption. It is possible to compute in advance what will be the length of the encryption for a given piece of data, but rather than working with a formula, a safer and simpler method is to just perform the encryption using EncryptByKey and pass the result to the datalength() builtin function. While the contents of the encrypted blob are non deterministic due to the use of a random IV, the length of the blob will be constant if the same method is used for encryption; in fact, the length of the encryption doesn't even depend on the data that is encrypted - it only depends on the length of that data.

So, when trying to decide the size of a column that is supposed to hold encrypted data, you only need to determine, as you would normally do in the absence of encryption, what is the maximum size of unencrypted data that you are willing to store; once you know this, you can just generate a piece of data of that size, encrypt it, and get the length of the result using datalength() - that will be the maximum length that you should define for the encrypted column.

Although I remember several posts on this topic, I could only find one with a quick search. Fortunately, it's a very good post from Raul:

http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=908990&SiteID=1

Update (later same day): This is the original post from Raul that explains in more detail the formula you could use to determine the length of an encrypted blob without actually encrypting. The blog on which it is posted has been discontinued, but there are some good posts made to it that are worth checking out. And if you are wondering about the blog name, Yukondoit was one of the slogans used on T-shirts around the launch of SQL Server 2005, whose code name was "Yukon".

SQL Server 2005: Restoring the backup of a database that uses encryption

lcris — Fri, 16 Nov 2007 22:41:00 GMT

I have addressed this topic in previous threads and comments (here, here, and here, for example), both on this blog and on various forums, but it looks like when you need the answer, it can be hard to dig out. So I'm hoping that by placing these steps in a dedicated post, they will become easier to find.

When you restore a database that uses encryption features, there is only one thing you need to take care off - if the database master key (DbMK) needs a service master key (SMK) encryption, you need to regenerate this encryption. Note that this encryption is made by default when you create the DbMK, but it may be intentionally dropped, if you want tighter control of access to the encrypted data. Anyway, if you did have such SMK encryption for the DbMK, the steps to regenerate it are the following:

OPEN MASTER KEY DECRYPTION BY PASSWORD = 'password'
ALTER MASTER KEY ADD ENCRYPTION BY SERVICE MASTER KEY
CLOSE MASTER KEY

That's it - the database encryption features should now work as when the backup was taken. Also note that it doesn't matter if you restore the database on the server where the backup was taken or elsewhere. The only thing that matters for this procedure is that you know one of the passwords protecting the DbMK (yes, there can be more than one - see this post).

SQL Server 2005: How to recover when the service master key (SMK) is not accessible

lcris — Thu, 15 Nov 2007 00:55:00 GMT

I wrote earlier today a reply on this topic on the public forums, but now that I checked, the reply appears to have got lost, although I still entertain the hope it may only have got delayed and will appear there in 24 hours. Anyway, this is the reason why I prefer to write longer posts on this blog rather than on forums - they don't get lost as easily, either due to forum bugs or to threads going old and forgotten.

So, the problem I want to discuss is about what can be done if the SMK, for some reason or another, becomes inaccessible. I had previously touched on this in a previous article, but this time I want to go in more details.

First, let's look into why such a thing could happen. For the SMK to become inaccessible, something out of the ordinary must happen - either a disk corruption, which would most likely impact more data than just the SMK, or a change in the system configuration that would invalidate the DPAPI encryption of the SMK, or some other problem that I don't know yet about - maybe some obscure software error we have yet to see. In the years since SQL Server 2005 has shipped, I have seen only one case where the SMK became inaccessible and the reason for that was that the master database had been restored on a different system, such that the DPAPI encryptions of the SMK were invalid - even this issue had happened in an internal testing environment and not in a real production scenario. So, there is a way you can realistically end up with an inaccessible SMK, but it is not an ordinary scenario.

The following discussion assumes that something happened to the SMK and just to the SMK - I don't make any assumption on what happened, I just assume that this SMK problem is not compounded by the existence of other problems affecting other areas than the SMK. Also, before launching yourself into a recovery operation, make sure that you can at least recover from that - you should backup your database files such that you don't get in a worse situation than when you started.

Let's review the encryption hierarchy around the SMK, because this helps us understand what we need to fix. On one hand, the SMK has two DPAPI encryptions: using the service account and the machine account credentials, so for the SMK to become inaccessible, the system must be incapable of decrypting both these encryptions (also see this article). On the other hand, the SMK is used for encrypting three different classes of entities: credentials, linked server passwords, and database master keys (also see this article). This information allows us to determine what must have happened for the SMK to become inaccessible - both DPAPI encryptions must be undecryptable, and what is the effect of the SMK becoming inaccessible - credentials, linked server passwords, and database master keys become undecryptable by the system. (If you're reading this article while working with future versions of SQL Server, keep in mind that the list of entities encrypted by the SMK may change.) We can also use this information to verify that the SMK is indeed inaccessible; to do this, we could attempt to use the SMK to encrypt a new entity, such as a new database master key (in a database created just for the purpose of this test) or a credential secret - if such operations fail, a final test can be made by attempting to regenerate the SMK - if this fails as well, it becomes pretty clear that the problem we're dealing with is related to the SMK.

So, the best fix for the problem would be to fix the DPAPI encryptions of the SMK. We can do this in two ways:

(A) If we have a backup of the SMK (this is why such backups are recommended), then we can restore that backup. We will need to use the FORCE option because the current SMK cannot be decrypted. This should fix our SMK and we can check this using the tests I mentioned above.

(B) If we don't have a backup of the SMK and if the issue happened because we moved the databases from a machine to another and if that machine is still available, then all is not lost. We can either make a backup of the SMK on the original machine (if it still has the database system on it) and then apply it according to the (A) solution, or we can copy the database files back to the original machine, to make such a backup. Either way, the purpose would be that we want to make a backup of the SMK and then restore it on the machine that has the SMK problem.

If neither of these apply - we don't have a backup of the SMK and there is no original system where we can make one, then we're in some trouble. Not a lot, but we'll have some non-trivial cleanup to do. So let's assess the situation in this case: we don't have a SMK backup and we can't access the SMK, so let's face it: we've lost the SMK; this means we've lost the credentials and linked server passwords encrypted by the SMK - we'll need to regenerate all of these with help from whoever created them. The good part is that we didn't loose the database master keys - unless we managed to also forget the passwords protecting them - that would be quite unfortunate. Because DbMKs are always encrypted by a password, besides a default SMK encryption, the latter can always be fixed as long as we know the DbMK password. So, let's enumerate the steps we need to follow to cleanup the situation:

(a) We need to regenerate the SMK, because the current one is now unrecoverable. So we need to use REGENERATE and the FORCE option to create a new and valid SMK.

(b) For each DbMK that had a SMK encryption, we need to open the DbMK using its password encryption and then we need to re-encrypt the DbMK using the SMK. This is the same kind of step that we would need to do when moving a database from one system to another. Here is the TSQL for it:

OPEN MASTER KEY DECRYPTION BY PASSWORD = 'password'
ALTER MASTER KEY ADD ENCRYPTION BY SERVICE MASTER KEY
CLOSE MASTER KEY

The steps above should bring our database system back to normal operational state as far as the SMK is concerned. It's going to be painful to do (c), but that's the price to pay because we didn't have a SMK backup. (b) and (c) need to be done after (a) because they need a valid SMK, but their order can be swapped, i.e. we can do (c) before (b).

SQL Server 2005: A note about the use of certificates

lcris — Fri, 05 Oct 2007 06:59:00 GMT

To avoid any confusion, this post is not about the use of certificates for securing the communication between a client machine and the server; instead, this refers to the use of certificates created via the CREATE CERTIFICATE DDL.

I am prompted in writing this post by a recent question I just saw, which I know I answered many times before, but I now realized I never wrote about it here. The question is simply around the differences between how certificates are used elsewhere and how certificates are used in SQL Server. I'll answer this question here and I'll add some additional information and references, which I hope you will find useful.

The main use of certificates that we are familiar with from daily browsing activities is about the signing of code on the Internet. This signing is done as a way to prove the authenticity of a piece of software, because a signature can help validate the manufacturer of a piece of software. I say "can help", because not everyone checks carefully the details of a digital signature to ensure it is indeed trustworthy. For more details around this type of use scenario, you can look into PKI - Public Key Infrastructure.

Certificates can be used in many ways, because asymmetric key encryption is a very powerful tool, so, when certificates were introduced in SQL Server, one of the design goals was to build a generic store for encryption keys, without putting any unnecessary restrictions on their use. Because of this, expiry settings or certificate revocation are not enforced. SQL Server just regards the certificate as a key container and keeps track of all of its settings, but it doesn't restrict its use - it is up to you to decide what to do with it. The main use scenarios within SQL Server do not require any restrictions, so the only thing that is done is to give some warnings at the time the certificate is imported - if it was expired, for example. I should also note that features within SQL Server may do additional checks on a certificates's settings: for example, the Service Broker feature does check for expiration of its certificates. If your intent is to plug into PKI, that is also possible through CLR.

So, what are the main use scenarios for having certificates in SQL Server? Here they are:

1. You can use certificates to sign T-SQL code.
2. You can use certificates to encrypt symmetric encryption keys or small pieces of data.

Let's discuss these a little. Signing T-SQL code is the most powerful security feature shipped with SQL Server 2005 (my opinion, of course, as is any other subjective evaluation you may read on this blog). But, really, think about it: signing allows you to grant permissions to code instead of granting them to principals calling the code - this opens a world of possibilities in terms of how you can manage permissions and how you can restrict access to objects. So that you don't get any wrong idea at my enthusiasm for this feature, I will add that I made no significant contributions to it, so I am not trying to push forward my work ;) For a simple example of the things you can do with signing, you can have a look at this example. Raul's blog has additional examples and information on this feature.

You can also use special builtin functions (SignByCert) to sign your own data, but unfortunately, due to a bug, their implementation uses the MD5 algorithm instead of the SHA1 algorithm used in T-SQL signing, so they are not interoperable - you cannot use these functions to simulate the internal code signing, so that you could manually compute the signature of a function, for example; instead, you would need to actually create the function and sign it via ADD SIGNATURE, then read the signature blob from the system catalogs.

The second use of certificates is quite natural for an encryption key - we would definitely like to encrypt something with it. The reason why certificates are more convenient to use for protecting symmetric keys than other methods is simply because they can be backed up individually. This also gives them an edge over their cryptographic sibling - the asymmetric keys. For a discussion of why there are two objects encapsulating RSA keys in SQL Server, see this explanation.

Some people think that certificate encryption is inherently safer than symmetric key encryption - it is not. There is no significant gain in strength from securing your data with an RSA key instead of using an AES one, as long as you are using equivalent key lengths - not equal, but equivalent - more on this below. The RSA encryption is slower than any symmertric key encryption and its larger key length is simply due to it being a fundamentally different technique and requiring longer key lengths than symmetric key algorithms, to offer the same strength of encryption. For a discussion of key lengths and a comparison of key lengths across symmetric and asymmetric algorithms, you can check this article or a cryptography book. This means that certificates are not appropriate for data encryption and their use should be restricted to the encryption of symmetric keys (which are small enough that the encryption performance is not a concern) and to the application of digital signatures (when the key is only used to encrypt what is basically a hash of the signed data, hence, again, performance is not a concern given that the length of a SHA1 hash value is 20 bytes/160 bits).

SQL Server 2008: Transparent data encryption feature - a quick overview

lcris — Thu, 04 Oct 2007 02:55:00 GMT

I have kept silent on this feature while it was being developed, but as it has now been publicly advertised in various ways (being mentioned here, here, here, and here, for example), I think it is probably time to write a bit about it. Given that my posts so far have covered SQL Server 2005, I'll point out again that this is a SQL Server 2008 feature!

The transparent data encryption feature in the next SQL Server version is the last feature I worked on while I was in the SQL Server security team, before my move to MSR. Transparent data encryption is quite a long name, so I'll just refer to it as TDE from now on.

TDE addresses two of the encryption requests we have received most often from forum questions and from discussions with customers:

(1) a way to enable data encryption without affecting applications that consumed that data (for an idea of what problems are raised when encrypting a table column in a database, see this post)
(2) a way to encrypt an entire database

TDE addresses both these requests by providing a database level encryption feature. Both the data and the log files of a database are encrypted on disk and get decrypted when they are read into memory. This makes encryption really transparent to applications. Because database applications attempt to minimize their I/O (a costly operation), tying encryption and decryption to I/O will also take advantage of existing application design optimizations to minimize the performance impact of encryption.

It is important to note that TDE should not be perceived as a replacement of the encryption solutions shipped with SQL Server 2005; instead, TDE should be viewed as a complementary feature providing encryption at a coarser granularity level. Whereas the encryption features introduced in SQL Server 2005 provide a way to encrypt individual pieces of data (something we refer to as cell-level encryption, because it offers finer granularity than column level encryption), the new TDE feature enables encryption at database level. Also, from a cryptographic point of view, the algorithms used in TDE have also been available with the cell-level encryption, so the two features offer the same encryption strength.

The encryption of a database with TDE is done using a special key called, simply, database encryption key (or DEK, for short). The DEK can be managed using standard DDL (CREATE/ALTER/DROP) and the actual state of the encryption can be turned on or off using the ALTER DATABASE statement and a new clause: SET ENCRYPTION [ON | OFF]. Information about DEKs can be gathered from a DMV called sys.dm_database_encryption_keys. If you like using a GUI for database operations, all these features are also exposed from within Management Studio. I prefer typing SQL queries when it comes to managing a database, which is the reason why in my posts you'll find more details about DDL and system catalogs than about menus and dialog options.

If you have a CTP build, you can quickly check for the presence of the TDE feature by issuing the following statement: alter database <db_name> set encryption on. A syntax error will mean that the feature is not present, while a more specific error message will indicate that the feature is available - note that the statement should result in no state changes anyway, as long as you have not used TDE before. As of today, to my knowledge, only the latest CTP build includes this feature.

Who needs encryption?

lcris — Fri, 01 Dec 2006 05:07:00 GMT

For those that read my previous posts, the question in the title may be startling. I want to reassure you from the start: this post is not about encryption being a useless technique; it is just about it not being a solution for certain problems and definitely not being a general solution for any problem. Also, this post is not product specific - I am discussing the technique of encryption in general, not the way encryption is implemented in one particular product. The limitations I will discuss are properties of current standard encryption techniques, not limitations of specific product implementations.

Over the last year, I have seen too many attempts of applying encryption to problems where encryption does not really solve anything. So, in this post, I would like to make an attempt and explain why encryption is not a magical band-aid that only needs to be applied to data, to have it secured. To understand this, we have to understand a few basic facts about encryption and about what it achieves. There are not many such facts, but they are extremely important.

No part of this post requires any deep cryptographical knowledge - the only thing you need to know about encryption is that it transforms data into a form that hides all its properties in such a way that cannot be reverted unless a piece of information referred to as the decryption key is known. The encryption operation requires the use of another piece of information referred to as the encryption key. There are encryption algorithms where the encryption key and the decryption key are the same - these are called symmetric algorithms. There are also algorithms in which these are different - these are called asymmetric algorithms.

Fact 1: Encryption does not eliminate the need to protect some data

A small detail, but most important, really. If we want to protect some data, encryption does not fully help - it just reduces the amount of data we need to protect. Instead of having to protect a large amount of data, we will just have to protect a small piece - the decryption key. This changes the difficulty of a problem: for example, if you want to keep secret 100 pages of information, one choice is to memorize them and then burn them, but few people can memorize so much information, so a more practical solution using encryption would be to encrypt those pages using a password and then to commit that password to memory. The bottom line is that you still have some information to protect, which, if discovered, would compromise your encrypted data.

So, encrypted data is only as secure as you can secure the key that decrypts it.

But how can we secure the decryption key? Encrypting it with another key only results in the same problem. So, eventually, we will need to protect a key with one or more of the following methods used to control access to data:

(1) - something that you know (for example: a password or a PIN)
(2) - something that you have (for example: a magnetic card)
(3) - something that only you are supposed to be able to produce (biometrics, for example: fingerprint, iris scan, voice recognition)

Debit cards, for example, are a combination of (1) and (2). (3) is tricky, because, for example, the verification method for voice recognition should be able to distinguish between the real voice and a recording. Credit cards, when the signature is verified or when they have a photograph attached, attempt to do a combination of (2) and (3).

Fact 2: Encrypted data is unusable until it is decrypted

This is pretty obvious, right? But this is often ignored and it makes a lot of difference in some scenarios I will discuss later here. Encryption is designed to hide the meaning of the data, which means we cannot make much sense out of it in encrypted form, so it is no longer as usable as it used to be when it was unprotected. If encryption would not achieve this, it would not be very useful for protecting the data.

Fact 3: Decryption cannot happen without having access to the decryption key

Again, a rather obvious fact. I am almost embarassed of having to mention it, as it is what we are relying on when we use encryption: we want to make sure the process cannot be reverted without knowing the decryption key. But there is a useful takoff from this obvious fact, which is overlooked more often than not.

Fact 4: Designing an encryption algorithm is not easy

Any decent cryptography book will tell us right away that designing a good encryption algorithm is not something anyone can do. Unless you have devoted years of your life with successful results to the science of cryptography, you should use standard algorithms validated by the scientific community, instead of designing and implementing your own.

Fact 5: There is nothing magic about encryption

If you can read this post to the end and it makes sense to you, then you will probably agree about this fact. If not, then I am not sure what I can suggest other than to post your questions in the comment section.

What are some key learnings from these facts?

Takeoff 1: Decryption key protection is essential for safeguarding the encrypted data

This derives from Fact 1. It means that if we cannot secure the decryption key, there was not much point in encrypting the data at all. Whoever can get the key and the encrypted data, can get access to the decrypted data.

Takeoff 2: A system that does any meaningful querying of encrypted data needs to have access to the decryption key

A system cannot use the encrypted data without decrypting it first (as per Fact 2) and it cannot do that without access to the decryption key (as per Fact 3), hence access to the key is required.

So, encryption will help against someone that can only get access to the encrypted data. This is a valid use scenario for encryption. For example, it will protect against someone that steals an encrypted database, if the decryption key was not stolen as well. But let us look at a few other database related scenarios that come up pretty often and for which encryption is of no benefit:

Scenario 1: DRM

DRM is mainly used in relation to media protection, such as music files, but in this case, I use this acronym for the following kind of database scenario:

"I would like to package my database application in a form that would allow a customer to use it, but without him ever being capable to access the actual data stored in it. I think encrypting the database should help".

Unfortunately, encryption will not help. The reason it will not help is that the customer still needs to be able to use the database, so, according to Fact 2 and Takeoff 2, the encryption key needs to be present somewhere on the customer's machine. At this point, the protection breaks, because there is no place on a machine where software can store some information in such a way that the machine owner cannot get to it. Once the key is found, the database can be decrypted and the protection is broken.

Encryption in this case would act as obfuscation. This is the case with the old SQL Server CREATE PROCEDURE ... WITH ENCRYPTION feature. This feature has been hammered hard because it attempted to do something impossible. Some people suggested improvements related to using a better encryption algorithm. This was not the problem - this feature uses RC4 and no one broke it by breaking the encryption algorithm - attackers just figured out where the system gets the key from. Any other solution for this problem would just change a bit the algorithm, but the issues remain the same.

To be more specific, the difference between obfuscation and encryption is the following: with obfuscation, the strength relies in the algorithm itself, disclosing it would allow someone to easily break it; with encryption, the strength relies in the decryption key, not in the algorithm - even knowing the algorithm would not be sufficient to break the encryption. Using encryption and obfuscating the decryption key makes the ensemble equate to obfuscation.

Obfuscation is not completely useless. It can serve a purpose if the cost of breaking it is higher than the benefit of breaking it. However, a successful commercial obfuscation method will most certainly be broken and the chance will increase with the degree of commercial success of the method, because the more data it protects, the more benefit can be derived from breaking it. It really is a matter of economics, more than security.

Scenario 2: Separation of duties from machine administrator

"I think encrypting the data in the database will prevent a machine administrator from being able to read it".

Unfortunately, encryption will not help with this scenario either. The problem is similar to the one in the previous scenario: the decryption key has to be available to the database system, which needs it to be able to process the data, but this also makes it available to the machine administrator, which has unlimited access to the system.

Unfortunately, PCs were not designed with separation of duties in mind. The PC acronym stands for personal computer - this machine was originally meant to be a personal device; it is only due to the amazing advancements in hardware over the last two decades that this originally humble machine became capable of handling increasingly complicated tasks, which require a different level of security. The result is that the PC does not permit information to be processed by software without the administrator of the machine being able to see it. Trying to protect information from the eyes of the machine administrator is similar to attempting to hide your valuables from your landlord - you can sue him after he steals them, but you cannot prevent him from doing so.

To be clear, it would be extremely expensive to build a machine and the software around it that could provide an absolute guarantee about separation of duties. And using the system would probably be fairly inconvenient. So, it should not come as a surprise that a PC does not provide such guarantee.

Scenario 3: Protections against a hacker attack on an online database

"If an attacker hacks into my database, he'll get my sensitive information, but if I encrypt it, it will be useless to him".

Well, depending on how the hacker gets in and how the data is used, this may or may not be true. In most practical situations, unfortunately, I expect that this will not hold true.

To take a recent data theft as an example, an attacker was able to use a SQL injection attack to steal SSNs from a database system. The investigation of this attack appeared to deplore the fact that this information was stored in clear. But would it have really helped if it were encrypted? The database system was probably using that information and was serving it in decrypted form to the authorized users that requested it; if the injection was made under an account that was authorized to see the data in decrypted form, the attacker would have been able to read that data as easily as if it was not encrypted.

Too many "if"s? Maybe, but what they mean is that there is no guarantee that the encrypted data is safe from an attacker that gets access to the live system. Especially if encryption was done "transparently", for usability purposes, then there would be no barrier against an attacker other than the regular authorization system - such encryption would not protect at all against a SQL injection attack.

To conclude this rather long article, encryption is not a "silver bullet" of security. Having data encrypted does not necessarily mean that it also becomes automatically secured. Unfortunately, encrypted data still has to be decrypted before being used, and at that moment it becomes vulnerable. We can make some strong security guarantees based on the use of encryption in some scenarios, but we cannot use encryption in an arbitrary scenario and claim security just for doing that.

If you want to learn more about encryption than what I covered here (I could not cover very much), then I recommend the books written by Bruce Schneier. He has been writing for years about the dangers of using cryptography (or other various procedures) without understanding security and, while I wrote this post in my own words, I cannot shake the feeling that I am repeating some of the ideas that Bruce keeps mentioning in his books - maybe for the same reason why he also keeps mentioning them, because some security mistakes are constanty being repeated too.

SQL Server 2005: An example for how to use counter signatures

lcris — Fri, 20 Oct 2006 04:27:00 GMT

A while ago, I wrote a post showing how signatures can be used to allow users to perform operations without explicitly granting them the permissions required for that operation. In this post I'll present more details about the use of signatures.

One important thing to keep in mind when working with signatures is that, normally, they will only have effect while the signed module is executed. When executing a signed module, the signatures will be temporarily added to the SQL token, but they will be lost if the module executes another one or if it terminates execution. In some situations, you may want the signatures to be persisted across multiple module calls. This is where counter signatures can be used. A counter signature is a special form of signature: by itself, it does not grant any permissions; however, it allows signatures made by the same certificate or asymmetric key to be kept for the duration of the call made to the counter signed module.

I bet this sounds rather complicated, so let's go over an example that is slightly less complicated (TSQL code is at the end of this post). Let's say we have user Alice calling procedure ProcAlice, which calls procedure ProcT, which selects from table T. Alice has EXECUTE permission on ProcAlice and ProcT, but she cannot select from T, and no ownership chaining is involved in this entire chain. So Alice cannot access T - neither directly, nor through the use of ProcAlice and ProcT. The tricky part is that we want Alice to always use ProcAlice for access - we don't want her to use ProcT. How can we accomplish this? Sure, we could sign ProcT, such that ProcT can access T, but then Alice can invoke ProcT directly - she doesn't have to call ProcAlice. We could deny EXECUTE permission on ProcT to Alice, but then Alice would not be able to call ProcT through ProcAlice either (remember that no ownership chaining is involved). Signing ProcAlice would not work by itself, because the signature would be lost in the call to ProcT. However, by counter signing ProcT with the same certificate used to sign ProcAlice, SQL Server will keep the signature across the call chain and will allow access to T; also, if Alice will attempt to call ProcT directly, she still won't be able to access T, because the counter signature doesn't grant any rights.

This is the essence of the TSQL example shown below. In the example, ProcAlice appears as proc_select_t_for_alice, and ProcT is proc_select_t. I have not tried to eliminate the ownership chaining between these two. While running through this example, I recommend examining how the state of the login and user tokens is changed by adding the signatures. Finally, do not use this example as an example for how to select passwords :)

-- A demo for counter signatures
--
-- create test database
--
create database test_cs
go
use test_cs

-- create table t
--
create table t (c int)
insert into t values (42)

-- create a bogus user to own t
--
create user bogus without login
alter authorization on t to bogus

-- create a certificate for signing
--
create certificate cs_select_t
encryption by password = 'SimplePwd01'
with subject = 'Certificate used to grant SELECT on t'
create user ucs_select_t from certificate cs_select_t
grant select on t to ucs_select_t

-- create a principal with low privileges
--
create login alice with password = 'SimplePwd01'
create user alice

-- verify alice cannot access t
--
execute as login = 'alice'
    select * from sys.login_token
    select db_name()
    select * from sys.user_token

select * from t
revert

-- create a procedure that directly accesses t
--
create procedure proc_select_t as
begin
    select * from sys.login_token
    select db_name()
    select * from sys.user_token

print 'Now selecting from t...'

select * from t
end
go
grant execute on proc_select_t to public

-- verify procedure
--
exec proc_select_t

-- verify alice cannot access t through procedure
--
execute as login = 'alice'
exec proc_select_t
revert

-- create special procedure for accessing t
-- this will call proc_select_t but will do some extra processing
--
create procedure proc_select_t_for_alice as
begin
    select * from sys.login_token
    select db_name()
    select * from sys.user_token

    if user_id() <> user_id('alice')
    begin
        print 'Only alice can use this'
        return
    end

exec proc_select_t
end
go
grant execute on proc_select_t_for_alice to public

-- verify procedure
--
exec proc_select_t_for_alice

-- alice still can't use the procedure yet
--
execute as login = 'alice'
exec proc_select_t_for_alice
revert

-- Sign procedure to grant it SELECT permission
--
add signature to proc_select_t_for_alice by certificate cs_select_t with password = 'SimplePwd01'

-- retry - it still won't work, but we'll see some nice tokens
--
execute as login = 'alice'
exec proc_select_t_for_alice
revert

-- Counter sign proc_select_t, to make this work
--
add counter signature to proc_select_t by certificate cs_select_t with password = 'SimplePwd01'

-- retry - now it finally works
-- note that calling proc_select_t directly still doesn't work
--
execute as login = 'alice'
exec proc_select_t_for_alice

exec proc_select_t
revert

-- cleanup
--
use master
go
drop database test_cs
drop login alice
--
-- EOD

SQL Server 2005: How to determine what key was used to encrypt a piece of data

lcris — Fri, 07 Jul 2006 02:23:00 GMT

Let's say we have some data that is encrypted and we would like to find out what key was used to perform the encryption. SQL Server 2005 knows what key was used to encrypt the data because the key identifier (the key_guid value) is prefixed to the encrypted data. We can find out the key same as SQL Server does with a one-line TSQL statement. Here's a small demo that wraps that one-line statement in a function for easier use:

-- Set up a database for this demo
--
create database test;

use test;

-- We use the identity_value clause to obtain the same fixed GUID on all systems
--
create symmetric key skey with algorithm = triple_des, identity_value = 'Test' encryption by password = 'Anss$pt@Ihnbwef&o!';

-- The following query should return
-- 5D910600-5D5F-874F-54F5-1892D884C477
--
select key_guid from sys.symmetric_keys where name = 'skey';

-- create a simple function to return the guid of the key that was used to encrypt a piece of data
--
create function key_guid_from_bytes (@data varbinary(8000))
returns uniqueidentifier
as
begin
return convert(uniqueidentifier, @data);
end;

-- encrypt using skey and use the key_guid_from_bytes function to examine the encrypted data
-- we should retrieve the key guid - 5D910600-5D5F-874F-54F5-1892D884C477 and the key name - skey
--
open symmetric key skey decryption by password = 'Anss$pt@Ihnbwef&o!';

select dbo.key_guid_from_bytes(encryptbykey(key_guid('skey'), 'Top Secret'));
select name from sys.symmetric_keys where key_guid = dbo.key_guid_from_bytes(encryptbykey(key_guid('skey'), 'Top Secret'));

close symmetric key skey;

-- Cleanup
--
use master;
drop database test;

SQL Server 2005: How to regenerate the same symmetric key in two different databases

lcris — Fri, 07 Jul 2006 01:26:00 GMT

In a previous post on using symmetric keys, I mentioned that keys can be recreated using the KEY_SOURCE and IDENTITY_VALUE clauses of CREATE SYMMETRIC KEY. In this post, I'd like to expand a little on this topic and present a small demo as well.

Because keys cannot be individually backed up and restored, there is no direct way of moving a key from one database to another. However, by specifying the same values for the ALGORITHM, KEY_SOURCE, and IDENTITY_VALUE clauses of CREATE SYMMETRIC KEY, the same key can be generated on different databases. The KEY_SOURCE is the most important clause: the passphrase specified here is used to determine the key bits, so the phrase should be protected as carefully as the key itself or the data protected by it are protected. By specifying the same KEY_SOURCE, you are guaranteed to obtain the same key, assuming of course that you specified the same encryption algorithm. However, this is not sufficient to allow us to decrypt data encrypted by the key in another database - we also need for this key to be identified by the system as the same key - that is, the key needs to have the same identifier, because this identifier is appended to the encrypted data and is used to determine the key that should be used for decryption. (The identifier of a key is shown in the key_guid column in the sys.symmetric_keys dialog and is also the value that needs to be passed to the encryptbykey functions). This is where the IDENTITY_VALUE clause comes into place - the phrase specified here will be used to generate a key identifier. The IDENTITY_VALUE clause doesn't have to be secret, but there's no reason it should be advertised either. The KEY_SOURCE is what you need to protect carefully.

And here is a small script that shows how to create a key using these clauses and how to decrypt data encrypted by it in a different database.

-- Set up the databases used for the demo
--
create database db_source;
create database db_destination;

use db_source;

-- Keep the key_source phrase carefully protected - it''s the key!!!
--
create symmetric key skey
with algorithm = triple_des,
identity_value = 'Data encryption key 07/06/2006',
key_source = 'Now he''s a clock-punching insurance claims adjuster fighting boredom and a bulging waistline.'
encryption by password = 'Avc#ptNO$cf@o!';

open symmetric key skey decryption by password = 'Avc#ptNO$cf@o!';

select * from sys.symmetric_keys;
select * from sys.openkeys;

-- Encrypt some data in a table
--
create table t (data varbinary(1024));
insert into t values (encryptbykey(key_guid('skey'), 'Top Secret!'));
select * from t;

close symmetric key skey;

-- Now copy the encrypted data to another table in another database
--
use db_destination;

create table t (data varbinary(1024));
insert into t (data) select t_src.data from db_source.dbo.t t_src;
select * from t;

-- Recreate the encryption key, so we can decrypt
-- The key can have a different name and can be protected with a different mechanism,
-- but it has to be obtained from the same algorithm, key_source, identity_value combo
-- In this database, we'll protect the key using a certificate
--
create master key encryption by password = 'Yahtf%pt@Hwht$f!O!';
create certificate cert_skey with subject = 'Certificate for accessing symmetric keys 07/06/2006';
create symmetric key skey2
with algorithm = triple_des,
identity_value = 'Data encryption key 07/06/2006',
key_source = 'Now he''s a clock-punching insurance claims adjuster fighting boredom and a bulging waistline.'
encryption by certificate cert_skey;

select * from sys.symmetric_keys;
select * from sys.openkeys;

-- Now use the key to decrypt the copied data
--
select convert(varchar(256), decryptbykeyautocert(cert_id('cert_skey'), NULL, data)) from t;

-- Cleanup
--
use master;

drop database db_source;
drop database db_destination;

You might want to experiment with the arguments used to create skey2. You will notice that if either the KEY_SOURCE or the IDENTITY_VALUE are not identical, the key will be created, but it will be different, and the decryption will not work.

Why encryption should be salted and a small C# demo

lcris — Tue, 09 May 2006 00:51:00 GMT

In my previous post on searching encrypted data, I mentioned that the SQL Server 2005 encryption procedures are salted and that this prevents an index on encrypted data from being useful for any type of cleartext searches. Today, I will illustrate why encryption should be salted by presenting a small C# encryption demo.

Interest in searching encrypted data comes primarily from the use of social security numbers and credit card numbers as identifying information, while these are at the same time supposed to be confidential information. Thus, the requirement for the ability to search these data is combined with the need to protect it via encryption. To make the demo relevant to these requirements, I will use two made-up credit card numbers, but the issues I will point out are relevant to social security numbers as well as to other types of data.

Before I present the code, I should explain some technical details about encryption. In the following discussion, plaintext will refer to the unencrypted data and ciphertext will refer to the encrypted data.

Block encryption algorithms: TripleDES is a block encryption algorithm with a block size of 64 bits (8 bytes). This means that the algorithm encrypts chunks of 64 bits at a time. It takes the first 64 bit block of the cleartext, encrypts it, then moves on to the next 64 bit block of cleartext. This also means that if we encrypt two pieces of cleartext and they have an identical 64 bit block, then the corresponding encrypted block will be identical in both encryptions. Most encryption algorithms today are block algorithms: AES (128 bit block) and RC2 (64 bit block), for example, are other examples of block encryption algorithms.

Chaining Modes: To mitigate the fact that the same plaintext block would encrypt to the same ciphertext block, various chaining methods can be used for encryption. The chaining mode I will use in this demo is called Cipher Block Chaining (CBC) and is a commonly used mode. In this mode, when a cleartext block is about to be encrypted, it is first combined with the previously encrypted block (through a XOR operation), before the actual encryption transformation takes place. This way, even if two blocks of plaintext are identical, the corresponding encrypted blocks would still be different except when all previous plaintext blocks were identical as well.

Initialization Vector: So the CBC mode will combine the currently processed plaintext block with the previous ciphertext block. But what will happen if we just started encryption and we're processing the first block of plaintext - there is no previously generated ciphertext block yet! This is where the Initialization Vector comes into play - an IV is just a randomly generated block that will play the role of the first ciphertext block. By using a different IV for each encryption, prefix patterns are hidden because the first ciphertext block is the IV and it is different for each encryption. Using a fixed IV will allow prefix patterns to be reflected in the output of the encryption, as we will see in the demo. The use of a random IV for each encryption is also referred to as salting.

So, putting these together, block encryption can leak information about the plaintext having identical blocks, because when this happens, the ciphertext blocks will be identical too. Chaining modes combined with random IVs for each encryption will strengthen the encryption by hiding patterns in the cleartext, but both these mechanisms have to be used for this to happen. The encryption procedures in SQL Server 2005 use both CBC and random IVs to prevent patterns in the cleartext data from being reflected in the ciphertext data.

In the demo, we will encrypt two credit card numbers using a TripleDES key. The credit card numbers will have the same first eight digits because we will process them as ASCII and 8 characters require 8 bytes - the size of a TripleDES block. The initial encryptions will be done using the same IV. We will also perform an encryption of a credit card number with the same TripleDES key but with a different IV, to show the difference made by changing the IV. Then we'll look at the encrypted data and see what we can learn from it.

1. First, let's start by creating a TestCryptoSalt.cs file and copying the code below into it:

//////////////////
///
/// TestCryptoSalt
///
/// Small demo of encryption and salting
///
/// compile command:
/// csc TestCryptoSalt.cs
///
//////////////////
using System;
using System.Collections;
using System.Text;
using System.Security.Cryptography;

namespace TestCryptoSalt
{
/// <summary>
/// Showing effect of salt on encryption
/// </summary>
    class TestCryptoSalt
    {
        /// <summary>
        /// The main entry point for the application.
        /// </summary>
        [STAThread]
        static void Main(string[] args)
        {
            crypto_test_no_salt();
        }

        /// <summary>
        /// Helper function for hexadecimal display of a byte array
        ///
        /// Arguments:
        /// str - a description to be output before the hex print
        /// data - the byte array to print
        /// </summary>
        static void print_bytes(string str, byte[] data)
        {
            Console.WriteLine("\n" + str);
            Console.Write("\t");
            for (int i = 0; i < data.Length; i++)
            {
                Console.Write(String.Format("0x{0:X}", data[i]));
                Console.Write(' ');
                if ((i + 1) % 8 == 0)
                {
                    Console.Write("\n\t");
                }
            }
            Console.WriteLine();
        }

        /// <summary>
        /// The main test
        /// </summary>
        static void crypto_test_no_salt()
        {
            // Two made-up credit card numbers with identical first half
            //
            string strCC1 = "4011023411013758";
            string strCC2 = "4011023410254622";

            // Display CC1
            //
            Console.WriteLine("\nCC1: " + strCC1);
            byte[] bytesCC1 = Encoding.ASCII.GetBytes(strCC1);
            print_bytes("CC1 (bytes):", bytesCC1);

            // Display CC2
            //
            Console.WriteLine("\nCC2: " + strCC2);
            byte[] bytesCC2 = Encoding.ASCII.GetBytes(strCC2);
            print_bytes("CC2 (bytes):", bytesCC2);

            // Create a TripleDES key
            // We'll use the Cipher Block Chaining encryption mode
            //
            TripleDES tdes = TripleDES.Create();
            tdes.KeySize = 128;
            tdes.Mode = CipherMode.CBC;
            tdes.GenerateKey();

            // Display the key and its Initialization Vector
            //
            print_bytes("Triple DES key (bytes):", tdes.Key);
            print_bytes("Triple DES IV (bytes):", tdes.IV);

            // Encrypt CC1
            //
            ICryptoTransform tdes_encr = tdes.CreateEncryptor();
            byte[] bytesCC1Encrypted = tdes_encr.TransformFinalBlock(bytesCC1, 0, bytesCC1.Length);
            print_bytes("Triple DES encryption of CC1 (bytes):", bytesCC1Encrypted);

            // Encrypt CC2
            //
            byte[] bytesCC2Encrypted = tdes_encr.TransformFinalBlock(bytesCC2, 0, bytesCC2.Length);
            print_bytes("Triple DES encryption of CC2 (bytes):", bytesCC2Encrypted);

            // Set a new Initialization Vector
            //
            tdes.GenerateIV();

            // Display the key and its Initialization Vector
            //
            print_bytes("Triple DES key (bytes):", tdes.Key);
            print_bytes("Triple DES IV (bytes):", tdes.IV);

            // Re-encrypt CC2 using the new IV
            //
            tdes_encr = tdes.CreateEncryptor(tdes.Key, tdes.IV);
            byte[] bytesCC2EncryptedNewIV = tdes_encr.TransformFinalBlock(bytesCC2, 0, bytesCC2.Length);
            print_bytes("Triple DES encryption of CC2 using new IV (bytes):", bytesCC2EncryptedNewIV);

            // Now attempt to decrypt, just for fun, to show CBC's nice recovery
            // Only the last encryption will be fully decrypted correctly,
            // because the first two were made using a different IV
            //
            ICryptoTransform tdes_decr = tdes.CreateDecryptor();
            Console.WriteLine("\nRetrieving encrypted CC1: "
                + Encoding.ASCII.GetString(tdes_decr.TransformFinalBlock(bytesCC1Encrypted, 0, bytesCC1Encrypted.Length)));
            Console.WriteLine("\nRetrieving encrypted CC2: "
                + Encoding.ASCII.GetString(tdes_decr.TransformFinalBlock(bytesCC2Encrypted, 0, bytesCC2Encrypted.Length)));
            Console.WriteLine("\nRetrieving encrypted CC2: "
                + Encoding.ASCII.GetString(tdes_decr.TransformFinalBlock(bytesCC2EncryptedNewIV, 0, bytesCC2EncryptedNewIV.Length)));
        }
    }
}

2. Now, let's compile this file using the csc compiler. For example, on my system, the command for compiling the demo looks like this:

C:\WINNT\Microsoft.NET\Framework\v2.0.50727\csc.exe TestCryptoSalt.cs

3. Let's execute the program. The output will differ each time the program is run because a new encryption key will be generated with each execution. On my system, I got the following output:

CC1: 4011023411013758

CC1 (bytes):
0x34 0x30 0x31 0x31 0x30 0x32 0x33 0x34
0x31 0x31 0x30 0x31 0x33 0x37 0x35 0x38

CC2: 4011023410254622

CC2 (bytes):
0x34 0x30 0x31 0x31 0x30 0x32 0x33 0x34
0x31 0x30 0x32 0x35 0x34 0x36 0x32 0x32

Triple DES key (bytes):
0xD 0xDC 0xFC 0xC4 0xD7 0xD 0xB7 0xB8
0x92 0xC5 0x77 0xA8 0xD 0x96 0x1 0xBD

Triple DES IV (bytes):
0x1A 0x42 0xD0 0x17 0x9E 0xA1 0xE6 0xF4

Triple DES encryption of CC1 (bytes):
        0x26 0x8B 0xEC 0x7C 0x56 0x5F 0x5A 0xD3
        0x13 0xB4 0xB6 0xCB 0x7F 0x80 0x52 0x49
        0x12 0xBB 0x89 0x7 0x37 0xF6 0x9F 0x69

Triple DES encryption of CC2 (bytes):
        0x26 0x8B 0xEC 0x7C 0x56 0x5F 0x5A 0xD3
        0x3B 0x8F 0xB0 0xAD 0x4C 0x14 0x1 0xB2
        0x76 0xBD 0x56 0x8C 0x52 0x6C 0x22 0xD

Triple DES key (bytes):
0xD 0xDC 0xFC 0xC4 0xD7 0xD 0xB7 0xB8
0x92 0xC5 0x77 0xA8 0xD 0x96 0x1 0xBD

Triple DES IV (bytes):
0x4D 0x25 0x8E 0xEC 0x38 0xF7 0xA6 0xE9

Triple DES encryption of CC2 using new IV (bytes):
        0xC6 0x49 0xD3 0x43 0x79 0x96 0xF7 0xE6
        0x50 0xC3 0xAC 0x5E 0x27 0x9C 0xF9 0xF
        0x60 0x3 0xB 0x2B 0x4A 0xA1 0x8D 0x2B

Retrieving encrypted CC1: cWo??ds)11013758

Retrieving encrypted CC2: cWo??ds)10254622

Retrieving encrypted CC2: 4011023410254622

4. Let's discuss the program output.

We start by printing out the two credit numbers in ASCII and hexadecimal representations. The hexadecimal printouts are arranged so that we have 8 bytes per line - the size of a TripleDES block. This will make it easier to compare identical blocks.

We then print the TripleDES key - it has 16 bytes - 128 bits. The IV is 8 bytes, the size of a block.

Looking at the encryptions of CC1 and CC2 that are done with the same IV, we can notice that the first blocks are identical. This happens because the two credit card numbers share their first eight digits, and the encryption discloses this fact. The credit card numbers actually share their first nine digits, but the nineth digit will be encrypted as part of the second block, so we cannot see that it is shared just by looking at the encrypted output.

The program is now generating a new IV and prints again the key and this IV. We can observe that the key is the same as before; only the IV has changed. The second credit card number is now encrypted using this new IV and this time the output has no common pattern with the previous outputs. We can no longer detect that this encryption was made for a plaintext that has anything in common with the plaintexts from which the previous two ciphertexts were generated.

Finally, using the last IV, we will attempt to decrypt the three ciphertexts. This is just a side-demo that illustrates how the CBC chaining mode recovers from an error. Because we are using the wrong IV for the first two ciphertexts, we are not able to decrypt those ciphertexts completely: the first block is garbled because the IV is incorrect, however, the subsequent blocks are decrypted correctly and we can retrieve the second halves of the credit card numbers. Of course, the decryption of the third ciphertext works as expected and decrypts the entire credit card number.

Conclusions

We have seen in this demo that by using a fixed IV, prefix patterns in cleartext data can be reflected in the ciphertext data. The common prefix has to be at least as large as an encryption block for this to happen. In the example, because the numbers were stored as ASCII, an 8 digit common prefix would be reflected in the ciphertext. If we would have encrypted credit card numbers as Unicode, a common 4 digit prefix would show up in the ciphertext. It's also not uncommon for credit card numbers or social security numbers to have common prefixes; for example, the first 4 digits of a credit card number are used to identify the card type.

Of course, knowing that two credit card numbers have the same prefix doesn't automatically allow us to find out what those credit card numbers are. But it's a very good start. If we can identify one number from some other information (maybe we inserted one of the records), then we automatically find out information about the other number, and this doesn't require any number crunching.

So, the reason why SQL Server 2005 encryption procedures are using a random IV for each encryption is to provide the safest use of encryption. It's not sufficient to use a good encryption algorithm, it's also necessary to use it well, and block encryption algorithms should be used with a chaining mode and a random IV for each encryption.