David LeBlanc's Web Log : Office Crypto

Office 2007 SP2 Encryption Settings

david_leblanc — Thu, 21 May 2009 06:41:00 GMT

Now that we've actually shipped SP2, some of you may be curious about how to use the shiny new encryption. Here's the registry settings:

Registry keys

Base keys (also corresponding Policy keys)

HKCU\Software\Microsoft\Office\12.0\<appname>\Security\Crypto

Name	Type	Default	Description
CompatMode	DWORD	0	Controls encrypted database compatibility: 0 - Legacy format for new files 1 - NextGen format for new files only 2 - All files saved with NextGen format
Context	String		Restrict encryption parameters to those defined in this CNG context
CipherAlgorithm	String		Cipher algorithm to use, optional, CNG string
CipherKeyBits	DWORD		Number of bits to use when creating the cipher key, rounded down to a multiple of 8, optional
CipherChaining	String		Cipher chaining mode to use, optional, CNG string
HashAlgorithm	String		Hash algorithm to use, optional, CNG string
RngAlgorithm	String		Random number generator algorithm to use, optional, CNG string
SaltBytes	DWORD	16	Bytes of salt to use, optional
PasswordSpinCount	DWORD	100000	Number of times to spin (e.g. rehash) the password verifier, optional
NewKeyOnPwdChange	DWORD	1	If non-zero, a new intermediate key is generated when the password is changed. This will cause any extra key encryptors to be removed on save.

Many thanks to my tester for giving me the information in such a nicely formatted and well documented table. Once you have Office 2010 Technical Preview available to you, the same settings should work there as well. Many more thanks to Dan Jump for carefully implementing our design. Note that if you use the new format, then the converter for Office 2003 and earlier won't be able to read them until we update the converters to understand the new encryption.

Legacy RC4 Example on Codeplex

david_leblanc — Sat, 07 Feb 2009 00:52:00 GMT

Just a quick note on this – a customer had a question about the old RC4 40-bit encryption yesterday, and this prodded me into taking some memory dumps of intermediate steps and figuring out where my own example code wasn't working. Fortunately, it wasn't really a problem with the documentation – I'd just made a dumb mistake where I put '5' in a loop instead of '16'. I also had a bug in the ManagedRC4 project, so if you already have that, pick up the new one, too.

At any rate, there are now working examples for all of the currently shipping encryption techniques specified in MS-OFFCRYPTO.

Maybe next we can work on getting code up to do signatures, though I think the xmldsig stuff is fairly well covered by the .NET framework already.

The project is at http://www.codeplex.com/offcrypto

MS-Offcrypto Example Update

david_leblanc — Tue, 13 Jan 2009 23:41:00 GMT

Just a quick note that I've updated the examples. I added an example for the CAPI RC4 encryption that does work. Along the way, I got smarter about managed C++ and C# interop, which turned out to be a bit of an adventure. I didn't find the documentation on MSDN exceptionally helpful in this area. Maybe there's a good book on the topic, but I haven't found it yet. We had a huge amount of snow for this area – accumulations of about 2-3 feet at my house, and I couldn't go anywhere, so that's what I ended up doing.

The reason for the interop adventure was that there is no RC4 implementation in .NET. There's a couple of those on the Internet, but I didn't want to burden people with getting and installing some 3^rd party library. Along the way, I figured out a mystery of CAPI RC4 encryption. Turns out that if you set a 40-bit RC4 key, there are 3 modes of operation:

CRYPT_NO_SALT – nothing is added to the key. Office doesn't set this flag.
Salt added – some bunch of random bits from the hash is added to the key – seems only useful for temporary keys. We don't use this one, either.
Default – basically, if you import a 5 byte (40-bit) key, it is the exact same thing as importing a 16-byte (128 bit) key where the last 11 bytes are all 0, since CAPI does that for you under the covers unless you set CRYPT_NO_SALT.

The sample code that I have now makes this explicit, and we'll update the document to make this clear as well. This only applies to 40-bit

A last thing that ought to be tidied up is that when you decrypt the two parts of the verifier (encrypted salt and encrypted hash of salt), you do not reset the RC4 stream between encryption operations.

I'm working on doing the legacy RC4 example, and when that's done, it will be posted to www.codeplex/offcrypto. After that, I'll probably move on to the new encryption and some of the signing. If you're interested in this, I'd suggest signing up to the RSS feed on the offcrypto project on CodePlex – I may not remember to post here when I change things.

MS-Offcrypto Examples

david_leblanc — Wed, 07 Jan 2009 00:30:00 GMT

In response to some questions I've gotten about details of MS-OFFCRYPTO, I've created a CodePlex project to contain sample code demonstrating the documentation. You can find it at http://www.codeplex.com/offcrypto. I had originally wanted to include sample code in MS-OFFCRYPTO itself, but we couldn't do that. Instead, we can put sample code on CodePlex. To keep it real, I did the work from my home system where I don't have access to the original source, and wrote it in C# instead of the original C++ to shake out library differences.

Please note that the sample code is not intended to replace proper documentation. In the course of helping a customer with their attempt to implement the AES encryption, we figured out a problem with the CryptDeriveKey documentation, and got that updated. If there are any nuances of the approach that are in the sample, but not the documentation, we'll update the document to match. The sample code is there to verify that the documentation is complete, and to help anyone who wants to do this.

Currently, there are 2 projects – the first is ExtractStream. I'd needed a way to get streams out of structured storage so that the rest of the sample code could be a lot simpler, and I'm also not too good at managed-unmanaged interop. You use this app to extract the stream you'd like to parse – the rest of the examples will use this. It may turn out to be a good thing having this as a stand-alone project – there's some other features we can build on this that might be helpful.

The second project is OoxmlEncrypt – it demonstrates parsing and validating an EncryptionInfo stream, as well as validating the password.

I have a third project that I need to post which does the same for the CAPI RC4 encryption, which is default for encrypted PowerPoint files, and can show up as a format for Word and Excel files. That's done – I just need to post it.

A fourth project that I'm working on (hindered by the fact that .NET has no RC4) is to demonstrate the legacy 40-bit RC4.

After that, as I find time, I'll move on to signatures. BTW, if someone else would like to contribute to the project, that's certainly possible – just let me know, and sign a release, then we can add other people as devs.

Office Crypto KDF Details

david_leblanc — Sat, 06 Dec 2008 02:08:00 GMT

I've gotten a couple of questions asking how our key derivation function works. The technique is very similar to that described in RFC 2898, also known as PKCS #5. There are two key derivation functions (KDF) documented in this RFC – PBKDF1 and PBKDF2. Our KDF implementation is very similar to PBKDF1 (section 5.1), with the following changes:

8 bytes of salt are recommended, we use 16 by default
1000 iterations are recommended, we use 50,000
The function documented hashes the previous hash a number of times. We concatenate the counter with the previous hash, and hash that. This was done on the recommendation of Paul Leach, and makes the hash harder to optimize.
We will allow more advanced hashing algorithms than SHA-1 in the Agile encryption.
The final output is passed into CryptDeriveKey as documented in MS-OFFCRYPTO, agile encryption does something slightly different.

So we're not absolutely using a completely standard KDF, but it's close – and stronger than the standard.

New, Improved Office Crypto

david_leblanc — Fri, 05 Dec 2008 04:39:00 GMT

If you're enough of an Office crypto geek to stay on top of the most recent changes in MS-OFFCRYPTO, you already know about some of this, but my assumption is that most people aren't going to want to parse something that hard to read. What we're doing is introducing some substantial improvements in our encryption in Office 2007 SP2, which are known to MS-OFFCRYPTO as 'ECMA-376 Agile Encryption'.

Our goal as a company is to make all of our encryption agile. Many government customers have requirements to use encryption algorithms specific to their own governments, and we'd like to eventually get to a point where (for example) Office 16 can emit a document with improved encryption that Office 2007 can just pick up and use. In Office, our first attempt at this was how we created password verifiers – we let you use any hashing algorithm we can support on the operating system, and the spin count is also agile. I don't think we created a way to configure Office 2007 to emit password verifiers with a configurable spin count, but if one shows up with a higher spin count created by a future version, it should work – which was the short-term goal.

The encryption we used in Office 2007 for the new file format is pretty robust – even with very advanced techniques, Elcomsoft reports getting around 5000 cracks/sec. Compare this to Acrobat 9, which allows 74.5 million cracks/sec – meaning it takes about 15,000 times more computing power to crack a password on an Office 2007 document than an Acrobat 9 encrypted PDF. If you're interested in the details of where Adobe made some mistakes, check out With 256-bit encryption, Acrobat 9 passwords still easy to crack. Our default encryption uses an iterated SHA-1 hash with an iteration count of 50,000, and AES128 as the encryption algorithm. In order to check a password on our system, you have to perform 50,002 SHA-1 hashing operations, and 2 AES128 decryptions. On Acrobat 9, that's just one SHA-256 hashing operation. To make matters worse, SHA-256 is more efficient than SHA-1. To be fair, we very nearly made the same error until our tester found the problem and I fixed it prior to release. The key here is that when a password is used to protect something, that's going to be the weakest link, and using more bits for the encryption algorithm will not make anything harder to access. Breaking the encryption itself (assuming a modern block cipher) isn't feasible. Brute-forcing the password is feasible, so the order of the day is one of the least well regarded of the Saltzer and Shroeder design principles – work factor. If you can't absolutely stop something, make it so much work that it isn't worth it.

With the new encryption, we're introducing a bunch of very cool stuff:

Configurable symmetric encryption
Cipher-block chaining (CBC or CFB)
Configurable hashing algorithms
Support for block ciphers with block sizes from 2 to 4096 bytes
Configurable salt, up to 64k bytes
Iterated hashing of passwords up to 10 million iterations (default raised to 100,000)
Integrity checking

If you're running on Vista, this means you can use anything that you can write a CNG plug-in to support, which means we'll be able to support Suite-B compliant encryption. To be clear, if you're using a strong password, the existing Office 2007 encryption will do a very good job, but you're restricted to one set of algorithms, and you're missing cipher block chaining and integrity protection.

The reason we're shipping this in a service pack is that we'd like customers to be able to be Suite-B compliant using Office 2007 SP2, or use other private encryption mechanisms. The second reason is that we need Office 2007 SP2 users to be able to consume encrypted documents created by Office 14 when we ship it. We do have some more cool stuff we're working on and I'll blog about that when we get closer to being able to release Office 14. When I first came to Office, we had some fairly awful encryption (see Office Crypto Follies), and with the help of a PM who also serves on the Microsoft crypto board, and some really great devs and testers, I think we now have a first-class product with some of the best document encryption in the industry.

MS-OFFCRYPTO, W7 Engineering blog, etc

david_leblanc — Fri, 17 Oct 2008 01:10:00 GMT

We have a new version of MS-OFFCRYPTO out. The big change is that how CryptDeriveKey was documented on MSDN was incorrect, we copied it, which made our document also incorrect. As it turns out, CryptDeriveKey always uses the same code path for AES as it does as if the hash output is shorter than the needed key. Several people have written me about this, which I do appreciate – it's how we first knew about the problem and got it fixed. If you're trying to make something work with Office 2007 encryption, get the new version.

There's also some interesting stuff that gives a preview of how we'll be doing encryption in the future. Some of this will change in the next update, but the general outline still holds. Some neat stuff that I'll talk about in more detail later – don't have time today.

The other thing that came to my attention is that there's a really interesting blog – the "Engineering Windows 7" blog. Today's entry is written by someone I've known the longest at MS – Larry Osterman. You can read his post here - http://blogs.msdn.com/e7/archive/2008/10/15/engineering-7-a-view-from-the-bottom.aspx. The overall blog is some interesting stuff, and explains a lot of why we make certain trade-offs.

A quick story about Larry that's funny in hindsight. Back around 1997, I was working at ISS, and one of our devs had a VPN into our corporate network. Everything on the ISS corporate network at the time was subject to abuse/testing by the ISS Internet Scanner, and I was dev lead on the Windows version. Our dev had is POP3 server keep falling over, the service would hang, and he had to restart his server. Since we tested the scanner many times per day, his server fell over a lot. We got to the bottom of it, and it had a simple buffer overrun. Since ISS has always done responsible disclosure (before it even had a name), the dev called the company responsible, and they didn't do anything. So I called them, trying to explain how serious this was. They replied to my e-mail by telling me that buffer overruns on Windows were not exploitable (ROTFLMAO, yeah, right – even in 1997, we knew better). I tried in vain to convince them that this was something serious, and told them I would go public with an advisory if they didn't do something.

Finally, it became apparent they weren't going to do anything – they'd quite rudely told me I didn't know what I was talking about, they wouldn't fix it, and that was that. So I wrote up an advisory – pretty informal stuff by today's standards – and because I don't encourage criminal activity, I didn't bother to create an exploit – I just said it crashed, looked exploitable, and the company wouldn't fix it, so your only real option was to use someone else's product. As soon as I did that, I got a very special letter from the company's legal department the very next day. Sigh. One of the reasons I came here – everything I ever asked Microsoft to fix when I was at ISS eventually did get fixed, so I've liked Microsoft's attitude towards fixing security issues since I first interacted with Jim Kelly (another story here) back in early 1996. If you search BUGTRAQ archives on my name and POP3, you might be able to dig up the only time I've ever gone public without a fix.

A public discussion ensued, I think on the old NTBUGTRAQ list. The company continued to assert I was an idiot and the problem wasn't exploitable (simple stack overrun), and Larry came to my defense, which I've always appreciated.

An interesting thing about Larry's blog post – Steven Sinofsky used to run Office. How Larry describes Windows 7 working now is how Office has run for quite a while, thought the "feature crew" concept was new to the O12/Office 2007 development cycle. It works well, and makes life as a dev much better than some other experiences I've had, and also helps ship more predictable, higher quality code, which benefits everyone.

Office Crypto Follies

david_leblanc — Fri, 04 Jul 2008 03:44:00 GMT

What I've been working on lately that has kept me from doing nearly anything else can be found at:

http://msdn.microsoft.com/en-us/library/cc313071.aspx

MS-OFFCRYPTO is very detailed documentation of exactly how we do cryptography for binary and OOXML documents. Overall, it covers:

IRM
Encryption and obfuscation
Password to modify
Digital signing

Before someone else figures out some of this and makes fun of us on Slashdot, let me be the first to detail what's really going on. Hang on – some of it is (in the immortal words of Warren Zevon) not that pretty at all. On a happier note, in the even more immortal words of Monty Python, "it got better" – what we're shipping now is quite good (for encrypting OOXML documents), and we have plans to make it even better.

Let's start with the worst of it – XOR. You may note that I consistently refused to ever say "XOR encryption", preferring the more accurate "XOR obfuscation". Not only is it the worst way to protect a document, but it was horrible to try and explain. We did all sorts of silly things to make this hard to figure out, it did nearly nothing to actually protect the data, but it sure was no fun to try and document in a normative style. I believe this obfuscation dates back to around 1994. Here's some pseudo-code to show you the sheer horror of it all – this is from one of the two password verifier approaches:

FUNCTION CreatePasswordVerifier_Method1
PARAMETERS Password
RETURNS 16-bit unsigned integer
DECLARE Verifier AS 16-bit unsigned integer
DECLARE PasswordArray AS array of 8-bit unsigned integers

SET Verifier TO 0x0000 SET PasswordArray TO (empty array of bytes)
SET PasswordArray[0] TO Password.Length

APPEND Password TO PasswordArray
FOR EACH PasswordByte IN PasswordArray IN REVERSE ORDER
IF (Verifier BITWISE AND 0x4000) is 0x0000
SET Intermediate1 TO 0
ELSE
SET Intermediate1 TO 1
ENDIF

SET Intermediate2 TO Verifier MULTIPLED BY 2
SET most significant bit of Intermediate2 TO 0
SET Intermediate3 TO Intermediate1 BITWISE OR Intermediate2
SET Verifier TO Intermediate3 BITWISE XOR PasswordByte

ENDFOR
RETURN Verifier BITWISE XOR 0xCE4B
END FUNCTION

We'd have been much better off just taking a CRC16 of the password. I wanted to see just how bad this was, and wrote up a quick app to try the first 2^40 alpha passwords, and I started seeing cycles in the collisions. Values would go from under-represented to over-represented very quickly. Further inspection shows that for reasonably short passwords, you can immediately tell the number of characters from this value. Seems that for a 1 character password, the first 7 bits vary, 2 characters vary 8 bits, and so on.

Oddly enough, this is so bad that it actually has a benefit. There's so many collisions that while you may find a password that will work, there's no assurance that you found the password used to obfuscate the file, so you're not as likely to be able to get into other things by brute-forcing the 16-bit verifier. Somewhere around 8 billion passwords only generate about 16,000 verifiers, so there's literally hundreds of thousands possible passwords that could have created any given verifier.

What we have here is that someone who is actually a very good general dev (he's now a well thought of dev manager) who tried to roll his own crypto, and implement a simple hashing function. Moral of the story is DO NOT DO THIS.

If you look deeply into the obfuscation array initialization, you'll see another fairly ghastly mistake – the part of the array that coincides with the number of characters in the password actually varies, but the remainder of it is initialized based on values hard-coded into the binary along with values that end up in the document. This makes it possible to write a tool to directly extract quite a bit of information, and then there's the obvious disaster of what happens when you XOR chosen plain-text.

Our next attempt at encryption first showed up in Office 97, and featured RC4. As those of you who are familiar at all with encryption, RC4 is really hard to do correctly, and this is an example of most of the mistakes you can make with RC4. The number one rule of stream ciphers is to NEVER re-use a key stream, since the crypt text is the result of the cipher stream XOR'd with the plain text. If you reuse a key stream, you can XOR them to get the XOR of the two input plain texts, and that's often very easy to sort out. We did this more than once. Next rule of RC4 is that you have to have an integrity check, or you're subject to bit-flipping attacks – there's no integrity check. Finally, you should toss out the first 1k or so of the cipher stream, but we didn't know to do that.

Next, take into account that encryption was considered a munition at the time, and we were limited to 40-bit initial keys. These days, you can work your way through a 40-bit key space in minutes using only one computer. No need to bother with password cracking, just go directly to the key and attack it. The moral of this story is that agile encryption is a serious requirement, because while it may have taken some time to brute force 2^40 keys on a 286, a modern system will make short work of it – and in fact, it would be possible to store all the keys in a single file – it would only consume about 18 GB (much less if we did it as a tree of some kind). A second and more interesting flaw that our tester uncovered was that they thought they were doing an iterated hash (though only 16 iterations), but what they were doing in reality was concatenating the first 40 bits of the MD5 hash of the password, a 16 byte salt, and then repeating this concatenation 16 times. The old encryption library we used made this an understandable error, and it's harder to do with CryptoAPI or CNG. Moral of this story is that crypto is unlike any other code, and you should always get an expert to review what you're doing.

The RC4 encryption was then "strengthened" to use CryptoAPI, and could be configured to go to 128-bit keys, though unfortunately, the old 40-bit stuff was still default – this all happened in Office XP. Sadly, most of the implementation flaws remained. I found one place where there was triple key stream re-use (though only for 8 bytes) in the same spot. The unfortunate attempt at an iterated hash dropped back to a non-iterated hash of the password for reasons I don't understand. Some of the applications, notably PowerPoint, don't suffer as much from key stream re-use as others, and if you chose to use 128-bit RC4 and used a good password, a presentation could be relatively well protected.

As of Office 2007, we do warn you that the encryption we do on the binary documents is weak. Most of the time, it's so weak that it will only act as a mild deterrent. In some cases, we missed encrypting things entirely (which is actually called out in a KB article some time ago). My advice is that if you must encrypt a binary document, use a 3^rd party tool to do it. These flaws are called out in the Security Considerations section of MS-OFFCRYPTO.

When we get to OOXML document encryption, the picture gets a lot better. I personally fixed some of the problems. We moved to AES, which is a block cipher, went to an iterated hash that far exceeds RFC2898 (50,000 cycles), and as I stated in a previous post, don't forget your password, or you may never see your data again. The only problems we had here was that we didn't support CBC (cipher block chaining) mode, and we didn't have an integrity check, though a 1 bit flip would result in 16 bytes of junk, so it isn't as high priority a problem as with RC4. We'll address both of these problems in the future, as well as some other improvements I'll talk about once we ship it.

If you take a good look at the ODF specification (as of 1.1) with regards to crypto, you see some of the same sorts of issues. They do some interesting things:

The number of times to iterate on the password hash is relatively low (1000), and is a fixed value.
The encryption algorithm must be Blowfish. Blowfish may well be a nice algorithm, but it hasn't been seriously validated, isn't FIPS compliant, isn't Suite-B compliant, and simply can't be used by some customers that require FIPS, Suite-B, or the analogues in their countries. Bruce Schneier himself has this to say about it – "At this point, though, I'm amazed it's still being used. If people ask, I recommend Twofish instead." It would be better to require an algorithm that has been validated, and better yet to allow it to be agile.
The integrity check and the password verifier are the same thing. There is no way to know whether the user has the wrong password, or whether a bit got flipped. This has the potential for data loss, though I'd suppose one could build a special tool to just try the decryption.

The really good news is that we have some people who are seriously good with cryptography on the Office TWC team now – our tester is really sharp, we've got a PM who previously worked in the crypto group in Windows and is on the Microsoft-wide crypto board, and we have some devs who know this stuff as well. I'm happy to say that when you encrypt an OOXML document that it will be very hard to brute force the password and retrieve the information – and it will keep getting better.