Larry Osterman's WebLog

Confessions of an Old Fogey
Blog - Title

Compressible Encryption

Compressible Encryption

  • Comments 23

Time to spread a smidge of dirt on Microsoft :).

One of my favorite dialog boxes is found in Outlook.  If you dig deep enough into your email accounts, you'll find the following dialog box:

  Outlook Offline File Settings Dialog

The reason I like this dialog box is the default setting "Compressible Encryption".  Why?  Because if you select it, you're not encrypting ANYTHING.  "Compressible Encryption" is really "compressed".  When this option is selected, the data in the OST in specified is compressed (I'm not sure of the algorithm).

Calling a compressed OST file "encrypted" is sort of like saying that a ZIP file is an encrypted version of the file.  After all, if you look at the contents of the ZIP file, you'll not find any the information directly represents the original file (ok, the filenames might be in the archive uncompressed but that's about it).  But of course it's not encrypted.

If you specify "High Encryption" then you get a truly encrypted OST file.  I'm not sure of the algorithms they use, but it IS really encrypted.

So why on earth do they call it compressible encryption?  Well, I'm not 100% sure, but I suspect that the answer is that some executive decided to type their PST file (or OST file) and noticed that their email was found in clear text within the file.

They also noticed that if they used compression on the PST file, then they weren't able to see the contents of the file.  So they equated compression with encryption (hey, they couldn't see the data, could they?).  And thus "compressible encryption" was born.

It's really just a silly affectation - they should never have called it "encryption" because someone might think that the data's actually hidden, but...  If the dialog was being designed today (the actual dialogs over 10 years old), the term "encryption" would never be used but nowadays it's sort-of a historical oddity.

If you do a search for "compressible encryption", the first Google and MSN search hit is Leo Notenboom's article on compressable encryption, here's the official KB article on compressible encryption.

There are other examples where similar obfuscation has occurred, and I'm sure that other vendors have done similar things (or worse).  For example, the Exchange MAPI client-to-Exchange Server protocol is obfuscated because an executive noticed that if he took a network sniff of the traffic between his client and the Exchange server he could see his email messages going by.  So he asked the team to obfuscated the stream - we knew that it did nothing, and so did the executive, but as he pointed out, it's enough to protect from casual attackers.  If you really want encrypted communications, then if you specify the "Encrypt data between Microsoft Office Outlook and Microsoft Exchange Server" option in the Security tab of the Microsoft Exchange Server dialog, then that specifies RPC_C_AUTHN_LEVEL_PKT_PRIVACY, which uses a encryption mechanism to protect the data (I believe it's DES-56 but I'm not 100% sure).  I believe that this option is the default in all current versions of Outlook, but I'm not 100% sure.

  • That's an amusing dialogue, alright! The two things are orthogonal. Generally compression will look to replace repeating sequences of bytes with a shorter sequence, but encryption aims to disguise such repeating sequences.
  • Hahaha. Every time I'd hit that dialog when exporting a PST, I'd wish they would support compression, but no encryption. Guess that answers that question.
  • So is it customary for Microsoft executives to type/cat data files and sniff network packets?
  • Kiliman,
    I don't know. Probably. Execs love poking at things, it's part of their job :)

    I don't know what's up with Outlook, but in the Exchange case (since I started on the team shortly after this happened), the issue was that the team was looking closely at the number of RPCs and number of bytes. The exec in question was being helpful when he noticed this. Since most of the executives at Microsoft come from the ranks, they like to get their feet wet from time-to-time.

    In general, it's a good thing - it means that the exectives are intimately involved in our product.

    I still remember getting mail from the VP of our division the day after we checked in some stuff to the main Windows tree - he was testing our stuff and found a problem (it turned out not to be our bug, but it doesn't matter).
  • @John

    The usual sequence is to compress first, then encrypt. It's double the overhead, but if it shortens transfer time/storage size overall it's worth it.

    Encrypted TIFFs and PSDs do this, among other formats.
  • We used to do a similar thing in a software project I worked on once. It had a "login" packet with the username and password, and somewhere in the midst of time (the product was about 15 years old when I started working on it) someone had decided that cleartext usernames and passwords were a Bad Thing.

    So they decided that "encrypting" the packet by XOR'ing it with a 32-bit integer would be enough.

    To be fair, the software ran on lots of different hardware: PCs, Unix, even mainframes, and it had to use the same scheme on all of them (since the same client apps had to be able to talk to it without knowing what they were talking to -- I'd hate to try and write a strong encryption algorithm on a mainframe in COBOL... hell I'd hate to write *anything* on a mainframe in COBOL ;) and it really was only meant to stop the "casual" obvserver...
  • The current IE dialog to "Open..." stuff bugs me. You hit browse, select a file and then have to hit the open button... But what else would you be trying to do? It adds an extra useless click and IMO was poorly designed.
  • Your post is in direct conflict with the offcial KB article. You say that a "Compressible Encryption" file IS compressed, the KB say that it CAN BE compressed by na external program.

    The way it looks to me is that they had initialy two options: No Encryption and Encryption. Them some PHB complained that the encrypted file was "incompatible" with WinZip, so the developers come up with a weak encryption that generates compressible files.
  • "I believe it's DES-56 but I'm not 100% sure"

    AFAIK it's negotiated and also depends on the SSPI.
    By default it's 56/128 bit RC4.

    Since RC4 is not FIPS 140-1 approved, compliant applications have to use the SChannel SSPI and some registry tweaks to get stronger encryption.

    I wish it would be way easier to specify this kind of stuff.
  • "The current IE dialog to "Open..." stuff bugs me. You hit browse, select a file and then have to hit the open button... But what else would you be trying to do? It adds an extra useless click and IMO was poorly designed."

    What happens if accidentally click on the wrong file? I don't want it opened automatically as soon as I click on it.
  • Daniel,
    I know the article is in conflict with the KB article.
    I'm trying to figure out which of the two is correct.

    If the KB article is right and I'm wrong, then "compressible encryption" isn't compression, but instead is simple obfuscation (ie xoring the data with 0x55AA).

    But it was originally explained as being compressed to me.
  • There are no two ways about it. If you call something encryption when it isn't encryption, you are lying. If you continue the lie for 10 years, you are dishonest.
    This is sad because Microsoft is normally more ethicals than its critics claim.
  • Some notes on compression and encryption:

    Encrypted data cannot be compressed. If it can, the encryption algorithm used is unsound because in any sound algorithm the resulting ciphertext appears random. Random data is generally uncompressable, a basic result from information theory.

    It is common to compress data *before* encrypting it. This reduces the final size of the ciphertext and is popular when encrypting large amounts of data.

    Counterintuitively, there are some negative security implications of encrypting compressed data as well. In particular, the final size of the ciphertext reveals more information about the original plaintext. A common example is that in a database of encrypted records, two records that compress to the same length give an attacker a clue that they may contain the same values. This represents information leakage to an attacker.
  • "What happens if accidentally click on the wrong file? I don't want it opened automatically as soon as I click on it."

    There are two dialogs -- the first one gives you a combo box to pick a recent address or type on in, and then if you click "Browse..." a standard "Open" dialog appears. Selecting a file from there takes you back to the original dialog, and you have to click "Open" on that one too. Totally pointless, and ugly to boot.
  • Fair point!
Page 1 of 2 (23 items) 12