Larry Osterman's WebLog

Confessions of an Old Fogey
Blog - Title

Should developers learn crypto?

Should developers learn crypto?

  • Comments 20
Over the weekend, Paul Maher posted an article in which he asked if developers needed to understand cryptography.

I responded in his comments, but it needs a bit more detail than I provided in the comments.

I'm all for developers learning about crypto.  But developers also need to understand that it's all well and good for them to understand crypto, as long as they don't ever actually attempt to IMPLEMENT crypto.

Because if they do attempt to implement crypo, they're going to get it wrong.

There have been WAY too many examples of this.  From the initial Microsoft PPTP implementation, to Netscape's SSL implementation (I can't find the reference right now, but the original Netscape SSL implementation used an easily discovered initialization vector which rendered the encryption essentially useless), to the authentication scheme for Dark Age of Camelot, the list goes on.  All of the above issues have long been fixed, but that doesn't matter, because they all share a common flaw.

The root cause of each of these failures was a developer that thought they understood crypto but didn't REALLY understand it.

Whenever a developer decides that they can implement crypto, they need to stop and rethink what they're doing, because they ARE going to get it wrong.

It makes sense for a developer to understand the relative strengths and weaknesses of different crypto solutions, to understand why SHA-1 is better than MD5, etc.

But developers also need to understand that doing cryptography right requires special skills that most developers don't have.

Instead of attempting to roll their own crypto, they should rely on the cryptographic solutions that are built into the platform (CryptoAPI is your friend).  If you stick to existing implementations, you're less likely to mess it up.

And whatever you do, don't attempt to roll your own authentication scheme - for every way you can mess up crypto, there are a dozen ways you can mess up authentication.  See this wonderful dialog (I've referenced it before) for an example of the kind of pitfalls you can hit designing an authentication system.

So it makes sense for a developer to LEARN crypto.  But developers shouldn't believe that they can IMPLEMENT crypto.  Because (with very few exceptions) they can't.

So feel free to learn about crypto - there's a lot of great stuff there.  I highly recommend Simon Singh's "The Code Book" (or the YA version of the same book)  And of course Schneier's Applied Cryptography is a classic.

But recognise that just because you've read a couple of books about cryptography, in general you're not competent to actually implement cryptography.

 

  • Oooo Harsh!

    But sooo true.
  • Even with developers not implementing their own cryptography routines, they still need to learn that cryptography != security. I hear way too often "just use some certificate to encrypt the data" when this is not the solution.
  • JimBob, that is SO true. I'd forgotten about that one :)

    If I hear "We're secure, we use 128 bit crypto" one more time...

    1024 bit crypto with a weak password is just as insecure as 40bit crypto.
  • Microsoft seems to favor S/MIME crypto (certificates, etc.) over PGP / GPG -style crypto (public keys, private keys)

    Is there a reason for this? I find PGP / GPG to be more appealing, as the keys can be easily and freely generated / stored by the end-user, without requiring a Certificate Authority.

    Can I request enhanced support for PGP / GPG in future Crypto API's?
  • Maurits, I have no idea on this one, but my guess is that S/MIME is an open standard, while PGP/GPG isn't (just because there's an open source implementation doesn't mean that the encryption is open).

    There may also be other legal issues associated with GPG/PGP, but IANAL, and wouldn't dream to speculate as to the reasons.
  • Another good book to read is "Security Engineering" by Ross Anderson. It's not primarily a crypto book (though it covers it) but deals with the implementation of security in real-world systems. It's full of fascinating stories (R.A. is a professor of computer security at Cambridge University) and it will teach you not to treat cryptography as a panacea.
  • DO NOT IMPLEMENT is not full clause. There are so many ways to loose security even by standard way.

    As you know, Whidbey introduces SecureString. Just think about how passwords used before Whidbey: they was simple strings!

    Don't you think GC shuffles string objects around and fills all the heap with your password? Oh, you should think so. Because GC really does shuffles password strings around the heap. And you get in trouble even you are not trying to IMPLEMENT any crypto!

    So really working clause is: don't know what you not know.

    In other words, not trying to play with security is much more secure than randomly adding security to some places.
  • FYI: Bruce Schneier has reported that SHA-1 has been effectively broken:
    http://www.schneier.com/blog/archives/2005/02/cryptanalysis_o.html

    Probably time to find a stronger algorithm...
  • I wouldn't say it has been "effectively broken", it's just that with this new result a 160-bit SHA-1 hash is as secure as a 124-bit hash.

    As he says in his blog, "that is just on the far edge of feasibility with current technology" and like with all security, there's a tradeoff. If you need your hash to be secure for 100 years, then SHA-1 might not be great (but then, nothing probably will be). If you only need to it be secure for a couple of hours or a couple of days, then it'll be fine for a while yet.

    But Larry's right in that we should be looking towards the standard APIs for our crypto. Even implementing a standard algorithm yourself isn't going to work, because the devil is most definately in the details with crypto, and the most innocuous bug can easily be your undoing.
  • Maurits:

    I'm a PKI tester, but not the new API fairy, so it's beyond my powers to grant your wish. I don't forsee any special support for PGP/GPG in the OS, but that's just my own personal opinion and not the official word of Microsoft.

    PKI does sort of have the public key/private key notion. It's just that the public key is effectively the certificate (not technically true, but kinda).

    Want to generate certs and private keys but don't want the hassle of a CA? Great! You can use makecert.exe (SDK tool) to create a self-signed cert. Why don't we make this easy to do via UI you might ask. Imagine all the trust implications in that. How easy would it be to spoof an identity if this were the way that most end users got their certificates? (Answer: really incredibly easy.) PGP/GPG deals with that problem by having key servers. If you trust the server (and your connection to it et al.) you can trust what it tells you about a key's owner. Getting certs from a CA avoids the problem by allowing users to trust the CA via it's own certificate (the private key of which was used to sign issued certs). Of course there's the problem of getting root and intermediate CA certs in some trustworthy way so that you can automagically trust end certs, but I think I've probably rambled enough. I don't want to start the great PGP vs PKI wars in Larry's blog comments.

    Also, since you seem to be asking about APIs, the API to create a self-signed cert is the aptly-named CertCreateSelfSignCertificate.

    Hope this helped.
  • Agreed. Most of us are not mathematicans of that particular field so we should not mass with the job that requires real experts.
  • I think I can safely say that "your absolutely correct." Developers by profession are not as mathematically astute to "write/invent" old/new forms of cryptography from scratch (some may be but they are generally the ones writing it anyways). I took a class in cryptography at my college and one of the first things we learnt was, use the API's provided and NEVER try and invent new forms of cryptography in a development enviroment (research is ok because someone will spot the errors if there are any).

    That said, I got to say the System.Security.Cryptography & System.Security.Cryptography2 (also the X509 certificate assemblies) API's are HORRENDOUS! Whoever designed these doesn't work in the real world. I had to use these api's to make a MSN Messenger style chat program with Public Key Authentication, end-to-end encrypted messages etc all using Digital Certificates, synchronous and asynchronous encryption schemes. Now I admit, I was new to the whole Crypto implementation domain so expected some work to learn this but when I had to start writing Win32 code in order to make this work (such as getting the modulus and exponents from a Digital Certificate), I threw up my hands in frustration. What is the point of having nice assemblies in .NET and pushing for security when your assemblies can't provide functionality. Not to mention how unstable it is. I managed to create havoc on my machine and even corrupt the execution engine and get a "System.ExecutionEngineExceptio­n"!!!

    I love crytography but never again do I hope to use it with the assemblies provided by .NET...until they are fixed. Sorry I'm being kinda harsh but you cannot believe the amount of time I wasted trying to get simple encryption/decryption to work using Digital Certificates.
  • Sushant:
    You're not being harsh.

    There's a lot of added support in Whidbey. I realize that doesn't help you now, but lots of folks here in Redmond had problems with what was previously offered in System.Security.Cryptography. We had to recommend that developers use Capicom to round out the missing functionality. P/Invoking all the Win32 APIs would be incredibly painful.

    That said (DevDiv is gonna hate me), if you write something you want government employees (nice market to grab) to run you're better off coding to the Win32 APIs. They either have or will soon (depending on OS and I've lost track of where each one is) meet Common Criteria. I haven't seen .Net do that.

    I should really start my own blog instead of spending my time responding to people in Larry's comments. Oh well.
  • "CryptoAPI is your friend", yet MS intitial PPTP implementation wasn't optimal (to say the least)? I call BS one way or another.

    Dogfood?

    Developers shouldn't have to learn cryptography (it's a very large area where you need to know quite a bit of math from more than one discipline) unless they try to implement anything touching the area. If they *ever* implement anything in the area, especially in the context of a company the size of MS, I expect them to be widely published and have a solid bacground and recognition.

    What? MS crypto "specialists" doesn't fulfill these simple requests?

    (while not in the same league: MS's rand() function still have only 15 bits of "randomness"...)
  • Spot on - developers should learn the basics of cryptography & when to use it & never implement it themselves. But they should also learn which algorithms have flaws and which ones they should be using depending on the situation.

    For instance, if you're looking for a hashing algorithm, and you're using .net you'll have the choice of MD5 and SHA1 through SHA512. But MD5 isn't as secure as previously thought... so you'd generally want to avoid using it in favour of SHA256 or SHA512, depending on the situation...

    The same goes for 2-way encryption. DES is obsolete, but you might still unwittingly use it in .NET...
Page 1 of 2 (20 items) 12