Last month, I worked on an incident where a customer couldn't access a web site from Internet Explorer 9 using TLS 1.2. Accessing the same web site was working fine using TLS 1.0. I first thought the issue was similar to the one described in this blog: http://blogs.msdn.com/b/ieinternals/archive/2011/03/25/misbehaving-https-servers-impair-tls-1.1-and-tls-1.2.aspx
However, after gathering a network trace, I couldn't see any RESET coming from the server so we were hitting another scenario than the one from above blog.
Activating SCHANNEL ETL tracing and SCHANNEL event logging (as shown here: http://support.microsoft.com/kb/260729) was showing a cryptic SSL fatal alert event 36888 "The following fatal alert was generated: 40. The internal error state is 252":
Since the issue could easily be reproduces, I debugged it and found out that the TLS 1.2 handshake failure was caused by extra security checks made on signature algorithms used in the server certificate chain. Specifically, the following certificate chain was sent in the server's HELLO and the use of MD5 algorithm was causing the handshake failure:
TLS 1.2 allows the ability to specify the signature algorithms that an end point is willing to accept (http://tools.ietf.org/html/rfc5246#section-126.96.36.199.1 ). This is specified in the client/server hello extension. Here's the client hello for our IE9/TLS 1.2 scenario:
Unfortunately, neither Netmon nor Wireshark provides parsing for the signature_algorithms field but you can do "manual parsing" using the RFC:
Type: signature_algorithms (0x000d)Data (16 bytes)00:0e:04:01:05:01:02:01:04:03:05:03:02:03:02:0204:01 sha256(4) rsa(1)05:01 sha384(5) rsa(1)02:01 sha1(2) rsa(1)04:03 sha256(4) ecdsa(3) 05:03 sha384(5) ecdsa(3)02:03 sha1(2) ecdsa(3)02:02 sha1(2) dsa(2)
As you can see md5 is not listed. This is due to the fact that schannel.dll restricts the use of md5 for security reasons:
[abstract of http://tools.ietf.org/html/rfc5246]
MD5 [MD5] is a hashing function that converts an arbitrarily long data stream into a hash of fixed size (16 bytes). Due to significant progress in cryptanalysis, at the time of publication of this document, MD5 no longer can be considered a 'secure' hashing function.
Currently, TLS 1.1 & 1.2 are not enabled by default in Internet Explorer but this may likely change in future versions. So, be aware that MD5 is no longer considered secure and it's probably time to get rid of your certificates with MD5 signature to avoid problems in the future!
i am experiencing the issue you are describing with IE 11 (TLS 1.2 enabled by default now). But my problem is slightly different since i have CLIENT certificates with an MD5 signature algorithm.
My question: would this problem also apply when MD5 is used as a signature algorithm for client certificates?
Thanks in advance.
Emmanuel - very helpful, thanks!
Marius - yes! We this problem with SHA512RSA hashed client certificates. See also www.michaelm.info/blog.