The NTLM Authentication protocol is an old relic. Microsoft, the inventor of the protocol, itself discourages its use and recommends using Kerberos.
But I expect NTLM to be around for a long time. The reason is Kerberos’s use of Key Distribution Center (KDC) and Ticket Granting Server (TGS). These two entities are generally co-located but even then, this is a third entity. In case of simple client and server authentication where the server is not KDC, Kerberos can not be used. Also, another requirement is that both the client and server should be joined to a domain. If any one of them is not, NTLM will be used.
Recently I was working on an issue regarding NTLM session security and I noticed that customer was using terminology for NTLM Authentication Protocol that is not present in [MS-NLMP], the official specification for the NTLM authentication protocol. The customer was using terminology presented in The NTLM Authentication Protocol and Security Support Provider.
This was not the first time that this had happened. We have had similar cases in the past. So, to make communication more efficient and productive, I decided to write this blog where I’ll present terminology used in The NTLM Authentication Protocol and Security Support Provider and its equivalent used in [MS-NLMP].
For the rest of this blog, I’ll use NTLM.html as an abbreviation for The NTLM Authentication Protocol and Security Support Provider. I’ll not list the terms that are not present in [MS-NLMP] but intuitive.
ConcatenationOf(DES(LMOWFv1[0..6], LmChallengeResponse[0..7]), DES(ConcatenationOf(LMOWFv1, 0xBDBDBDBDBDBD), LmChallengeResponse[0..7]))
Note: I reproduced this here since at the writing of this blog, MS-NLMP has a bug. The version here is correct and will appear in a future release.
This is the sessionBaseKey when only LM authentication is used. If NTLMSSP_NEGOTIATE_KEY_EXCH is set, this becomes KeyExchangeKey when NTLMSSP_NEGOTIATE_LMKEY flag is set.
Now that you know what LM hash is, a note about session security when NTLMSSP_NEGOTIATE_LMKEY is set.
As explained in Knowledge Base article 299656 (http://support.microsoft.com/kb/299656), newer versions of Windows do not store the LM hash of the password. This generally does not pose a problem since most modern implementations of NTLM do not use LAN Manager (LM) authentication. Even when an LM response is sent, the NTLM response is also sent in the Authenticate message; servers use that to authenticate (if not, authentication will fail). But in a rare case, if you are sending both the LM and NTLM responses and using NTLM session security (NTLMSSP_NEGOTIATE_LMKEY is set), your authentication will succeed but you will not be able to decrypt the messages from the server and the server will not be able to decrypt your messages. The reason is that server will be using an LM hash that is all zeros, since it does not store LM hashes. This will have an effect on calculation of key exchange keys and eventually on sealing keys. To avoid this problem, use Extended Session Security.