Understanding “Connection forcibly closed by remote host” Errors Caused by TOE/Chimney

re: Understanding “Connection forcibly closed by remote host” Errors Caused by TOE/Chimney

nojetlag — Wed, 07 Jan 2009 18:24:51 GMT

You nicely explain for which symptoms one has to look out, however I'm still looking for the reason why this happens. What causes exactly the connections to drop (forced closure) ? Is the TOE implementation so buggy or are their certain szenarios that forces the reset of the tcp connections ?

re: Understanding “Connection forcibly closed by remote host” Errors Caused by TOE/Chimney

SQL Protocols — Wed, 07 Jan 2009 22:42:54 GMT

I actually need to update this blog post, so thanks for reminding me! The root cause of this exact problem is a buggy implementation of keepalive in the NIC driver. Fortunately, in the latest driver version that is available for the affected NICs, that implementation has been fixed, so now rather than turning off this feature entirely, you can update your NIC drivers and get the benefits of TOE.

Hope this helps,

Dan

re: Understanding “Connection forcibly closed by remote host” Errors Caused by TOE/Chimney

nojetlag — Thu, 08 Jan 2009 01:07:55 GMT

Thanks for the very fast response, in our situation we see that behavior between two servers that are both running on VM Ware ESX. Would the mentioned cause also apply for a virtualisation environment like this ? Is there a KB that gives an overview which NICs are affected ?

re: Understanding “Connection forcibly closed by remote host” Errors Caused by TOE/Chimney

SQL Protocols — Thu, 08 Jan 2009 01:43:41 GMT

I believe the affected NICs are covered in this KB article: http://support.microsoft.com/kb/942861

I am not positive that this is possible in a VMWare environment, although I believe it should be possible if one or both of the host machines are using one of these NICs.

Note that this is not by any stretch the only possible cause of "Connection forcibly closed", though - malfuncioning network hardware could give exactly the same symptoms, and used to be the main source of these sorts of error messages before this problem came up. TCP Chimney is definitely the first place to look, though, both because it is a common problem and because it is easier to troubleshoot than other kinds of misbehaving switch, NIC, etc...

An existing connection was forcibly closed by the remote host

The Windows Directors' Cuts — Fri, 05 Jun 2009 17:21:45 GMT

An existing connection was forcibly closed by the remote host