Understanding “Connection forcibly closed by remote host” Errors Caused by TOE/Chimney

re: Understanding “Connection forcibly closed by remote host” Errors Caused by TOE/Chimney

Lina — Wed, 06 Jun 2012 19:46:43 GMT

can this error be related to a hack ?

re: Understanding “Connection forcibly closed by remote host” Errors Caused by TOE/Chimney

MRF — Mon, 05 Apr 2010 14:51:56 GMT

We had chronic issues with this type of error that was intermittent and extremely frustrating. It turned out that there was a remote SNMP query running to the firewall that we had to get through to make our database queries. we ran a Traceroute tool and foudn the 100% packet loss for a split second every 4or so minutes that gave us our first real clue. Once this was found and disabled all was well. It was a Cisco router and I do not know if a permanent solutions was found.

re: Understanding “Connection forcibly closed by remote host” Errors Caused by TOE/Chimney

Carlos Goncalves — Thu, 01 Apr 2010 23:41:17 GMT

A connection was successfully established with the server, but then an error occurred during the pre-login handshake. (provider: TCP Provider, error: 0 - An existing connection was forcibly closed by the remote host.) (Microsoft SQL Server, Error: 10054)

An existing connection was forcibly closed by the remote host

The Windows Directors' Cuts — Fri, 05 Jun 2009 17:21:45 GMT

An existing connection was forcibly closed by the remote host

re: Understanding “Connection forcibly closed by remote host” Errors Caused by TOE/Chimney

SQL Server Connectivity — Thu, 08 Jan 2009 01:43:41 GMT

I believe the affected NICs are covered in this KB article: http://support.microsoft.com/kb/942861

I am not positive that this is possible in a VMWare environment, although I believe it should be possible if one or both of the host machines are using one of these NICs.

Note that this is not by any stretch the only possible cause of "Connection forcibly closed", though - malfuncioning network hardware could give exactly the same symptoms, and used to be the main source of these sorts of error messages before this problem came up. TCP Chimney is definitely the first place to look, though, both because it is a common problem and because it is easier to troubleshoot than other kinds of misbehaving switch, NIC, etc...

re: Understanding “Connection forcibly closed by remote host” Errors Caused by TOE/Chimney

nojetlag — Thu, 08 Jan 2009 01:07:55 GMT

Thanks for the very fast response, in our situation we see that behavior between two servers that are both running on VM Ware ESX. Would the mentioned cause also apply for a virtualisation environment like this ? Is there a KB that gives an overview which NICs are affected ?

re: Understanding “Connection forcibly closed by remote host” Errors Caused by TOE/Chimney

SQL Server Connectivity — Wed, 07 Jan 2009 22:42:54 GMT

I actually need to update this blog post, so thanks for reminding me! The root cause of this exact problem is a buggy implementation of keepalive in the NIC driver. Fortunately, in the latest driver version that is available for the affected NICs, that implementation has been fixed, so now rather than turning off this feature entirely, you can update your NIC drivers and get the benefits of TOE.

Hope this helps,

Dan

re: Understanding “Connection forcibly closed by remote host” Errors Caused by TOE/Chimney

nojetlag — Wed, 07 Jan 2009 18:24:51 GMT

You nicely explain for which symptoms one has to look out, however I'm still looking for the reason why this happens. What causes exactly the connections to drop (forced closure) ? Is the TOE implementation so buggy or are their certain szenarios that forces the reset of the tcp connections ?