Updated 4/21/2010: The Windows patch to solve this problem has been released. It can be found here.
Recently, while dogfooding our internal pioneer server, a few members of our team started to notice this error message when downloading files from TFS:
C:\dev\code\somefile.txt: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.
Luckily, we were able to reproduce the issue on a consistent enough basis to investigate the issue. After a few fun days of client-side TFS logging, sever-side TFS logging, TFS activity logging, fiddler tracing, netmon tracing, procmon tracing and finally http.sys tracing we were able to determine that we were being affected by an http.sys bug that was introduced in Windows Server 2008 R2.
Are you experiencing the same issue that we were?
Let’s find out. First of all, this bug is new to Windows Server 2008 R2. If you are running any version of Windows Server 2003, Windows Server 2008 R1 RTM or Windows Server 2008 R1 SP1 then you are most likely not hitting the same bug that we were. The issue tended to occur more often if the file that was being downloaded was relatively large (> 2MB) and rate of occurrence seemed to grow with the size of the file. Finally, we were only able to reproduce the issue over slower networks (3 to 4 kb/sec file transfer rate).
Great news! The bug has been fixed!
This bug has been fixed by the Windows team and they have released a QFE for it. You can find the QFE here. You will need to install in on all of your ATs.
The following lists the initial workarounds that we were able to come up with before this bug was fixed. I am going to leave them up here for reference but the real solution should be to install the QFE mentioned above.
In order to lower the threshold follow these directions:
In general, the lower the threshold is, the lower the chances are that you hit the issue above. However, you may not want to lower the setting too much because then connections that should be killed by this timer will not be appropriately cleaned up. Play around with values to see what works best for you.
If you have any other questions, be sure to let us know.