Windows Core Networking : TCP/IP

Ready for Windows Server 2008?

wndpteam — Tue, 08 Jan 2008 02:48:46 GMT

Windows Server 2008 incorporates the completely rewritten TCP/IP stack we shipped in Windows Vista and since this is the first server release with the stack we'd like to ask our readers to load up Windows Server RC1 and see if your applications are ready.

The new features (compared to Windows Server 2003) that may pose potential compatibility issues include: removed filter hooks, receive window auto tuning, dual IP layer architecture for IPv6, compound TCP, ECN support, default strong host model, easier kernel mode programming, extensive protocol offload.

Windows Server 2008 is also the very first Microsoft server product to have the firewall ON by default. Third party applications that are not aware of this behavior may break.

-- Ari and Katarzyna

Receive Window Auto-Tuning on Vista.

wndpteam — Fri, 06 Jul 2007 02:02:00 GMT

Hi, my name is Katarzyna and I am the Program Manager within the Internet Protocols team. I have been asked a few times about the Receive Window Auto-Tuning feature on Vista and some associated issues people are having.

One of the many cool new features on Windows Vista, Receive Window Auto-Tuning enables the networking stack to receive data more efficiently than on XP. Auto-Tuning allows the operating system to continually monitor the routing conditions (bandwidth, network delay, application delay) and configure connections (scale the TCP Receiving Window) so as to maximize the network performance.In some high bandwidth, high latency links, we have seen SMB performance improvement up to 20 times!

In every TCP packet there is a "window" field, which informs the receiver how much data the sender can accept back. This window controls the flow by setting a threshold on data kept "in flight" and prevents overwhelming the receiver with data that it cannot accept.

The TCP window field is 16 bits wide, allowing for a maximum window size of 64KB, which used to meet requirements of many older networks. Nowadays, however, network interfaces can handle larger packets and keep more of them in flight at any given time. Thus, a larger TCP window has become necessary; especially on high-speed, high latency networks. To fill such a long, fat pipe and make use of the available bandwidth, the sending system can often require very large windows for good performance.

The solution to this demand is called "window scaling”, described back in 1992 in RFC 1323. It introduces an eight-bit scale factor, which serves as a multiplication factor for the window width. After the factor has been negotiated, window values used by that system on a given connection will be shifted to the left by that scale factor; a window scale of zero, thus, implies no scaling at all, while a scale factor of six implies that window sizes should be shifted six bits, thus multiplied by 2^6 = 64. Now a window greater than 64KB can be easily expressed (e.g., 128KB) by setting the scale factor (e.g., 6) and keeping the window field under the original 16 bits (here, 2048).

The window size included in all packets is modified by the scale factor, which is negotiated once at the very beginning of a TCP connection. The connection requestor suggests window scaling factor in its original SYN packet and if the SYN+ACK packet sent in response contains the option, then this particular value will be used on this connection. The scale factor cannot be changed after the initial setup handshake; remaining data transfers on this connection will implicitly use the negotiated value.

Older routers and firewalls however do not handle window scaling correctly leaving the option in the original SYN packet but setting the connection’s scale factor to zero. Seeing the option on, the receiver responds with its own window scale factor. Believing that its scale factor has been accepted, the initiator scales the window appropriately while the receiver thinks that a scale factor of zero is applied and thus a small window of data should follow. As a result, the communication is slow at best. Sometimes, small window packets are dropped by the routers, essentially breaking the connection.

The resulting slow data transfers or loss of connectivity, users may experience as slow or hung networking applications. Remote Desktop Connection and network file copy are two scenarios particularly hurt by misbehaving routers.

If your connection from a Vista machine appears slow or hung, here are some steps to isolate the cause:

First, make sure that your firewall and router can support window scaling. Some devices from Linksys, Cisco, NetApp, SonicWall, Netgear, Checkpoint, D-Link were reported as having problems with window scaling. (Some of the incompatible devices are given here. You can check with the manufacturer or run the connectivity diagnostic suite (especially, TCP High Performance Test) provided by Microsoft to determine your gateway device’s compliance.
Second, check with the manufacturer if a firmware update has been issued for your device that can fix the problem. Replace the problematic device or update the firmware as suggested by the manufacturer. If the router cannot be replaced or if it the device is remote (e.g., a firewall of your ISP or corporation)
Third, If the problem still persists, you can restrict autotuning by running “netsh interface tcp set global autotuninglevel=restricted” from the command prompt. We have found that restricted mode will often allow some of the benefits of autotuning with a number of problematic devices.
Lastly, if all else fails, in order to disable this feature, run "netsh interface tcp set global autotuninglevel=disabled".
(In order to reenable autotuning, run “netsh interface tcp set global autotuninglevel=normal”.)

Please refer to the following KB articles for more information:

-- Katarzyna

Updated: Broken link to KB 932170
Update 2: Changed the guidance to do restricted before disabled.
Update 3: tunning doesn't have two "n"s. :)
Update 4: no really, tuning doesn't have two "n"s.

TDI Client to Winsock Kernel (WSK) Porting Survey

wndpteam — Thu, 19 Oct 2006 19:19:42 GMT

As you know per previous posts on this blog, Winsock Kernel, a new transport-independent kernel mode Network Programming Interface (NPI), is available on Windows Vista and Windows Server Longhorn platforms. On Windows Vista & Windows Server Longhorn, TDI is still supported for compatibility reasons; however, it has been implemented using a translation layer and thus its performance is sub-optimal resulting in performance degradation for TDI clients. For this reason, as well as others (TDI on path to deprecation, etc -- see resource links below), drivers should opt to use WSK whenever possible.

To help us understand how we can best make WSK adoption and porting from TDI to WSK straightforward, we would appreciate it if you could respond to the following questions. Responses can be sent to wskapi at microsoft.com

If you have an existing TDI Client driver:

1) Which operating systems does your TDI client support (Windows XP and/or Windows Server 2003, etc)?

2) Do you plan to port your TDI client to WSK on Windows Vista / Windows Longhorn server OR do you plan to continue to use your TDI-based driver on Windows Vista / Windows Longhorn Server?

a. If WSK was also supported on Windows XP and/or Windows Server 2003, would your answer to question #2 above change? If so, how?

If you are working on a new project which requires a kernel mode driver with network capabilities:

3) Do you plan to support Windows XP / Windows 2003 with the project deliverable? If so, would you use WSK for the project if it was available on these platforms?

Additional Requirements:

4) Do you have any requirements (performance, etc) that would need to be met by WSK (on pre Windows Vista platforms) in order for WSK to be an option on such platforms?

WSK Resources

Overview of WSK: http://blogs.msdn.com/wndp/archive/2006/02/24/538746.aspx

Network Programming with Winsock Kernel: http://blogs.msdn.com/wndp/archive/2006/05/04/Winhec-blog-wsk.aspx

WSK MSDN Documentation: http://msdn.microsoft.com/library/en-us/NetLH_d/hh/NetLH_d/wsk_2bb30f2b-db42-465a-9abd-61a7a7515273.xml.asp?frame=true

WDK RC1 Download: See http://connect.microsoft.com, Windows Driver Kit Program, Downloads Section

If you have any further comments or questions, please feel free to contact us at wskapi at microsoft.com.

We look forward to your feedback.

--Mike Flasko

Advances in Windows TCP/IP Networking

wndpteam — Tue, 25 Apr 2006 22:00:00 GMT

Significant advances have been made, both in hardware and in software, over the past few years that have enabled gigabit networking. From being unavailable just a few years back, it has become mainstream technology with GigE NICs available in an increasing number of clients and servers today. The ever increasing networking capacity and the need for high throughput over long distance transfers makes it imperative that we develop software which functions efficiently while taking advantage of all the available bandwidth. There are several bottlenecks that prevent high performance transfers end-to-end: host system on the sender and receiver, the network and of course the applications themselves. For Windows networking stack to deliver high throughput in gigabit networks and beyond, we have to alleviate some or all of these bottlenecks.

Let’s try and analyze some of these bottlenecks.

CPU bottleneck: Sending and receiving data at gigabit speeds requires a tremendous amount of processing: to compute checksums, form TCP/IP headers, validate received packets, copy data across buffers, and so on. Doing all this processing in software has significant processing demands with most of the CPU consumed in just sending and receiving data even with the fastest available off the shelf processors. Technologies like task offload have helped alleviate some of CPU load by offloading tasks like checksum computation. However, the CPU continues to be the bottleneck.

Scalability bottleneck: Due to physical limitations, scalability in processing power has been made possible by using multiple processors or multi-core processors. Most servers ship with multiple processors these days. However, the networking stack has always supported processing of received packets on a single processor only. This implies that for servers with significant network load, performance is limited to the processing power of one CPU and does not scale even when more processors are added.

TCP congestion control: the TCP protocol was designed in late 1980s when the bandwidths were fairly low. Principles of congestion control were developed to ensure that the end systems cooperate and do not cause congestion collapse on the network. TCP has scaled amazingly well to several hundred Mbps bandwidth as well as to millions of host using it across the Internet. However, as bandwidth continues to increase, inherent limitations in TCP are becoming evident. TCP was designed to be quite conservative in probing for spare bandwidth but quite aggressive to responding to any congestion indication. Congestion indications are derived based on packet losses. This imposes a theoretical limit on the throughput of TCP connection for a given loss rate. This is problematic for high bandwidth scenarios e.g. In data intensive grids and networks for high energy and nuclear physics, led by CERN, and projects like Terraservice for astronomy research, there is a need to move huge amounts of data across high bandwidth links. TCP faces challenges in scaling to such bandwidths.

TCP receive window limitations: Applications that use TCP are limited in throughput by the maximum default receive window. Traditionally, different implementations of TCP have used a maximum default receive window value of 64KB. (The receive window field in the TCP header is a 16-bit field.) This effectively limits the amount of data the sender can send, thereby directly impacting the maximum throughput achievable on the connection. Although it is possible to change this value in the registry on Windows, it is really not easy to guess what the correct value is for a given connection. The optimal receive window size varies by connection and it is usually dependent upon the bandwidth-delay product and the application’s consumption capacity, none of which can be pre-configured in the registry.

Application bottlenecks: For gigabit throughputs, applications must be optimized so as to minimize overhead in sending and receiving data. They must also ensure that there is always enough data to send to keep the pipes full, and enough buffers posted to receive incoming data in time. Today this requires a lot of manual tuning, something that’s clearly not an option as we go forward.

In the coming days and weeks we will discuss the advances made in the networking stack in Windows Vista and for the Longhorn Server to solve some of these problems.

Please stay tuned…

Murari Sridharan

TCP/IP Networking, Internet Protocols Team