Windows Core Networking : offload

Advances in Windows Vista TCP/IP

wndpteam — Fri, 05 May 2006 21:00:00 GMT

The Windows Vista TCP/IP stack has made tremendous improvements in its efficiency, taking full advantage of hardware advances (e.g. gigabit networking). As explained by Murari in a previous posting (Advances in Windows TCP/IP Networking), there are a number of bottlenecks that affect TCP throughput. Here, I will give some examples of how we’ve addressed these bottlenecks in the Windows Vista TCP/IP stack.

TCP auto-tuning: At any given time, the amount that TCP can send is governed by three factors: the congestion window, the receive window and the number of bytes available to send. Without using TCP window scaling (which is disabled by default in previous versions of Windows), the maximum receive window a receiver can advertise is 64K bytes. Since the congestion window is usually greater than 64K bytes in high-bandwidth/high-latency networks, the receive window is often the limiting factor if the application is submitting enough data.

In previous versions of Windows, users can work around this problem by setting the TcpWindowSize registry key value. However, TcpWindowSize is a global setting applied to all connections, and it’s often hard for users to know the appropriate window size to set.

To address this issue in Windows Vista, we implemented TCP auto-tuning. It enables TCP window scaling by default and automatically tunes the TCP receive window size based on the bandwidth delay product (BDP) and the rate at which the application reads data from the connection. With TCP auto-tuning, we have seen 1000% (10x) throughput improvements in internal testing over underutilized wide-area network links.

Receive-Side Scaling: Networking stacks face a number of challenges in scaling their receive processing across processors on multi-processor systems. For instance, on previous versions of Windows all packets indicated in a single interrupt service routine (ISR) are typically processed in a single deferred procedure call (DPC) queued to a specific processor to avoid packet reordering. Until the outstanding DPC completes, no more receive indication interrupts can be triggered. As a result, only one processor can be used at any given time for processing received packets for a single network adapter.

Receive-side scaling (RSS) is our solution for this issue in the new networking stack: it enables parallelized processing of received packets on multiple processors, while avoiding packet reordering. It achieves parallelism by allowing ISRs to queue DPCs on multiple processors, enabling packet processing on multiple processors at the same time. It avoids packet reordering by separating packets into flows, and using a single processor for processing all the packets for a given flow. Packets are separated into flows by computing a hash value based on specific fields in each packet, and the resulting hash values are used to select a processor for processing the flow. Using TCP as an example, this approach ensures that all packets belonging to a given TCP connection will be queued to the same processor, in the same order that they were received by the network adapter.

TCP offload: Previous Windows releases already support network task offload for stateless per-packet operations (e.g. LSO, checksum offload etc). In Windows Vista, in addition to the offloads supported on previous Windows releases, we’ve also introduced support for TCP chimney offload. TCP chimney offload enables Windows to offload all TCP processing for a connection to a network adapter. Offloads are initiated on a per-connection basis, based on heuristics. Compared to task offload, TCP chimney offload further reduces networking-related CPU overhead, enabling better overall system performance by freeing up the CPU for other tasks.

We have also responded to customer feedback by making the Windows Vista TCP/IP stack much smarter and more adaptive in a number of scenarios. One such improvement we’ve made is to enable TCP black-hole detection by default in Windows Vista.

Historically, problems due to the presence of black-hole routers have been among the highest product support call generators for the previous Windows networking stacks. To understand why, it’s important to know that TCP/IP relies on ICMP packet-too-big error messages to discover the maximum transmission unit (MTU) for any given connection’s path, so that it can reduce the size of the packets that it sends if they’re too large. If a router along the path does not send back ICMP error messages, or if a firewall drops ICMP error messages, TCP will never find out that its packets are too big. As a result, it will retransmit the packets repeatedly with the same size, up to its maximum number of retransmissions and, when it gets no responses, it will terminate the connection.

Black hole router detection is a mechanism used in this scenario to automatically reduce the size of the packets sent for a connection, based on the current status of the connection, in the absence of feedback from ICMP packet too big error messages. This mechanism was disabled by default in previous versions of Windows, because previous approaches would often yield too many false positives, lowering the packet size unnecessarily and reducing performance. In Windows Vista, our improvements have reduced the likelihood of false positives and, consequently, minimized the adverse performance impact, enabling us to turn on black hole detection by default in the upcoming Beta 2 release.

There are many, many more innovations that we’ve made in the network stack, far more than I can write about in this one posting. Stay tuned for more…

Xinyan Zan

Software Development Engineer, TCP/IP Networking

What is NetDMA?

wndpteam — Tue, 02 May 2006 21:00:00 GMT

The NetDMA term was coined by the networking team to imply a DMA (Direct Memory Access) engine that is used for moving networking data in memory. During WinHEC I will present the NetDMA architecture but for now I will give you the problem that NetDMA is trying to address.

When the networking card receives data from the wire, it performs a DMA transfer to copy it into system memory. It then informs the networking stack that a certain number of received packets are ready for processing, most commonly by raising an interrupt. When the networking stack processes the packets, it copies the data into buffers posted by the application waiting for it. This second copy operation is performed by the CPU, which means that receive processing can be a CPU-intensive task. NetDMA tries to address the following question: how can we reduce CPU utilization when doing the second copy?

Besides presenting the NetDMA architecture, I will present (in another talk) the networking test tools that are integrated in the Windows Driver Kit. This will be a great opportunity to see how network devices will be tested for the “Designed for Windows Vista” logo program, to ask questions, and to provide us with feedback.

There is something for everyone, so come and join us at WinHEC 2006 in Seattle.

Rade Trimceski

Program Manager

Advances in Windows TCP/IP Networking

wndpteam — Tue, 25 Apr 2006 22:00:00 GMT

Significant advances have been made, both in hardware and in software, over the past few years that have enabled gigabit networking. From being unavailable just a few years back, it has become mainstream technology with GigE NICs available in an increasing number of clients and servers today. The ever increasing networking capacity and the need for high throughput over long distance transfers makes it imperative that we develop software which functions efficiently while taking advantage of all the available bandwidth. There are several bottlenecks that prevent high performance transfers end-to-end: host system on the sender and receiver, the network and of course the applications themselves. For Windows networking stack to deliver high throughput in gigabit networks and beyond, we have to alleviate some or all of these bottlenecks.

Let’s try and analyze some of these bottlenecks.

CPU bottleneck: Sending and receiving data at gigabit speeds requires a tremendous amount of processing: to compute checksums, form TCP/IP headers, validate received packets, copy data across buffers, and so on. Doing all this processing in software has significant processing demands with most of the CPU consumed in just sending and receiving data even with the fastest available off the shelf processors. Technologies like task offload have helped alleviate some of CPU load by offloading tasks like checksum computation. However, the CPU continues to be the bottleneck.

Scalability bottleneck: Due to physical limitations, scalability in processing power has been made possible by using multiple processors or multi-core processors. Most servers ship with multiple processors these days. However, the networking stack has always supported processing of received packets on a single processor only. This implies that for servers with significant network load, performance is limited to the processing power of one CPU and does not scale even when more processors are added.

TCP congestion control: the TCP protocol was designed in late 1980s when the bandwidths were fairly low. Principles of congestion control were developed to ensure that the end systems cooperate and do not cause congestion collapse on the network. TCP has scaled amazingly well to several hundred Mbps bandwidth as well as to millions of host using it across the Internet. However, as bandwidth continues to increase, inherent limitations in TCP are becoming evident. TCP was designed to be quite conservative in probing for spare bandwidth but quite aggressive to responding to any congestion indication. Congestion indications are derived based on packet losses. This imposes a theoretical limit on the throughput of TCP connection for a given loss rate. This is problematic for high bandwidth scenarios e.g. In data intensive grids and networks for high energy and nuclear physics, led by CERN, and projects like Terraservice for astronomy research, there is a need to move huge amounts of data across high bandwidth links. TCP faces challenges in scaling to such bandwidths.

TCP receive window limitations: Applications that use TCP are limited in throughput by the maximum default receive window. Traditionally, different implementations of TCP have used a maximum default receive window value of 64KB. (The receive window field in the TCP header is a 16-bit field.) This effectively limits the amount of data the sender can send, thereby directly impacting the maximum throughput achievable on the connection. Although it is possible to change this value in the registry on Windows, it is really not easy to guess what the correct value is for a given connection. The optimal receive window size varies by connection and it is usually dependent upon the bandwidth-delay product and the application’s consumption capacity, none of which can be pre-configured in the registry.

Application bottlenecks: For gigabit throughputs, applications must be optimized so as to minimize overhead in sending and receiving data. They must also ensure that there is always enough data to send to keep the pipes full, and enough buffers posted to receive incoming data in time. Today this requires a lot of manual tuning, something that’s clearly not an option as we go forward.

In the coming days and weeks we will discuss the advances made in the networking stack in Windows Vista and for the Longhorn Server to solve some of these problems.

Please stay tuned…

Murari Sridharan

TCP/IP Networking, Internet Protocols Team

An Interview with Alireza Dabagh, NDIS development lead

wndpteam — Tue, 18 Apr 2006 04:37:00 GMT

Tell us about what you do in Windows.

I’m NDIS development lead. I lead a team which works on NDIS and related components, and I also contribute to the architecture, coding and design of our components.

How long have you been participating in WinHEC?

Quite some time! Maybe 4 or 5 years.

What are the challenges and opportunities you see ahead for core networking and the connectivity technologies that you work on?

One of the biggest challenges that we have is supporting multigigabit networking. As the raw bandwidth increases, the role of the entire stack becomes an obstacle to taking advantage of that bandwidth. It wasn’t an issue at 10/100Mbps, but it used to be an issue at 1Gbps. Today it’s not an issue anymore for 1Gbps but its an issue for 10Gbps. The challenge is to make sure we can take advantage of all the capacity that the network has to offer.

The other challenge is manageability and diagnosability. In today’s environment, many of the people who rely on networking are new to networking and they need a way to understand what’s going on, why connectivity is lost,or why performance isn’t what they expect, in a way that they can act on rather than through some cryptic error message. They need clearer explanations of these problems. For example, if you lose Internet connectivity at home, it’s hard to tell if it’s because your cable is disconnected, or if your cable or DSL modem is hung, or because your router needs to be reset, or because your provider isn’t giving you an address. It’s really hard for a regular user to diagnose these problems.

The third thing I’d mention is scalability. By that I mean being able to use multiple links, with features link load-balancing and fail-over. Today we have multiple solutions from various vendors, which behave in different ways and are often hard to use and managed. We’ve heard a lot of requests for supporting these features natively as part of the platform to provide a consistent experience.

How does virtualization affect the area that you work on?

I should have mentioned that as one of the challenges! When you only have one real interface in the system, but you want each guest OS to see its own virtual interface, you run into various issues. For example, how do you get the real NIC to receive packets for all the virtual interfaces? Today each NIC has one unicast MAC address. If you want to get frames for all those virtual MAC addresses, today you have to put the hardware into promiscuous mode which has its own implications. Dispatching those incoming packets to multiple virtual machines has its own performance and security implications. Having resources like DMA engines, I/OAT engines and being able to use it in multiple virtual machines is also challenging particularly with security. It’s challenging to take advantage of offloads like TCP chimney offload and task offload in virtual machines without any conflicts. Remember that you can only have one driver running the hardware, and that driver is ignorant of there being multiple virtual interfaces running over it. So you need a layer that manages virtualization requirements on top of the driver.

Any messages for the WinHEC audience?

Be sure to follow the progress of our NDIS APIs closely, particularly what we’re doing with IPsec offload, TCP chimney offload and task offload. We don’t want anyone to be left behind, so the WinHEC attendees should make sure they can take advantage of that. It’s no longer enough to just provide plain vanilla hardware, because competitors can provide these cool new capabilities with minimal expense. Networking hardware that doesn’t offer these capabilities will be at a great disadvantage. Getting this functionality isn’t the difference between $10 and $100 hardware for OEMs; it’s the difference between $10 and $12 hardware. So my advice is: don’t be left behind!

-Abolade Gbadegesin