Making things go fast on a network, part 2

Larry Osterman s WebLog Making things go fast on a network part 2 | storage bench

Fri, 19 Jun 2009 11:00:59 GMT

PingBack from http://thestoragebench.info/story.php?id=62

Larry Osterman s WebLog Making things go fast on a network part 2 | Weak Bladder

Larry Osterman s WebLog Making things go fast on a network part 2 | Weak Bladder — Tue, 09 Jun 2009 21:45:17 GMT

PingBack from http://weakbladder.info/story.php?id=5204

I wish Larry hadn't written that...

Tales from the Crypto — Tue, 27 Jun 2006 07:13:47 GMT

Oh, Larry, Larry, Larry...
Articles 1 and 2 were great - really necessary reading to a lot of would-be...

re: Making things go fast on a network, part 2

Tom H — Tue, 13 Jun 2006 18:19:01 GMT

Larry, the axioms you've stated are all true, but then I guess I didn't understand the rule:

"Always make sure that you have a receive request down BEFORE someone tries to send you data."

From what Dave's said above, and your agreement with him, I interpret this as being that you should have let the OS know what buffer to put the data in before the network delivers the data. That makes perfect sense in a high-performance application, but I didn't have a clue what "hav[ing] a receive request down" meant, and even after your explanation I wonder if that is NetBEUI terminology?

I remember lots of research at UCSD in the early 1990s about reducing buffer copies and boundary crossings when you receive network packets. Good to know that some of it seems to have seen light. This kind of optimization sounds much easier to do with transport protocols that expose the frame explicitly, like UDP and (according to Google?) NetBEUI, but I suppose you could do it in TCP too, even though the application can't see frame boundaries.

But in TCP, the implementation is really expected to have buffer space in the OS, and the flow control protocol lets the client know how much buffer space there is, so you should never see the "No Memory" kind of response. The drawback is that the protocol had some trouble with high bandwidth-delay connections (standard flow control only allows 64kB of buffer space), but I think they've extended around that by now.

(You refer to corrolaries and rules, but I see only the one rule bolded or otherwise picked out between this and the previous post - am I missing something?)

re: Making things go fast on a network, part 2

Larry Osterman [MSFT] — Tue, 13 Jun 2006 06:59:48 GMT

Dave's absolutely right - you don't have to have the receive down, unless you care about perf. If you care about perf, then the additional data copy can kill your perf. After all, what would you rather your CPU be doing - copying data from an in-kernel buffer or doing work in your application? If you want the CPU to be doing your work, get the receive down before the sender sends it.

Tom, you're right, my knowledge IS NetBEUI-centric, but I think you'll discover that the principles involved apply on TCP. The details are absolutely different (TCP's sliding windows are based on offsets, not packet sequence numbers, etc) but at a high enough level it doesn't matter.

In particular, the axioms, corrolaries and rules I've mentioned apply pretty-much universally.

re: Making things go fast on a network, part 2

Dave Beaver — Thu, 08 Jun 2006 00:51:06 GMT

Reply to nick about recv:

Yes, it's not necessary to get the receive down before the data arrives at the node. But the focus of these articles is speed, and if you're interested in speed, you want to have that receive in place. Why? Because the OS will go out of it's way to land the data into the buffer you provide as it's received. It does this for the obvious efficiency reasons; if it doesn't, a data copy is required to get the data from the transport buffer to the application buffer. And that copy is relatively expensive.
Note that this apparently even more true in Vista/Longhorn, from my (admittedly brief) looks at it so far.

re: Making things go fast on a network, part 2

Andreas Johansson — Thu, 25 May 2006 04:14:06 GMT

I don't think there are many developers today that actually get this. They are using abstractions that hide all this. Really nice to get it mentioned.

re: Making things go fast on a network, part 2

Tom H — Thu, 18 May 2006 16:59:16 GMT

Here again, Larry, I think your presentation may be NetBEUI-centric (or some other protocol that I'm not familiar with). TCP handshaking establishes a receive window size; the sender can't send until it knows there's a nonzero receive window. Stevens has a basic discussion of this in Chapter 20 of Volume 1 of TCP/IP Illustrated and spends some time talking about bandwidth-delay product and determining the optimum window size.

re: Making things go fast on a network, part 2

mjb — Wed, 17 May 2006 20:02:06 GMT

maybe I'm being obtuse, but shouldnt the last Send Request of A.1 (of 3) have gotten an ACK?

re: Making things go fast on a network, part 2

nick — Wed, 17 May 2006 10:22:31 GMT

"Always make sure that you have a receive request down BEFORE someone tries to send you data."

Thats why you have per-connection receive buffers in OS-es, unless you turn them off (set the buffer size to 0 for a connection). For instance, on Windows, Winsock sets up an initial buffer for both receives and sends....I believe the default buffer sizes are ~ 32 KB, I might be mistaken (easy to check with getsockopt).

So there's no desperate need to race to use a recv on a TCP client application for example. Just make sure you always have a recv posted *before* the recv buffer fills up.

As you said, servers and clients are symmetrical in this respect (i.e. once the initial handshake is done) so the above observations apply equally to them.

From an MSDN article on writing Scalable apps
[ http://msdn.microsoft.com/msdnmag/issues/1000/Winsock/ ]
"As just mentioned, AFD.SYS handles buffer management for applications that use Winsock to talk to the transport protocol drivers. This means that when an application calls the send or WSASend function to send data, the data gets copied by AFD.SYS to its internal buffers (up to the SO_SNDBUF setting) and the send or WSASend function returns immediately. The data is then sent by AFD.SYS behind the application's back, so to speak. Of course, if the application wants to issue a send for a buffer larger than the SO_SNDBUF setting, the WSASend call blocks until all the data is sent.
Similarly, on receiving data from the remote client, AFD.SYS will copy the data to its own buffers as long as there is no outstanding data to receive from the application, and as long as the SO_RCVBUF setting is not exceeded. When the application calls recv or WSARecv, the data is copied from AFD.SYS's buffers to the application-provided buffer."

and

"Turning off receive buffering in AFD.SYS by setting SO_RCVBUF to 0 offers no real performance gains. Setting the receive buffer to 0 forces received data to be buffered at a lower layer than Winsock. Again, this leads to buffer copying when you actually post a receive, which defeats your purpose in turning off AFD's buffering."
NOTE: You also can get major perf loss if you do this and then fail to post recvs in a timely fashion.