Using "negative sleeps" to improve responsiveness in VB web apps

Using "negative sleeps" to improve responsiveness in VB web apps

  • Comments 19

[NOTE: please also read the followup to this article] 

 

.NET 4.0 will introduce many new threading and concurrency classes -- SpinLock, Parallel Linq, and ConcurrentDictionary to name but a few.

The new feature that excites me the most is the ability to pass a negative argument to Thread.Sleep. This article describes how you can use it to speed up a program by compensating for internet latency.

Let's start with a simple VB program which retrieves an RSS feed and produces from it a new feed with only those items from a particular date of the year:

Option Strict On

Imports System.Net

Imports System.IO

 

Module Module1

    Sub Main()

        Dim xml = Fetch(New Uri("http://blogs.msdn.com/vbteam/rss.xml"))

 

        Dim items = From item In xml.<channel>.<item>

                    Select title = item.<title>.Value,

                           url = item.<link>.Value,

                           time = DateTime.Parse(item.<pubDate>.Value)

                    Where time.Month = 4 AndAlso time.Day = 1

 

        Dim rss = <rss version="2.0">

                      <channel>

                          <title>Refeed</title>

                          <description>The last week of the feed</description>

                          <language>en</language>

                          <ttl>120</ttl>

                          <pubDate><%= Now.ToString("r") %></pubDate>

                          <%= From item In items Select

                              <item>

                                  <title><%= item.title %></title>

                                  <pubDate><%= item.time.ToString("r") %></pubDate>

                                  <link><%= item.url %></link>

                              </item> %>

                      </channel>

                  </rss>

 

        Console.WriteLine(rss)

    End Sub

 

    Function Fetch(ByVal url As Uri) As XElement

        Dim x = WebRequest.Create(url)

        Using r = x.GetResponse, rs As New StreamReader(r.GetResponseStream)

            Return XElement.Parse(rs.ReadToEnd)

        End Using

    End Function

 

End Module

How to optimize this? The important rule of thumb is that "everything done over the internet is much slower than anything done locally". In this case the bottleneck is the call to Fetch / WebRequest.GetResponse, so it's this that we have to optimize.

The key idea is to use the new "negative argument" feature of Threading.Thread.Sleep:

Public Shared Sub Sleep (millisecondsTimeout As Integer)

The argument says how long must elapse before the next statement executes. Up to .NET3.5, the millisecondsTimeout had to be positive, or "-1" to indicate infinity, and if you passed any other number then it threw an ArgumentOutOfRange exception.

 

Negative sleeps

Starting with .NET4.0, you can pass any negative argument to Thread.Sleep. As you'd expect, a negative argument indicates that the next statement should be executed earlier. Exception: for backwards compatibility, Thread.Sleep(-1) still has the special meaning "Infinite Sleep". If you really want to sleep for -1ms rather than infinity you should use Thread.SleepEx(-1), which doesn't have the special interpretation for -1.

And so we'll rewrite the function "Fetch" to use negative sleeps. The logic is subtle, so I give a few lines of code and then explain what they're doing:

Function FetchFaster(ByVal url As Uri) As XElement

    Static Delays As New LinkedList(Of Integer) ' Typical latencies (ms) 

As shown above, we're going to keep a record of typical past latencies for the web request in the "Delays" variable. That lets us calculate the average latency of web requests given our internet connection.

    ' Do a negative sleep before sending the request, so that the

    ' response will be ready at approximately the time it's needed

    Dim expectedLatency = If(Delays.Count = 0, 0, CInt(Delays.Average()))

    Thread.Sleep(-expectedLatency)

In the code above, Thread.Sleep(-expectedLatency) passes a negative argument to Thread.Sleep before executing the web request. Thus, if typical latency is 140ms, all we're doing is issuing our web request 140ms earlier than would otherwise have been the case.

    Dim startTime = Now

    Dim x = WebRequest.Create(url)

    Using r = x.GetResponse, rs As New StreamReader(r.GetResponseStream)

 

        ' Calculate the latency on this call, and add it into the "Delays" log

        Dim actualLatency = CInt((Now - startTime).TotalMilliseconds)

        If (Delays.Count >= 10) Then Delays.RemoveFirst()

        Delays.AddLast(actualLatency)

 

        ' It might be that our initial guess of latency was wrong. If so,

        ' correct it now so we don't try to return the result before it's ready

        If actualLatency < expectedLatency Then Thread.Sleep(expectedLatency - actualLatency)

The final line of code above requires explanation. Latency can vary from request to request, and it might turn out that we over-estimated how much latency there might be. Obviously in this case we have to compensate with an extra delay before returning the result -- otherwise, we might end up trying to return a result before it had been requested. This is called a race condition, it must be avoided, and in this case we avoid it with a positive call to Thread.Sleep.

        Return XElement.Parse(rs.ReadToEnd)

    End Using

 

End Function

 

Conclusion

Negative arguments to Thread.Sleep can make code become complicated very quickly -- especially if your code attempts retransmission or makes a series of web calls, and so has to protect against many more possible race conditions. I think though that the pattern in this article is simple enough to use and covers most typical scenarios.

If you have other ideas where negative arguments to Thread.Sleep might help, please post them in comments! If there are common ideas then I'll try to write up some more prototypical uses of the pattern. Until then, have a merry April 1st!

 

 

Update: optimising the retransmit delays

A reader sent in the following fascinating version of "Fetch":

Function Fetch3(ByVal url As Uri) As XElement

    Dim startTime = Now

 

    ' We will make three attempts to fetch the URL

    For attempts = 1 To 3

        Dim x = WebRequest.Create(url)

        x.Timeout = 200

        Try

 

            Using r = x.GetResponse, rs As New StreamReader(r.GetResponseStream)

                Return XElement.Parse(rs.ReadToEnd)

            End Using

 

        Catch ex As WebException

            ' If an attempt failed due to a timeout, then backtrack and try again.

            ' Question: how far back should we backtrack?

            ' Answer: so that our resend goes at the same time as the original send

            Dim elapsed = CInt((Now - startTime).TotalMilliseconds)

            Threading.Thread.Sleep(-elapsed)

            Continue For

 

        End Try

    Next

    Return Nothing

End Function

The reason this works has to do with TCP "rate control algorithm". Basically, your computer's has to make a best guess at the bandwidth of its internet connection. In this case our application knows better -- knows that it's only sending a short message (x.GetResponse) -- and so it can manage timeouts more aggresively.

 

Leave a Comment
  • Please add 5 and 2 and type the answer here:
  • Post
  • Our recent pos t on "negative sleeps in VB" was an April Fool's joke. VB doesn't have negative sleeps,

  • That is a truly amazing feature introduced in .NET 4.0

  • Java, Ruby and PHP all have this feature for ages...

  • Ou comment un premier avril prend un tour encore plus drôle quand les anti-MS s'y mettent....

Page 2 of 2 (19 items) 12