Using "negative sleeps" to improve responsiveness in VB web apps

Using "negative sleeps" to improve responsiveness in VB web apps

  • Comments 19

[NOTE: please also read the followup to this article] 

 

.NET 4.0 will introduce many new threading and concurrency classes -- SpinLock, Parallel Linq, and ConcurrentDictionary to name but a few.

The new feature that excites me the most is the ability to pass a negative argument to Thread.Sleep. This article describes how you can use it to speed up a program by compensating for internet latency.

Let's start with a simple VB program which retrieves an RSS feed and produces from it a new feed with only those items from a particular date of the year:

Option Strict On

Imports System.Net

Imports System.IO

 

Module Module1

    Sub Main()

        Dim xml = Fetch(New Uri("http://blogs.msdn.com/vbteam/rss.xml"))

 

        Dim items = From item In xml.<channel>.<item>

                    Select title = item.<title>.Value,

                           url = item.<link>.Value,

                           time = DateTime.Parse(item.<pubDate>.Value)

                    Where time.Month = 4 AndAlso time.Day = 1

 

        Dim rss = <rss version="2.0">

                      <channel>

                          <title>Refeed</title>

                          <description>The last week of the feed</description>

                          <language>en</language>

                          <ttl>120</ttl>

                          <pubDate><%= Now.ToString("r") %></pubDate>

                          <%= From item In items Select

                              <item>

                                  <title><%= item.title %></title>

                                  <pubDate><%= item.time.ToString("r") %></pubDate>

                                  <link><%= item.url %></link>

                              </item> %>

                      </channel>

                  </rss>

 

        Console.WriteLine(rss)

    End Sub

 

    Function Fetch(ByVal url As Uri) As XElement

        Dim x = WebRequest.Create(url)

        Using r = x.GetResponse, rs As New StreamReader(r.GetResponseStream)

            Return XElement.Parse(rs.ReadToEnd)

        End Using

    End Function

 

End Module

How to optimize this? The important rule of thumb is that "everything done over the internet is much slower than anything done locally". In this case the bottleneck is the call to Fetch / WebRequest.GetResponse, so it's this that we have to optimize.

The key idea is to use the new "negative argument" feature of Threading.Thread.Sleep:

Public Shared Sub Sleep (millisecondsTimeout As Integer)

The argument says how long must elapse before the next statement executes. Up to .NET3.5, the millisecondsTimeout had to be positive, or "-1" to indicate infinity, and if you passed any other number then it threw an ArgumentOutOfRange exception.

 

Negative sleeps

Starting with .NET4.0, you can pass any negative argument to Thread.Sleep. As you'd expect, a negative argument indicates that the next statement should be executed earlier. Exception: for backwards compatibility, Thread.Sleep(-1) still has the special meaning "Infinite Sleep". If you really want to sleep for -1ms rather than infinity you should use Thread.SleepEx(-1), which doesn't have the special interpretation for -1.

And so we'll rewrite the function "Fetch" to use negative sleeps. The logic is subtle, so I give a few lines of code and then explain what they're doing:

Function FetchFaster(ByVal url As Uri) As XElement

    Static Delays As New LinkedList(Of Integer) ' Typical latencies (ms) 

As shown above, we're going to keep a record of typical past latencies for the web request in the "Delays" variable. That lets us calculate the average latency of web requests given our internet connection.

    ' Do a negative sleep before sending the request, so that the

    ' response will be ready at approximately the time it's needed

    Dim expectedLatency = If(Delays.Count = 0, 0, CInt(Delays.Average()))

    Thread.Sleep(-expectedLatency)

In the code above, Thread.Sleep(-expectedLatency) passes a negative argument to Thread.Sleep before executing the web request. Thus, if typical latency is 140ms, all we're doing is issuing our web request 140ms earlier than would otherwise have been the case.

    Dim startTime = Now

    Dim x = WebRequest.Create(url)

    Using r = x.GetResponse, rs As New StreamReader(r.GetResponseStream)

 

        ' Calculate the latency on this call, and add it into the "Delays" log

        Dim actualLatency = CInt((Now - startTime).TotalMilliseconds)

        If (Delays.Count >= 10) Then Delays.RemoveFirst()

        Delays.AddLast(actualLatency)

 

        ' It might be that our initial guess of latency was wrong. If so,

        ' correct it now so we don't try to return the result before it's ready

        If actualLatency < expectedLatency Then Thread.Sleep(expectedLatency - actualLatency)

The final line of code above requires explanation. Latency can vary from request to request, and it might turn out that we over-estimated how much latency there might be. Obviously in this case we have to compensate with an extra delay before returning the result -- otherwise, we might end up trying to return a result before it had been requested. This is called a race condition, it must be avoided, and in this case we avoid it with a positive call to Thread.Sleep.

        Return XElement.Parse(rs.ReadToEnd)

    End Using

 

End Function

 

Conclusion

Negative arguments to Thread.Sleep can make code become complicated very quickly -- especially if your code attempts retransmission or makes a series of web calls, and so has to protect against many more possible race conditions. I think though that the pattern in this article is simple enough to use and covers most typical scenarios.

If you have other ideas where negative arguments to Thread.Sleep might help, please post them in comments! If there are common ideas then I'll try to write up some more prototypical uses of the pattern. Until then, have a merry April 1st!

 

 

Update: optimising the retransmit delays

A reader sent in the following fascinating version of "Fetch":

Function Fetch3(ByVal url As Uri) As XElement

    Dim startTime = Now

 

    ' We will make three attempts to fetch the URL

    For attempts = 1 To 3

        Dim x = WebRequest.Create(url)

        x.Timeout = 200

        Try

 

            Using r = x.GetResponse, rs As New StreamReader(r.GetResponseStream)

                Return XElement.Parse(rs.ReadToEnd)

            End Using

 

        Catch ex As WebException

            ' If an attempt failed due to a timeout, then backtrack and try again.

            ' Question: how far back should we backtrack?

            ' Answer: so that our resend goes at the same time as the original send

            Dim elapsed = CInt((Now - startTime).TotalMilliseconds)

            Threading.Thread.Sleep(-elapsed)

            Continue For

 

        End Try

    Next

    Return Nothing

End Function

The reason this works has to do with TCP "rate control algorithm". Basically, your computer's has to make a best guess at the bandwidth of its internet connection. In this case our application knows better -- knows that it's only sending a short message (x.GetResponse) -- and so it can manage timeouts more aggresively.

 

Leave a Comment
  • Please add 5 and 2 and type the answer here:
  • Post
  • PingBack from http://blog.a-foton.ru/index.php/2009/04/01/using-negative-sleeps-to-improve-responsiveness-in-vb-web-apps/

  • PingBack from http://blog.a-foton.ru/index.php/2009/04/01/using-negative-sleeps-to-improve-responsiveness-in-vb-web-apps/

  • Wow, that's truly amazing! You guys should consider integrating this feature into IE8.1, that would really improve web page download times!

  • i tried it and its much faster - thankx for the Great TIP

    A)

    F)

  • I really like this feature!

    I would also suggest another useful extension: Thread.SleepExEx()

    This method works exactly as Sleep, but if you call Thread.SleepExEx(-1), -1 indicate an *negative* infinite sleep.

    In this way you can get responses also from non-existing servers, or with you network cable disconnected, or in all those

    situations where you have to deal with an infinite latency.

  • Sorry for my ignorance, but could you give some overview of how the process is launching a method earlier than it "should"? You cannot get the sleep interval till reaching its corresponding line, can you? so I'm a bit puzzled about how this may work.

  • This is awesome, exactly what we've needed in the Framework for years. In fact if you put a Thread.Sleep(220752000000) in the installer for .NET 4.0 we could have had it already.

    But in-line with this paradigm shift have you given any consideration to adding the ComeFrom statement to the VB language? I think that feature, coupled with this, would make for some very powerful applications indeed!

  • I wonder if this clever principle couldn't be extended to alarm clocks. This way I could perhaps awake at the exact same time it rings as my own personal latency is currently quite high in response to a wake up request...

  • We've had this in .NET 5.0 for months.  What's the big deal?  One thing you forgot to discuss in your article is anachronistic anomaly compensation (AAC).  I can forward you a link to the .NET 5.0 AAC toolkit if you like - whoops I already did.  You may have received the released version prior to the CTP - that's AAC dogfooding.  Happy Thanksgiving.  I have to get ready for prom now.  My Mapping the Atari book just arrived.  This is the best skating party ever.  What do you mean I can't eat my snack now?  Mommy I don't want to take a nap.  Druel...cry...eat shiny keys...

  • @Patrice Lol!

  • &quot;Life&#39;s a train that goes from February on day by day But it&#39;s making a stop on April first&quot;

  • Wow, that really speeds things up!  You can get even more of a boost by scaling up the expected latency value by some constant, and then forward-caching the results.  That way the request is actually completed before it's made so instead of hitting the web server, you can just load the response from the cache.  Nice post

  • Didn't realize this was posted today, 1st April, lol, I'm a fool.

  • This is the most entertaining thing I have read in a while. Much better than Google's joke this year.

  • Very great !

    At beginning of reading the article, I was quite amazed by the functionnality (I was thinking that the call was by done asynchronously before the true call was encountered in the stack by analysing the produced compiled code and by rearanging it, if it's possible to do so)

    When you came with you Try/Catch, I clearly understood that it was a 1 April Joke, however [:)]

Page 1 of 2 (19 items) 12