Luca Bolognese's WebLog : VB

My PDC session is online - “Future directions for C# and Visual Basic”

lucabol — Mon, 23 Nov 2009 21:17:44 GMT

In case you are training to play the part of a first generation Italian immigrant in a Broadway show or if you want to understand why I never short msft stock: http://microsoftpdc.com/Sessions/FT11

Luca at PDC 2009 next week

lucabol — Fri, 13 Nov 2009 18:49:18 GMT

I’ll be in Los Angeles next week for PDC 2009. My session is called “Future Directions for C# and Visual Basic” and it comes on Tuesday immediately after the first keynote.

I’m planning on spending the first half of the session talking about the biggest trends influencing C# and VB. The second part will be about more future looking features we are playing with these days.

My deck has just two slides, as always. It’s all about watching me typing code on stage, hoping that I make crazy errors …

There are several other interesting language related sessions at PDC. Here is a list of them (please notice that the times might change). Also feel free to drop by the languages booth and chat. My team and I will be there most of the time.

Day	Time	Title	Presenter	Room
Tues 11/17	11:00 - 12:00	Future Directions for C# and Visual Basic	Luca Bolognese	Hall F
Tues 11/17	11:00 - 12:00	Accelerated Windows Application Development with Microsoft Visual C++ 2010	Boris Jabes	408B
Tues 11/17	12:30 - 13:15	Dynamic Binding in C# 4	Mads Torgersen	Hall F
Tues 11/17	12:30 - 13:15	Using Dynamic Languages to Build Scriptable Applications	Dino Viehland	403AB
Tues 11/17	13:30 - 14:30	C++ Forever: Interactive Applications in the Age of Manycore	Rick Molloy	515B
Tues 11/17	16:30 - 17:30	Manycore and the Microsoft .NET Framework 4: A Match Made in Microsoft Visual Studio 2010	Stephen Toub	502A
Tues 11/17	16:30 - 17:30	Code Contracts and Pex: Power Charge Your Assertions and Unit Tests	Mike Barnett, Nikolai Tillmann	408A
Wed 11/18	11:30 - 12:30	Microsoft Perspectives on the Future of Programming	Panel	Petree Hall C
Wed 11/18	12:00 - 13:00	F# for Architects – Hitting the Sweet Spot	Chris Smith	VS and .NET Pavillion
Wed 11/18	13:00 - 13:45	Code Like the Wind with Microsoft Visual Basic 2010	Lucian Wischik	Petree Hall D
Wed 11/18	13:00 - 13:45	Future of Garbage Collection	Patrick Dussud	Petree Hall C
Wed 11/18	13:00 - 13:45	Microsoft Project Code Name “M”: The Data and Modeling Language	Don Box, Jeff Pinkston	408A
Thu 11/19	08:30 - 09:30	PLINQ: LINQ, but Faster!	Ed Essey, Igor Ostrovsky	515A
Thu 11/19	10:00 - 11:00	A Lap around Microsoft Visual Studio 2010 for the Visual Basic Developer	Lisa Feigenbaum	VS and .NET Pavillion
Thu 11/19	10:00 - 11:00	Axum: A .NET Language for Safe and Scalable Concurrency	Niklas Gustafsson	515A
Thu 11/19	11:30 - 12:30	F# for Parallel and Asynchronous Programming	Luke Hoban	515A
Thu 11/19	12:45 - 13:30	Microsoft Visual C# IDE Tips and Tricks	DJ Park	Petree Hall D
Thu 11/19	12:45 - 13:30	Microsoft Visual Basic IDE Tips and Tricks	Dustin Campbell	Petree Hall C

A version of the AsyncCache found its way into the Parallel Programming samples …

lucabol — Thu, 21 May 2009 20:28:13 GMT

Go here to download them. It is in \ParallelExtensionsExtras\CoordinationDataStructures. It has a slightly different design in that it returns Tasks. I’m trying to get Stephen to blog about it so that you can compare them.

I talk about C# and VB Co-Evolution on Channel9 (and some F# …)

lucabol — Thu, 14 May 2009 20:26:18 GMT

The title says it all. If you are interested, go here.

An Async Html cache – part II – Testing the cache

lucabol — Fri, 08 May 2009 18:34:00 GMT

Other posts:

Part I – Writing the cache

Let’s try out our little cache. First I want to write a synchronous version of it as a baseline.

    Private Shared Sub TestSync(ByVal sites() As String, ByVal sitesToDownload As Integer, ByVal howLong As Integer)
        Dim syncCache As New Dictionary(Of String, String)
        Dim count = sites.Count()
        Dim url1 = "http://moneycentral.msn.com/investor/invsub/results/statemnt.aspx?Symbol="

        For i = 0 To sitesToDownload - 1
            Dim html As String = ""
            Dim url = url1 & sites(i Mod count)
            If Not syncCache.TryGetValue(url, html) Then
                html = LoadWebPage(url)
                syncCache(url) = html
            End If
            DoWork(html, howLong)
        Next
    End Sub

This is a loop that loads webpages in the cache if they are not already there. sites is a list of tickers used to compose the urls; sitesToDownload is the total number of sites to download, so that a single url can be loaded multiple times; howLong represents the work to be done on each loaded page.

In this version the cache is simply a Dictionary and there is no parallelism. The two bold lines is where the cache is managed.

DoWork is this.

    Public Shared Sub DoWork(ByVal html As String, ByVal howLong As Integer)
        Thread.Sleep(howLong)
    End Sub

Let’s take a look at the asynchronous version.

    Private Shared Sub TestAsync(ByVal sites() As String, ByVal sitesToDownload As Integer, ByVal howLong As Integer)
        Dim htmlCache As New HtmlCache
        Dim count = sites.Count()
        Dim url = "http://moneycentral.msn.com/investor/invsub/results/statemnt.aspx?Symbol="
        Using ce = New CountdownEvent(sitesToDownload)
            For i = 1 To sitesToDownload
                htmlCache.GetHtmlAsync(
                    url & sites(i Mod count),
                    Sub(s)
                        DoWork(s, howLong)
                        ce.Signal()
                    End Sub)
            Next
            ce.Wait()
        End Using

There are several points worth making on this:

The lambda used as second parameter for GetHtmlAsync is invoked on a different thread whenever the html has been retrieved (which could be immediately if the cache has downloaded the url before)
CountDownEvent allows a thread to wait for a certain number of signals to be sent. The waiting happens on the main thread in the ce.Wait() instruction. The triggering of the event happens in the lambda described in the point above (the ce.Signal() instruction)

This is the driver for the overall testing.

    Private Shared Sub TestPerf(ByVal s As String, ByVal a As Action, ByVal iterations As Integer)
        Dim clock As New Stopwatch

        clock.Start()
        For i = 1 To iterations
            a()
        Next
        clock.Stop()
        Dim ts = clock.Elapsed
        Dim elapsedTime = String.Format(s & ": {0:00}:{1:00}:{2:00}.{3:00}", ts.Hours, ts.Minutes, ts.Seconds, ts.Milliseconds / 10)
        Console.WriteLine(elapsedTime, "RunTime")
    End Sub

There is not much to say about it. Start the clock, perform a bunch of iterations of the passed lambda, stop the clock, print out performance.

And finally the main method. Note that all the adjustable parameters are factored out before the calls to TestPerf.

    Public Shared Sub Main()
        Dim tickers = New String() {"mmm", "aos", "shlm", "cas", "abt", "anf", "abm", "akr", "acet", "afl", "agl", "adc", "apd",
                                           "ayr", "alsk", "ain", "axb", "are", "ale", "ab", "all"}

        Dim sitesToDownload = 50
        Dim workToDoOnEachUrlInMilliSec = 20
        Dim perfIterations = 5

        TestPerf("Async", Sub() TestAsync(tickers, sitesToDownload, workToDoOnEachUrlInMilliSec), perfIterations)
        TestPerf("Sync", Sub() TestSync(tickers, sitesToDownload, workToDoOnEachUrlInMilliSec), perfIterations)
    End Sub

Feel free to change (tickers, sitesToDownload, workToDoOnEachUrlInMilliSec, perfIterations). Depending on the ratios between these parameters and the number of cores on your machine, you’re going to see different results. Which highlights the fact that parallelizing your algorithms can yield performance gains or not depending on both software and hardware considerations. I get ~3X improvement on my box. I attached the full source file for your amusement.

An Async Html cache – Part I - Writing the cache

lucabol — Tue, 28 Apr 2009 02:57:00 GMT

Other posts:

Part II - Testing the cache

In the process of converting a financial VBA Excel Addin to .NET (more on that in later posts), I found myself in dire need of a HTML cache that can be called from multiple threads without blocking them. Visualize it as a glorified dictionary where each entry is (url, cachedHtml). The only difference is that when you get the page, you pass a callback to be invoked when the html has been loaded (which could be immediately if the html had already been retrieved by someone else).

In essence, I want this:

    Public Sub GetHtmlAsync(ByVal url As String, ByVal callback As Action(Of String))

I’m not a big expert in the .Net Parallel Extensions, but I’ve got help. Stephen Toub helped so much with this that he could have blogged about it himself. And, by the way, this code runs on Visual Studio 2010, which we haven’t shipped yet. I believe with some modifications, it can be run in 2008 + .Net Parallel Extensions CTP, but you’ll have to change a bunch of names.

In any case, here it comes. First, let’s add some imports.

Imports System.Collections.Concurrent
Imports System.Threading.Tasks
Imports System.Threading
Imports System.Net

Then, let’s define an asynchronous cache.

Public Class AsyncCache(Of TKey, TValue)

This thing needs to store the (url, html) pairs somewhere and, luckily enough, there is an handy ConcurrentDictionary that I can use. Also the cache needs to know how to load a TValue given a TKey. In ‘programmingese’, that means.

    Private _loader As Func(Of TKey, TValue)
    Private _map As New ConcurrentDictionary(Of TKey, Task(Of TValue))

I’ll need a way to create it.

    Public Sub New(ByVal l As Func(Of TKey, TValue))
        _loader = l
    End Sub

Notice in the above code the use of the Task class for my dictionary instead of TValue. Task is a very good abstraction for “do some work asynchronously and call me when you are done”. It’s easy to initialize and it’s easy to attach callbacks to it. Indeed, this is what we’ll do next:

    Public Sub GetValueAsync(ByVal key As TKey, ByVal callback As Action(Of TValue))

        Dim task As Task(Of TValue) = Nothing
        If Not _map.TryGetValue(key, task) Then
            task = New Task(Of TValue)(Function() _loader(key), TaskCreationOptions.DetachedFromParent)
            If _map.TryAdd(key, task) Then
                task.Start()
            Else
                task.Cancel()
                _map.TryGetValue(key, task)
            End If
        End If

        task.ContinueWith(Sub(t) callback(t.Result))
    End Sub

Wow. Ok, let me explain. This method is divided in two parts. The first part is just a thread safe way to say “give me the task corresponding to this key or, if the task hasn’t been inserted in the cache yet, create it and insert it”. The second part just says “add callback to the list of functions to be called when the task has finished running”.

The first part needs some more explanation. What is TaskCreationOptions.DetachedFromParent? It essentially says that the created task is not going to prevent the parent task from terminating. In essence, the task that created the child task won’t wait for its conclusion. The rest is better explained in comments.

        If Not _map.TryGetValue(key, task) Then ' Is the task in the cache? (Loc. X)
            task = New Task(Of TValue)(Function() _loader(key), TaskCreationOptions.DetachedFromParent) ' No, create it
            If _map.TryAdd(key, task) Then ' Try to add it
                task.Start() ' I succeeded. I’m the one who added this task. I can safely start it.
            Else
                task.Cancel() ' I failed, someone inserted the task after I checked in (Loc. X). Cancel it.
                _map.TryGetValue(key, task) ' And get the one that someone inserted
            End If
        End If

Got it? Well, I admit I trust Stephen that this is what I should do …

I can then create my little HTML Cache by using the above class as in:

Public Class HtmlCache

    Public Sub GetHtmlAsync(ByVal url As String, ByVal callback As Action(Of String))
        _asyncCache.GetValueAsync(url, callback)
    End Sub

    Private Function LoadWebPage(ByVal url As String) As String
        Using client As New WebClient()
            'Test.PrintThread("Downloading on thread {0} ...")
            Return client.DownloadString(url)
        End Using
    End Function

    Private _asyncCache As New AsyncCache(Of String, String)(AddressOf LoadWebPage)

End Class

I have no idea why coloring got disabled when I copy/paste. It doesn’t matter, this is trivial. I just create an AsyncCache and initialize it with a method that knows how to load a web page. I then simply implement GetHtmlAsync by delegating to the underlying GetValueAsync on AsyncCache.

It is somehow bizarre to call Webclient.DownloadString, when the design could be revised to take advantage of its asynchronous version. Maybe I’ll do it in another post. Next time, I’ll write code to use this thing.