Lucian's VBlog

MVVM with Async - sample code

Lucian Wischik, MSFT — Tue, 26 Feb 2013 20:53:00 GMT

How do you combine MVVM with async? -- Well, really, there's not much to it. Just the same as always. Here's an example Silverlight MVVM project. It first shipped with the Async CTP, and I've updated it to use VS2012 and Microsoft.Bcl.Async.

Download AsyncMVVM-Silverlight-VB.zip [2mb, requires VS2012]
Download AsyncMVVM-Silverlight-CS.zip [2mb, requires VS2012]

One thing that makes it easier with async/await is that these keywords don't introduce new threads. This makes things a lot easier. It means that all your work can be done on the UI thread: even when you update databound properties, or add elements to a databound observable collection, it's all on the UI thread and so works fine.

I just want to call out ICommand.Execute. We all know that async voids should only be used for top-level event-handlers, or event-like things. Well, ICommand.Execute is an top-level event-like thing, so it's okay for it to be async void. Just remember that all exceptions that come out of an async void will be posted straight to the UI thread.

Public Class FetchCommand

Implements ICommand

Friend vm As MainPageViewModel

Friend Sub Invalidate()

RaiseEvent CanExecuteChanged(Me, EventArgs.Empty)

End Sub

Public Function CanExecute(p As Object) As Boolean Implements ICommand.CanExecute

Return Not vm.Fetching

End Function

Public Event CanExecuteChanged(s As Object, e As EventArgs) Implements ICommand.CanExecuteChanged

Public Sub Execute(parameter As Object) Implements ICommand.Execute

vm.DoFetch() ' DoFetch is an Async Sub (i.e. void-returning async)

End Sub

End Class

Talk: The New Async Design Patterns

Lucian Wischik, MSFT — Mon, 18 Feb 2013 06:02:00 GMT

Talk: The New Async Design Patterns

Async involves some new concepts. They're not difficult; just unfamiliar. Over the past year I've been watching how people use it. This talk distils out the three top async patterns and anti-patterns.

Async void is for top-level event-handlers only, and event-like things. Don't use it elsewhere in your code.
You can wrap events up in Task-returning APIs and await them. This can dramatically simplify code.
It's crucial to distinguish CPU-bound work (should be done on threadpool) from IO-bound work (which needn't).

If you understand the basic flow of control in an async method, then those three points all fall naturally into place.

I gave this talk at Techready16 in Seattle, USA on 2013.02.05

Watch videos online: Intro [5mins], Tip1: async void [20mins], Tip2: async and threads [16mins], Tip3: async-over-events [15mins]
Download slides.pptx [6.3mb]
Download script.docx [35k]
Download VB source_code.zip [460k, requires VS2012 and Win8]

PS. The first anti-pattern is "async void". People of course ask: "If it's so wrong, then why did you allow it?" That's a good question. I'll write a blog post solely on this question next week.

Talk: Async Part 2 - for architects, under the hood

Lucian Wischik, MSFT — Sun, 17 Feb 2013 19:01:00 GMT

Talk: Async Part 2 - for architects, under the hood

This talk tackles some more advanced async topics.

Async idioms - the "TAP" Task Asynchronous Pattern, when and when not to expose async APIs, the unexpected trouble with Task.Yield
Integrating async into existing code - call sync from async and vice versa, user-defined awaitables
Async codegen and perf - some small tweaks you can make, if performance analysis shows that an async method is in your critical path

For an architect or a library-author, I think you need to know a little more about the conventions and what's going on under the hood

I gave this talk at NDC2012 in Oslo, Norway on 2012.06.06

Watch video online
Download slides.pptx [190k]
Download script.docx [21k]
Download source_code.zip [10k, requires VS2012 and Win8]

Similar material elsewhere

Stephen Toub blogged about "Should I expose asynchronous wrappers for synchronous methods"
He also blogged about the converse "Should I expose SYNC wrappers for ASYNC methods"
Together, these topics get to the heart of the interplay between sync and async in your code

How to write a custom awaiter

Lucian Wischik, MSFT — Wed, 12 Dec 2012 01:45:00 GMT

[This post is part of a series How to await a storyboard, and other things]

The normal behavior of the "await" operator on a task is to suspend execution of the method; then, when the task operand has finished, to resume execution on the same SynchronizationContext and with the same ExecutionContext.

You can replace the part in italics with your own logic by awaiting an operand of your own type (not a task) and writing a custom awaiter for it. It's not a common need. Stephen Toub has already written an excellent article that covers this topic ("Await anything"). He explains that a custom awaiter is for when "you need full control over how (rather than when) the method resumes". I wanted to build on his article from a compiler-writer's perspective and with some more example code. Here are some examples I've seen of custom awaiters.

Await task1.ConfigureAwait(false). Here execution doesn't make the effort to go back to the prior synchronization context, and often resumes on whatever thread the task happened to signal its completion upon. This is an efficiency boost, since it's faster than having to post back onto the prior SynchronizationContext.
Await Task.Yield(). Here there isn't an operand that finishes, as such. Instead this awaiter schedules execution to resume shortly on the same SynchronizationContext for some vague meaning of "shortly"... (Actually, the meaning of "shortly" is so vague that the only predictable use of this API is to force a suspension of an async method).

The above two scenarios were thought common enough that they're provided in the .NET45 framework. The rest aren't...

Await HibernateAsync("a.hib"). I wrote an awaiter like this which didn't have any useful resumption. Instead it used the fact that a custom awaiter can (undocumentedly) use reflection to get at the current state of the async method, and thereby serialize it to disk.
Await task1.OnCoroutine(crm1). I saw some code which bypassed the SynchronizationContext mechanism completely. Instead it used its own co-routine managers: a set of lists-of-actions, which they executed in round-robin fashion.
Await SwitchToThreadpool() / SwitchToDispatcher(dispatcher). I saw some code which used these two primitives to switch between threadpool and UI thread, as an alternative to the more common "Await Task.Run(...)" and "Await Dispatcher.RunAsync(...)" which both involve lambdas. It came from a wish to expunge all lambdas from a user's code.
Await SwitchPriority(High). Similar to the above example, it might be useful to start executing at a higher priority.
Dim x = Await thread1.Enter() : ... : Await x.Leave(). I thought of this as a synchronization primitive. The idea is that most of my code runs happily on multiple threads, but I block off certain critical sections which must be executed on a particular thread maybe because they use thread-local-storage.
Await task1.Log(). Here I was trying to track down a bug where Await wasn't resuming. I wrote a custom awaiter which wrote to a debug log at every stage along the way of suspending and resuming an async method.
Await task1.WithCulture(). Normally, CurrentThread.CurrentCulture doesn't flow across await points. You can write a custom awaiter which does make it flow.
Await task1.TemporarilyConfigureAwait(false). Maybe you might want the efficiency-boost of task.ConfigureAwait(false), but you still want it to restore SynchronizationContext.Current so that subsequent awaits will return to it.

Here's some background reading to explain the concepts...

ExecutionContext vs SynchronizationContext – blog article by Stephen Toub
Await anything – blog article by Stephen Toub, including one custom-awaiter that flows culture, one that switches to the threadpool, and one that switches to the UI
Await pattern – blog article by Stephen Toub that describes the signature of a custom awaiter.
INotifyCompletion and ICriticalNotifyCompletion – MSDN documentation about these two interfaces that a custom awaiter will have to implement.

What the compiler requires from a custom awaiter

When you use the Await operator, the compiler expands it out into something a bit like this:

' ORIGINAL CODE

Dim r = Await t

' APPROXIMATELY WHAT THE COMPILER GENERATES

Dim tmp = t.GetAwaiter()

If Not tmp.IsCompleted Then

Dim ec = ExecutionContext.Capture()

CType(tmp, INotifyCompletion).OnCompleted(Sub() ExecutionContext.Run(ec, K1))

Return

End If

K1:

Dim r = tmp.GetResult()

' Some things not shown in the above simplification:

' The two lines in italics are executed from within a routine inside mscorlib

' The argument "K1" represents a delegate that, when executed, resumes execution below.

' If "tmp" is a struct, any mutations after IsCompleted before GetResult may be lost.

' If tmp implements ICriticalNotifyCompletion, then it calls tmp.UnsafeOnCompleted instead.

' The variable "ec" gets disposed at the right time inside the Sub()

' If ec is null, then it invokes K1 directly instead of within ExecutionContext.Run

' There are optimizations for the case where ExecutionContext is unmodified.

From the compiler perspective, here's the bare minimum that you need to implement to be able to compile "Await t":

Dim tmp = t.GetAwaiter() must compile, and GetAwaiter must be either an instance method which takes no parameters, or an extension method which takes just one "this" parameter. (From this you can deduce that GetAwaiter cannot take optional parameters, and can take generic type parameters only if it's an extension method).
Dim b = tmp.IsCompleted must compile, and IsCompleted must be a property which takes no parameters and returns a Boolean. (From this you can deduce that IsCompleted must be an instance property).
tmp must implement INotifyCompletion. It can optionally also implement ICriticalNotifyCompletion. These must be "CLR implements" – the compiler will not use dynamic casts or user-defined conversions. If the delegate passed on OnCompleted/UnsafeOnCompleted is executed zero times then the async method will never resume. If it is executed more than once, then behavior of the async method is unspecified.
tmp.GetResult() must compile, and GetResult must be an instance method which takes no parameters. If GetResult is a function, then this function's return-value is used as the result of the Await operator. If GetResult is a sub (i.e. void-returning), then the Await operator returns void.

If you do a late-bound await then the rules are a little different. A late-bound await is "await t" where t has type dynamic (C#) or Object (VB with Option Strict Off).

1. Dim tmp = t.GetAwaiter() must compile as a late-bound invocation, and it will use the normal late-binder rules: it works with optional parameters, and IDynamicMetaObjectProvider, and even with a delegate field named GetAwaiter.

2. Dim b = tmp.IsCompleted must compile as a normal late-bound property access.

3. tmp must implement INotifyCompletion or ICriticalNotification. As before, it must implement them through CLR interface implementation; not through dynamic casts.

4. tmp.GetResult() must compile as a normal late-bound invocation, and the result of the late-bound invocation is the result of the late-bound Await operator.

These are only the bare-minimum requirements for the code to compile and have kind of specified behavior at runtime. The rest of it, of how you implement GetAwaiter/IsCompleted/OnCompleted/GetResult, is entirely up to you and convention.

What conventions the user expects from a custom awaiter

Here is code for a custom awaiter that satisfies all the compiler's requirements, and behaves approximately similarly to how the default TaskAwaiter behaves ("suspend execution of the method; then, when the operand has finished, resume execution on the same SynchronizationContext and with the same ExecutionContext").

This code is pointless! The only reason you'd write a custom awaiter is because you want something different from the default TaskAwaiter. (After all, if you were happy with the default TaskAwaiter behavior, then it'd be more efficient and more robust to just create one via TaskCompletionSource.Task.GetAwaiter). I'm putting this code here just as a starting point, to diverge from it. I've also made fields public to keep the code simple.

Public Class MyTask(Of T)

Public result As T

Public isCompleted As Boolean

Public continuations As New List(Of Action)

Public mutex As New Object

Public Function GetAwaiter() As MyTaskAwaiter(Of T)

Return New MyTaskAwaiter(Of T) With {.task = Me}

End Function

Public Sub SetResult(value As T)

Dim cc As List(Of Action)

SyncLock mutex

result = value

isCompleted = True

cc = continuations

continuations = Nothing

End SyncLock

For Each c In cc : c() : Next

End Sub

Public Sub AddContinuation(continuation As action)

SyncLock mutex

If Not isCompleted Then continuations.Add(continuation) : Return

End SyncLock

Task.Run(continuation)

End Sub

End Class

Public Class MyTaskAwaiter(Of T)
Implements System.Runtime.CompilerServices.INotifyCompletion

Public task As MyTask(Of T)

Public ReadOnly Property IsCompleted As Boolean

Get

SyncLock task.mutex

Return task.isCompleted

End SyncLock

End Get

End Property

Public Function GetResult() As T

Return task.result

End Function

Public Sub OnCompleted(continuation As Action) _
Implements INotifyCompletion.OnCompleted

Dim sc = If(SynchronizationContext.Current, New SynchronizationContext)

task.AddContinuation(Sub() sc.Post(Sub() continuation(), Nothing))

End Sub

End Class

Alternatively, if your code allows partially trusted callers, then the above code would be a security hole. Here's how we could write MyTaskAwaiter instead for this case:

<Assembly: Security.AllowPartiallyTrustedCallers>

Public Class MyTaskAwaiter(Of T) : Implements ICriticalNotifyCompletion

Public task As MyTask(Of T)

Public ReadOnly Property IsCompleted As Boolean

Get

SyncLock task.mutex

Return task.isCompleted

End SyncLock

End Get

End Property

Public Function GetResult() As T

Return task.result

End Function

<Security.SecurityCritical>

Public Sub UnsafeOnCompleted(continuation As Action) _
Implements ICriticalNotifyCompletion.UnsafeOnCompleted

Dim sc = If(SynchronizationContext.Current, New SynchronizationContext)

task.AddContinuation(Sub() sc.Post(Sub() continuation(), Nothing))

End Sub

Public Sub OnCompleted(continuation As Action) _
Implements INotifyCompletion.OnCompleted

Throw New NotSupportedException()

End Sub

End Class

AllowPartiallyTrustedCallers. Part of the .NET security model is that, if a method in your assembly can be invoked from a partially trusted caller, then none of your APIs can allow that caller to break free of the ExecutionContext they started in. A partially-trusted attacker might call the first MyTaskAwaiter.OnCompleted() directly (i.e. bypassing the Await operator) and hence execute an arbitrary lambda in the ExecutionContext of whoever invoked MyTask.SetResult(). The second version works around this by implementing ICriticalNotifyCompletion and putting <SecurityCritical> on the UnsafeOnCompleted method to prevent partially-trusted callers from invoking it. We already saw that the compiler prefers to use the ICriticalNotifyCompletion interface if it's available, over INotifyCompletion. We also saw that the generated code's call to UnsafeOnCompleted is made indirectly via a routine in mscorlib, which is why it's able to invoke <SecurityCritical> code even if your assembly is only partially trusted. (The code for MyTask would also have to be tightened up, so that continuations/AddContinuation can't be touched by partially trusted callers).

Race conditions. I believe the above code is free of race conditions and other concurrency bugs but I'm not sure. My first version suffered from several concurrency bugs. First bug: I forgot the lock on MyTask.AddContinuation(), allowing this interleaving:

Thread1 invokes MyTask.AddContinuation()
Thread1 passes the "Not isCompleted" test
Thread2 invokes MyTask.SetResult()
Thread2 sets isCompleted to true and executes all delegates in "continuations"
Thread1 adds its continuation to "continuations"
RESULT: the continuation from Thread1 never gets executed.

Second bug: I forgot the lock on MyTaskAwaiter.IsCompleted, so the code was vulnerable to memory reordering:

Thread1 invokes SetResult
WRITE:result
WRITE:isCompleted = true
Thread2 invokes IsCompleted followed by GetResult
READ:isCompleted returns true
READ:result
If the hardware re-ordered the two WRITE operations (allowed by ECMA but not CLR), then we might get this interleaving:
WRITE:isCompleted = true
READ:isCompleted returns true
READ:result returns an invalid value
WRITE:result
If the hardware did speculative execution and re-ordered the two READ operations (I don't know if this is even allowed!) then we might get this interleaving:
READ:result returns an invalid value
WRITE:result
WRITE:isCompleted = true
READ:isCompleted returns true
RESULT: in both cases the GetResult() call retrieved an invalid value

Third bug: In MyTask.SetResult(), the first version of my code looped over the continuations and executed them from inside the SyncLock. But this would have led to deadlock if any of the continuations themselves tried to await mytask from a different thread. The solution was to execute the continuations outside the SyncLock.

Fourth bug: In AddContinuation, the first version of my code executed the continuation directly. But this would have blown the stack if I had a loop which awaited a MyTask that had already completed. The solution was to schedule the continuation for later execution with Task.Run(continuation).

NotSupportedException. The AllowPartiallyTrustedCallers version of the code throws a NotSupportedException for anyone who calls its implementation of INotifyCompletion.OnCompleted. I figured there was no need, and it's hard to get it right so I'm better off just throwing the exception. After all, I'm only doing my custom awaiter to support the Await operator, and the compiler specs spell out the fact that the compiler won't invoke INotifyCompletion.OnCompleted on an awaiter that implements ICriticalNotifyCompletion.

What to do if SynchronizationContext.Current is null? My custom awaiter uses this code:
Dim sc = If(SynchronizationContext.Current, New SynchronizationContext) ' VB

var sc = SynchronizationContext.Current ?? new SynchronizationContext(); // C#

I did as part of implementing the common convention, that the "Await" operator will resume execution on whatever SynchronizationContext was current before it started. The question arises of what to do if Synchronization.Current was null, as is the case in a Console application or a unit test, or in the body of Task.Run(): where should the continuation be posted? In my code, the new synchronization context will post the continuation-delegate to the threadpool, which is decent-enough behavior.

Conclusions

The custom awaiter code is difficult – difficult to follow the conventions, and difficult to write without concurrency bugs. If at all possible it's better to stick to the standard TaskAwaiter, or at least to build your own custom awaiter as a wrapper around TaskAwaiter.

How to await a command-line process, and capture its output

Lucian Wischik, MSFT — Sat, 08 Dec 2012 03:27:00 GMT

[This post is part of a series How to await a storyboard, and other things]

I want to invoke an external executable, and await until it’s finished, and get back its output. (In my case, the external executable is called “tidy.exe” – an excellent open-source utility for turning HTML into clean parseable XML.) To cut a long story short, here’s my final code. The rest of the article has explanations of how it works and why.

Async Function Attempt4Async() As Task

Dim client As New HttpClient

Dim html = Await client.GetStringAsync("http://blogs.msdn.com/lucian")

Dim tidy = Await RunCommandLineAsync("tidy.exe",

"-asxml -numeric -quiet --doctype omit",
html)

Dim xml = XDocument.Parse(tidy)

End Function

Async Function RunCommandLineAsync(cmd As String, args As String, stdin As String,

Optional cancel As CancellationToken = Nothing

) As Task(Of String)

Using p As New Process

p.StartInfo.FileName = cmd

p.StartInfo.Arguments = args

p.StartInfo.UseShellExecute = False

p.StartInfo.RedirectStandardInput = True

p.StartInfo.RedirectStandardOutput = True

p.StartInfo.RedirectStandardError = True

If Not p.Start() Then Throw New InvalidOperationException("no tidy.exe")

Using cancelReg = cancel.Register(Sub() p.Kill())

Dim tin = Async Function()

Await p.StandardInput.WriteAsync(stdin)

p.StandardInput.Close()

End Function()

Dim tout = p.StandardOutput.ReadToEndAsync()

Dim terr = p.StandardError.ReadToEndAsync()

Await Task.WhenAll(tin, tout, terr)

p.StandardOutput.Close()

p.StandardError.Close()

Await Task.Run(AddressOf p.WaitForExit)

Dim stdout = Await tout

Dim stderr = Await terr

If cancel.IsCancellationRequested Then

Throw New OperationCanceledException(stderr)

ElseIf String.IsNullOrEmpty(stdout) Then

Throw New Exception(stderr)

Else

Return stdout

End If

End Using

End Function

Process.Exited event

Stephen Toub wrote a very useful article “Await Anything” that I always use as my starting point. He included an example of awaiting for a process. That article used the Process.Exited event, and lead me to this code:

Async Function Attempt1Async() As Task

Dim p As New Process

p.StartInfo.FileName = "notepad.exe"

p.Start()

Dim tcs As New TaskCompletionSource(Of Integer)

p.EnableRaisingEvents = True

AddHandler p.Exited, Sub() tcs.TrySetResult(p.ExitCode)

If p.HasExited Then tcs.TrySetResult(p.ExitCode)

Await tcs.Task

End Function

Note the check for p.HasExited comes after adding the handler. If we didn’t do that, and if the process exited before adding the handler, then we’d never hear about it.

The code works great as is. But once I added in code to capture the output then I uncovered a race condition inside the Process.Exited event – it seems that the triple combination of ReadToEnd(), Dispose() and HasExited can cause the Exited event to fire more than once!

Repro: The following code will non-deterministically throw an exception because the process finishes and fires an event, then “await tcs.Task” finishes, then the event is fired a second time, and its attempt to read “p.ExitCode” is now invalid. The exception also seems to happen even if I RemoveHandler before EndUsing.

Async Function Attempt2Async() As Task

For i = 0 To 1000

Using p As New Process

p.StartInfo.FileName = "cmd.exe"

p.StartInfo.Arguments = "/c dir c:\windows /b"

p.StartInfo.UseShellExecute = False

p.StartInfo.RedirectStandardOutput = True

p.Start()

Dim stdout = Await p.StandardOutput.ReadToEndAsync()

Dim tcs As New TaskCompletionSource(Of Integer)

p.EnableRaisingEvents = True

AddHandler p.Exited, Sub() tcs.TrySetResult(p.ExitCode)

If p.HasExited Then tcs.TrySetResult(p.ExitCode)

Await tcs.Task

End Using

Next i

End Function

If the mechanism of communication is a wait-handle, then you need to block some thread

There might be clever ways to make Process.Exited work, but I didn’t care to risk them. So here’s my next attempt at awaiting until the process has finished. I tested it inside a while loop, and it doesn’t suffer from the same race condition as before.

Async Function Attempt3Async() As Task

Dim p As New Process

p.StartInfo.FileName = "notepad.exe"

p.Start()

Await p

End Function

<Extension> Function GetAwaiter(p As Process) As TaskAwaiter

Dim t As Task = Task.Run(AddressOf p.WaitForExit)

Return t.GetAwaiter()

End Function

Here’s an explanation for why the GetAwaiter() is written as it is. Process has a method p.WaitForExit() which blocks a thread until the process has finished. It also exposes an IntPtr property “Handle”, which is the raw win32 handle that will be signalled once the process has completed. I want to await until the process has finished, or equivalently until the handle is signaled. If the mechanism of communication is a wait handle, then I need to block some thread. Since I have to block some thread anyway, I might as well use WaitForExit(), which is slightly easier than the handle.

RunCommandLineAsync

I already started this article with the finished code, “RunCommandLineAsync”. Its code is subtle. Here’s an explanation.

Cancellation. If cancellation is requested, then I need to be able to terminate the process. I do that with “cancel.Register(Sub() p.Kill())”. As soon as cancellation is requested, this will invoke the lambda, which will kill the process. This will terminate the process abruptly, closing the handles cleanly (i.e. without exception). In this state I have no guarantee that stdout is complete. Therefore, I throw an OperationCancelledException. Note that even if cancellation is requested and the process gets killed, I still await until the process has completely finished before returning (rather than returning immediately and letting it clean up asynchronously in the background). I figure this is a more hygienic design.

Tin, Tout, Terr. The precise way that “RedirectStandardInput/Output/Error” will work depends on the internal details of the process we’re launching. All we can say with any certainty is that (1) the process might not finish until we close its StandardInput; (2) we might have to read data from StandardOutput/StandardError before we can write any more data into StandardInput; (3) we might have to write more data into StandardInput before we can read more data from StandardOutput/StandardError. Those constraints imply that the three tasks “tin/tout/terr” must all be launched concurrently, and then awaited using Task.WhenAll().

Await tout. The code does “Dim stdout = Await tout” even after we know that tout has finished (because we awaited Task.WhenAll). That’s fine. It merely means that the call to “Await tout” will complete immediately.

Async vs threads. A few years ago, I wrote similar code using threads instead of async. I'm happy that the async version is cleaner and more readable.

Wrapping it up

My ultimate goal behind this project was to be able to scrape web-pages. And I wanted to use VB’s XML-literals to make that easy. Here’s how I wrapped it up, to scrape all img tags from a page:

Async Function Attempt5Async(Optional cancel As CancellationToken = Nothing) As Task

Dim client As New HttpClient

Dim xml = Await client.GetXmlAsync("http://blogs.msdn.com/lucian", cancel)

For Each img In (From i In xml...<img> Select i.@src Distinct)

Console.WriteLine(img)

End Function

Async Function GetXmlAsync(client As HttpClient, uri As String,

Optional cancel As CancellationToken = Nothing

) As Task(Of XDocument)

Using response = Await client.GetAsync(uri, cancel)

Dim html = Await response.Content.ReadAsStringAsync()

Dim tidy = Await RunCommandLineAsync("tidy.exe",

"-asxml -numeric -quiet --doctype omit", html, cancel)

Return XDocument.Parse(tidy)

End Using

End Function

Await HttpClient.GetStringAsync() and cancellation

Lucian Wischik, MSFT — Sat, 08 Dec 2012 02:39:00 GMT

I’m a big fan of supporting cancellation on async methods. I was recently wondering how to do this for HttpClient.GetStringAsync(). This post has some answers.

Async Function TestAsync(uri As Uri, Optional cancel As CancellationToken = Nothing

) As Task(Of String)

Dim client As New HttpClient

Return Await client.GetStringAsync(uri, cancel)
' error: no overload of GetStringAsync takes a cancellation token

End Function

My first attempt is above. But it doesn’t compile, because HttpClient.GetStringAsync doesn’t have any overloads that take a CancellationToken.

The answer? Instead of using GetStringAsync, we have to use GetAsync, which does take a cancellation-token parameter:

Dim client As New HttpClient

Using response = Await client.GetAsync(uri,
HttpCompletionOption.ResponseContentRead,
cancel)

Return Await response.Content.ReadAsStringAsync()

End Using

The “ResponseContentRead” option can be omitted, since it is the default for calls to HttpClient.GetAsync. I included it just for clarity. What it means is that the entire body of the response is read by the call to HttpClient.GetAsync(), and so is subject to the cancellation token. Therefore the next call to ReadAsStringAsync() will complete immediately and needn’t worry about cancellation.

If you wanted to use the option “ResponseHeaderRead” instead, then cancellation looks different:

Try

Using response = Await client.GetAsync(uri,

HttpCompletionOption.ResponseHeadersRead,

cancel)

Using cancelreg = cancel.Register(Sub() response.Dispose())

Return Await response.Content.ReadAsStringAsync()

End Using

Catch ex As ObjectDisposedException

If cancel.IsCancellationRequested Then Throw New OperationCanceledException

Throw

End Try

Explanation: in this case, only the headers of the response are read by HttpClient.GetAsync(), and so only they are subject to cancellation by the cancellation token. When we next invoke ReadAsStringAsync(), this operation might take a long time, and so we need to figure out how to cancel it. No overload of ReadAsStringAsync() takes a cancellation token. The best we can do is dispose of the HttpResponseMessage. This will cause the ReadAsStringAsync() method to terminate abruptly with an ObjectDisposedException.

How to await a button click

Lucian Wischik, MSFT — Wed, 28 Nov 2012 17:07:00 GMT

[This post is part of a series How to await a storyboard, and other things]

Sometimes we want to await until a button has been clicked. Once place where this is useful is, for instance, when displaying the message “Click button to continue”.

Await button1.WhenClicked()

Why do it this way? Well, if you did it normally by writing top-level click handler event, then the event would be "always on". You'd need to enforce the invariant that the button was invisible when you didn't need it. Or, if the button were shared for different tasks at different times, then you'd have needed a top-level state field to record which of those tasks it's currently in use for. Or maybe you'd switch around which delegate was assigned to its Completed handler. Those approaches are all valid, but yhis "Await" approach works out nicer in some cases.

Here’s some code to support the above syntax. It's for win8, using RoutedEventHandler, but is easy to adapt to other platforms:

<Extension> Function WhenClicked(btn As Button) As Task

Dim tcs As New TaskCompletionSource(Of Object)

Dim lambda As RoutedEventHandler = Sub(s, e)

RemoveHandler btn.Click, lambda

tcs.SetResult(Nothing)

End Sub

AddHandler btn.Click, lambda

Return tcs.Task

End Function

Note on naming convention: The .NET async guidelines say that async methods should end in the suffix "Async". I've violated that guideline here. When we made the guidelines, I don't think we thought through the cases where you use async for events. This alternative name is closer in spirit to Task.WhenAll and Task.WhenAny. Thanks to Quartermeister for the suggestion.

How to await a drag operation

Lucian Wischik, MSFT — Wed, 28 Nov 2012 17:03:00 GMT

[This post is part of a series How to await a storyboard, and other things]

In my app, I want to await a drag operation. Here’s how I’d like to await it:

Private Async Sub rectangle1_PointerPressed(sender As Object,
e As PointerRoutedEventArgs) Handles rectangle1.PointerPressed
Await DragAsync(rectangle1, e)
End Sub

Here below is my implementation for win8, for dragging a UIElement on a canvas. I didn't bother supporting a cancellation-token because it's easy for the user to cancel: just by letting go!

Function DragAsync(shape As UIElement, origE As PointerRoutedEventArgs,

Optional progress As IProgress(Of Point) = Nothing) As Task

Dim tcs As New TaskCompletionSource(Of Point)

Dim origPos = New Point(Canvas.GetLeft(shape), Canvas.GetTop(shape))

Dim origPt = origE.GetCurrentPoint(Nothing).Position

Dim lambdaMoved As PointerEventHandler = Nothing

Dim lambdaReleased As PointerEventHandler = Nothing

lambdaMoved = Sub(s, e)

Dim pt = e.GetCurrentPoint(Nothing).Position

If Not progress Is Nothing Then progress.Report(pt)

Canvas.SetLeft(shape, origPos.X + pt.X - origPt.X)

Canvas.SetTop(shape, origPos.Y + pt.Y - origPt.Y)

End Sub

lambdaReleased = Sub(s, e)

RemoveHandler shape.PointerMoved, lambdaMoved

RemoveHandler shape.PointerReleased, lambdaReleased

shape.ReleasePointerCapture(origE.Pointer)

tcs.SetResult(e.GetCurrentPoint(Nothing).Position)

End Sub

shape.CapturePointer(origE.Pointer)

AddHandler shape.PointerMoved, lambdaMoved

AddHandler shape.PointerReleased, lambdaReleased

Return tcs.Task

End Function

Actually, let’s make it fancy with a rubber-band for the drag operation, relying on the fact that we wrote DragAsync to report back its progress:

Private Async Sub rectangle1_PointerPressed(sender As Object,

e As PointerRoutedEventArgs) Handles rectangle1.PointerPressed

Dim origPt = e.GetCurrentPoint(Nothing).Position

Dim rubber As New Shapes.Line With

{.X1 = origPt.X, .Y1 = origPt.Y,

.Stroke = New SolidColorBrush(Colors.Red),

.Visibility = Xaml.Visibility.Collapsed}

canvas1.Children.Add(rubber)

Dim p As New Progress(Of Point)(Sub(pt)

rubber.X2 = pt.X : rubber.Y2 = pt.Y

rubber.Visibility = Visibility.Visible

End Sub)

Await DragAsync(rectangle1, e, p)

canvas1.Children.Remove(rubber)

End Sub

How to await a MediaElement (PlaySound in Windows 8)

Lucian Wischik, MSFT — Wed, 28 Nov 2012 17:02:00 GMT

[This post is part of a series How to await a storyboard, and other things]

Let’s look at making MediaElement awaitable. This is the kind of idiom I’d like to use:

Try

Await mediaElement1.OpenAsync(New Uri("ms-appx:///Assets/boooo.mp3"))

Await mediaElement1.PlayAsync()

Catch ex As Exception

...

End Try

Q. Why do I want to await it? Why not just set its AutoPlay property to True, then set its Source property, and let it go?

A. Well, AutoPlay is fine for simple scenarios. But it doesn’t compose in the nice way that await does. It doesn’t let me use my familiar exception handling blocks - inside I’d have to sign up to a MediaFailed event handler. And it doesn’t let me compose in other interesting ways, e.g.

Await Task.WhenAll(storyboard1.PlayAsync(), mediaElement1.Play())

ALERT. MediaElement must be in the visual tree. Typically that means you create it as a child of your current Xaml page. If you want it to play globally, i.e. independent of which page is being shown, read this thread: http://social.msdn.microsoft.com/Forums/en-US/winappswithcsharp/thread/241ba3b4-3e2a-4f9b-a704-87c7b1be7988/

Jumping straight to the answer, here’s my attempt at code that supports awaitability. There may be bugs in it: it’s complex, and this is my first attempt. Requirements: neither of these operations is re-entrant: i.e. if other operations are performed on a given media element while either one is in progress, it might go awry.

<Extension> Function OpenAsync(media As MediaElement, uri As Uri,
Optional cancel As CancellationToken = Nothing) As Task

Dim tcs As New TaskCompletionSource(Of Object)

If cancel.IsCancellationRequested Then

tcs.SetCanceled()

Return tcs.Task

End If

If media.CurrentState = MediaElementState.Buffering OrElse

media.CurrentState = MediaElementState.Opening OrElse

media.CurrentState = MediaElementState.Playing Then

tcs.SetException(New Exception("MediaElement not ready to open"))

Return tcs.Task

End If

Dim lambdaOpened As RoutedEventHandler = Nothing

Dim lambdaChanged As RoutedEventHandler = Nothing

Dim lambdaFailed As ExceptionRoutedEventHandler = Nothing

Dim cancelReg As CancellationTokenRegistration? = Nothing

Dim removeLambdas As Action = Sub()

RemoveHandler media.MediaOpened, lambdaOpened

RemoveHandler media.MediaFailed, lambdaFailed

RemoveHandler media.CurrentStateChanged, lambdaChanged

If cancelReg.HasValue Then cancelReg.Value.Dispose()

removeLambdas = Sub() Return

' in case two lambdas get fired one after the other

End Sub

lambdaOpened = Sub()

removeLambdas()

tcs.TrySetResult(Nothing)

End Sub

lambdaFailed = Sub(s, e)

removeLambdas()

tcs.TrySetException(New Exception(e.ErrorMessage))

End Sub

lambdaChanged = Sub()

If media.CurrentState <> MediaElementState.Closed Then Return

removeLambdas()

tcs.TrySetCanceled()

End Sub

AddHandler media.MediaOpened, lambdaOpened

AddHandler media.MediaFailed, lambdaFailed

AddHandler media.CurrentStateChanged, lambdaChanged

media.Source = uri

If Not tcs.Task.IsCompleted Then

' The above condition guards against lambas being invoked by Source assignment

cancelReg = cancel.Register(

Sub()

Dim dummy = media.Dispatcher.RunAsync(Core.CoreDispatcherPriority.Normal,

Sub()

media.ClearValue(MediaElement.SourceProperty)

End Sub)

End If

Return tcs.Task

End Function

<Extension> Function PlayAsync(media As MediaElement,
Optional cancel As CancellationToken = Nothing) As Task

Dim tcs As New TaskCompletionSource(Of Object)

If cancel.IsCancellationRequested Then

tcs.SetCanceled()

Return tcs.Task

End If

If media.CurrentState <> MediaElementState.Paused Then

tcs.SetException(New Exception("MediaElement not ready to play"))

Return tcs.Task

End If

Dim lambdaEnded As RoutedEventHandler = Nothing

Dim lambdaChanged As RoutedEventHandler = Nothing

Dim cancelReg As CancellationTokenRegistration? = Nothing

Dim removeLambdas As Action = Sub()

RemoveHandler media.MediaEnded, lambdaEnded

RemoveHandler media.CurrentStateChanged, lambdaChanged

If cancelReg.HasValue Then cancelReg.Value.Dispose()

removeLambdas = Sub() Return

End Sub

lambdaEnded = Sub()

removeLambdas()

tcs.TrySetResult(Nothing)

End Sub

lambdaChanged = Sub()

If media.CurrentState <> MediaElementState.Stopped Then Return

removeLambdas()

tcs.TrySetCanceled()

End Sub

AddHandler media.MediaEnded, lambdaEnded

AddHandler media.CurrentStateChanged, lambdaChanged

media.Play()

If Not tcs.Task.IsCompleted Then

cancelReg = cancel.Register(

Sub()

Dim dummy = media.Dispatcher.RunAsync(Core.CoreDispatcherPriority.Normal,

Sub()

media.Stop()

End Sub)

End If

Return tcs.Task

End Function

What is the interactive behavior of MediaElement?

There’s no way I could make MediaElement awaitable without knowing the details of how it behaves. I did my PhD on the “pi calculus”, a sort of mathematical algebra for describing and specifying interactive behaviors. Now I’m not going to delve into mathematics here, but I believe that software engineering requires the same careful approach when you get down to the edge cases of async. Turning a complex event-based thing into a Task-based thing is one of those edge cases.

When studying the interactive behavior of a thing, then you HAVE to understand its complete state diagram. You have to understand in which states you’re allowed to trigger which stimuli (e.g. invoking the Play() method), and what their effect will be. You have to understand in which states the thing might have its own internal stimuli (e.g. reaching the end of a piece of media) and what effects these will have. If the documentation fails to specify all this, then you'll have to discover it by yourself.

I’m not a XAML expert. I don’t know the full details of how MediaElement behaves. But from experiment, this is what I believe is its state diagram:

State	Possible internal transitions	Upon setting Source	Upon reset of Source	Upon invoking Play()	Upon invoking Stop()
Closed		-> Opening
Opening	-> Paused (MediaOpened) -> Closed (MediaFailed)	?	-> Closed
Paused		-> Opening	-> Closed	-> Buffering -> Playing	-> Stopped
Buffering	-> Playing	?	?		-> Stopped
Playing	-> Buffering -> Paused (MediaEnded)	?	?		-> Stopped
Stopped		-> Opening	?	-> Buffering -> Playing

Read the table like this: “If the media element is currently in <State>, then it might transition “->” to another state either due to its internal workings or due to some method being invoked. If it does transition, then first its CurrentState property is changed, then it may fire a specified event on the UI thread, then it fires the CurrentStateChanged event on the UI thread.”

The media element starts in state “Closed” upon construction. If its Source property is set in XAML, then that has the same effect as setting the source property in code. The column for “reset of source” refers to invoking the method “mediaElement1.ClearValue(MediaElement.SourceProperty)”. That is the only way I have found to completely reset the Source property from code.

I noticed that the transition “Opening -> Closed (MediaFailed)” can take up to 10 seconds in the case of an http:// uri which either doesn’t exist (returns an error http code) or where your app lacks Internet Client permissions.

I haven’t tried to characterize every possible method than can be invoked on mediaElement. I only characterized the ones needed to support my goals.

Conclusions

Does this code look way more complicated to you than it should? It does to me. To be clear, the use of the code is very simple

Await mediaElement1.OpenAsync(New Uri("ms-appx:///Assets/boooo.mp3"))

Await mediaElement1.PlayAsync()

It’s just the implementation of awaitability-support that’s complex. Why so complex? Actually, my initial attempt was much simpler. It merely played a media element which already had its Source property set to the right value, and had already opened it successfully.

<Extension> Function PlayAsync1(media As MediaElement) As Task

Dim tcs As New TaskCompletionSource(Of Object)

Dim lambda As RoutedEventHandler = Sub()

RemoveHandler media.MediaEnded, lambda

tcs.SetResult(Nothing)

End Sub

AddHandler media.MediaEnded, lambda

media.Play()

Return tcs.Task

End Function

It got more complex because (0) I wanted to support setting the source property, (1) I wanted to support cancellation, (2) I wanted to figure out the complete state diagram, to know in which states it’s valid to call PlayAsync, and (3) I wanted to know whether PlayAsync might ever end in failure. People often complain that “the Play() method doesn’t do anything”, and the answer is that Play() is only valid after MediaOpened has fired.

My analysis isn’t complete. I don’t know what happens if you set the source at the wrong time. What would happen if you did “Dim t1 = media1.OpenAsync(uri1), t2 = media1.OpenAsync(uri2)” ? Would it end up with a random one of the two URIs opened? Or would it end up in some broken internal state? Or what happens if, in response to a button-click, you did “Await media1.OpenAsync(uri)”, and the user impatiently clicks the button a second time while it was still opening the first one? Would it finish the first open operation? or abort it and start the second open operation? Or would it end in some broken state?

The ultimate problem is that MediaElement has two very different uses – either as a full-blown media playing control with start/stop/seek controls, or just for doing PlaySoundAsync(). The first use isn’t much like a task at all. That’s because there’s no single obvious completion event, and because it isn’t monotonic. A monotonic state diagram is one where you never return to an earlier state. Readonly variables are also monotonic, since they transition from “unassigned” to “assigned-with-value”. Monotonic things are much easier to program against. Compare the state-diagram for MediaElement above, with the simpler state diagram for Tasks that come out of async methods. It’s easy to know where you are with tasks.

State	Possible internal transitions
WaitingForActivation	-> RanToCompletion -> Canceled -> Faulted
RanToCompletion
Canceled
Faulted

I do wonder whether it might be best to abandon awaitability of MediaElement itself, and encapsulate it into a simpler API like PlaySoundAsync. But this then would have its own problems: how do I pause it? how do I declare it in XAML? how do I open it just once but then play it multiple times? What forms of uri does it support? How to set the volume? How does it find the visual-tree? My encapsulation might end up just as complicated as MediaElement!

Function PlaySoundAsync(uri As Uri, Optional cancel As CancellationToken) As Task

...

End Function

How to await a storyboard, and other things

Lucian Wischik, MSFT — Wed, 28 Nov 2012 16:54:00 GMT

This post is part of a series about an important new design pattern, awaiting events... I also made a Channel9 video introduction "Async Over Events". In this blog series:

Other resources:

AwaitableUI - a codeplex project that provides similar wrappers
Awaitable ManagedAnimation class for WinRT

I want to be able to “await a storyboard”.

Await storyboard1.PlayAsync()

Actually, there are several syntaxes I might choose for awaiting a storyboard:

Await storyboard1

Await storyboard1.PlayAsync()

Await PlayStoryboardAsync(storyboard1)

Indeed there are lots of other things that I’d like to be awaitable but aren’t (and for each of them I’ll have to choose which of the three syntaxes). This article is about making things become awaitable. First we’ll see how to make things awaitable. Then we’ll evaluate which of the three syntaxes is appropriate in which circumstances.

How to await a storyboard

I’ll jump straight to the punch-line. This is the code you need to await a storyboard in win8:

<Extension> Function PlayAsync(sb As Animation.Storyboard) As Task

Dim tcs As New TaskCompletionSource(Of Object)

Dim lambda As EventHandler(Of Object) = Sub()

RemoveHandler sb.Completed, lambda

tcs.TrySetResult(Nothing)

End Sub

AddHandler sb.Completed, lambda

sb.Begin()

Return tcs.Task

End Function

For this example, and for the next few articles, I’m working on a simple “blank page” Windows 8 app with the following in its Mainpage.xaml:

<Page

x:Class="AppVB.MainPage"

xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"

xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"

xmlns:local="using:AppVB"

xmlns:d="http://schemas.microsoft.com/expression/blend/2008"

xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"

mc:Ignorable="d">

<Page.Resources>

<DoubleAnimation Storyboard.TargetName="rectangle1"

Storyboard.TargetProperty="(Canvas.Left)"

From="0" To="500" Duration="0:0:1"/>

</Storyboard>

</Page.Resources>

</Canvas>

</Page>

Why make things awaitable?

There are three main reasons for wanting something to be awaitable:

(1) If I can await the storyboard, then I can use the “await” keyword instead of adding a callback handler to the Storyboard.Completed event. I don’t like callbacks because they turn into spaghetti code; I prefer to use “await” because it keeps my code readable.

(2) If I can await the storyboard via the last two syntaxes, then it opens up powerful possibilities, e.g.

Await Task.WhenAll(storyboard1.PlayAsync(), mediaElement1.PlayAsync())

In a future article I’ll dive into the new async choreography design patterns that this opens up, for coding the sequences of button-clicks and animations and events that make up an interactive application.

(3) If I make something awaitable, then it gives me some interesting flexibility. I can change the way that code resumes after an await, similar to how “Await t.ConfigureAwait(false)” resumes more efficiently. I can make an awaitable that resumes at a different priority or even a different thread, e.g. “Await SwitchThread()”. I can implement co-routines. These advanced techniques all require custom awaiters, which I’ll address in a future article.

How to support the raw Await syntax

The first await syntax is the most striking:

Await e

If the compiler encounters the above expression, then under the hood it makes a call to

Dim temp = e.GetAwaiter()

Therefore: to allow users to write “Await e”, we must provide a GetAwaiter method. It has to return a value which satisfies the awaiter pattern. I’ll spell out the full details of that in a future post, but for now we’ll just return an instance of System.Runtime.CompilerServices.TaskAwaiter, or its generic version TaskAwaiter(Of T). Here’s how I might do it for Storyboard. Storyboard is a built-in type, so I can’t add GetAwaiter as an instance method, so instead I’ll provide it as an extension method:

<Extension> Function GetAwaiter(sb As Animation.Storyboard) As TaskAwaiter

Dim tcs As New TaskCompletionSource(Of Object)

Dim lambda As EventHandler(Of Object) = Sub()

RemoveHandler sb.Completed, lambda

tcs.TrySetResult(Nothing)

End Sub

AddHandler sb.Completed, lambda

sb.Begin()

Dim t As Task = tcs.Task

Dim awaiter As System.Runtime.CompilerServices.TaskAwaiter = t.GetAwaiter()

Return awaiter

End Function

NB. The first code in this function was for “<Extension> Function PlayAsync(...) As Task”. The reason that works is because the built-in Task type has its own GetAwaiter() instance method which returns TaskAwaiter.

If I want to allow users to write “Dim x As Integer = Await storyboard1”, then I’d have had to return TaskAwaiter(Of Integer) instead. And likewise, Task(Of T) has a GetAwaiter() instance method which returns TaskAwaiter(Of T).

There are some additional restrictions. GetAwaiter must be a method (not a property or delegate field). It cannot have any optional parameters. It’s allowed to be an instance or an extension method, and if an extension method then it’s allowed to be generic. (Technical note: if the call to GetAwaiter were late-bound, i.e. if “e” had type Object in VB or dynamic in C#, then the restrictions are relaxed and normal late-bound rules apply.)

You might be wondering about callbacks. The whole point of “await” is to liberate you from callbacks because they’re unpleasant to code with. But we’ve used a callback right here in the GetAwaiter method. Well, what we’ve done is localize the callbacks. The entire rest of my code can happily use “Await Storyboard” without ever using callbacks. It’s only this one small localized routine that encapsulates the callback.

Note: I’m doing this trick with “TaskCompletionSource” because I’m awaiting something that fires events. TaskCompletionSource is the way to turn an event-based API into a Task-based API. (It’s my experience that most things I want to await, if they aren’t already awaitable, are based around events.) If I were awaiting something easier, I might have been able to skip TaskCompletionSource entirely and instead just done this:

<Extension> Function GetAwaiter(sb As Animation.Storyboard) As TaskAwaiter

Dim t As Task = SomeInternalFunctionAsync(...)

Dim awaiter As System.Runtime.CompilerServices.TaskAwaiter = t.GetAwaiter()

Return awaiter

End Function

Note: There doesn’t exist a plain non-generic “TaskCompletionSource” for creating a non-generic Task. The only thing you can do is use the generic “TaskCompletionSource(Of T)”, create a generic “Task(Of T)”, and then cast the result to the non-generic Task.

What syntax to choose for awaiting?

This article started with three possible syntaxes for awaiting:

Await storyboard1

Await storyboard1.PlayAsync()

Await PlayStoryboardAsync(storyboard1)

It’s up to us as coders which syntax we’ll support. The first option looks slick, but I think it is poor style – because it’s not clear what is being awaited. The next two options are equally fine.

Lucian’s personal recommendation: I think it is good style to await an imperative verb (method) that ends in the suffix “Async”. It is good style to await a noun (expression) so long as the type of that noun has the characteristics of a hot task, and not much more, and it’s obvious which completion event you’re thinking about.

Here are some examples of how I might chose to make something awaitable.

Dim iaa As IAsyncAction = ...

Await iaa ' good

Await iaa.StartAsync() ' bad

Await iaa.AsTask() ' good

Windows.Foundation.IAsyncAction is always given to you “hot” i.e. already running. Also it really is very much like a task, i.e. it starts then finishes then can’t be restarted. Its only additional behaviors are cancellation and progress, which are also task-like. Therefore “Await iaa” is easily understandable. The form “Await iaa.StartAsync()” is confusing because it has already started.

Dim timeout_ms = 150

Await timeout_ms ' bad

Await Task.Delay(timeout_ms) ' good

The first form is just confusing. There’s no obvious completion event for a number!

Dim sb As StoryBoard = ...

Await sb ' bad

Await sb.PlayAsync() ' good

Windows.UI.Xaml.Media.Animation.Storyboard feels like a very rich type, with far more properties and behaviors than just a Task-like thing, so it’s good to call out explicitly which of those behaviors we’re awaiting. And again, the second form makes it clear that we’re encapsulating a call to sb.Start().

Dim media As MediaElement = ...

Await media ' bad

Await media.PlayAsync() ' good

Windows.UI.Xaml.Controls.MediaElement has a common well-understood verb, “Play”. When we make it awaitable, of course we should it clear that we’re awaiting completion of that verb.

Dim r As Rectangle = ...

Await r ' bad

Await r.DragAsync() ' bad
Await DragAsync(r) ' good

Windows.UI.Xaml.Shapes.Rectangle. Here I want to await until a drag operation has finished. The first form is terrible because it’s not clear what’s being awaited. The second form is ugly because DragAsync() isn’t an inherent property of shapes, and should be an extension method on them.

Await button1 ' ?

Await button1.ClickAsync() ' ?

Await ButtonClickAsync(button1) ' ?

Await ButtonClick(button1) ' ?

What do you think of the first form? It’s fairly obvious that we’re waiting until the button gets clicked. The second form I think looks bad because it gives the impression we’re performing the click. I’m undecided between the third and fourth forms.

Dim proc As Process = ...

Await proc ' ?

Await proc.RunAsync() ' good

System.Diagnostics.Process is often thought of like a task, i.e. something with a well-defined lifetime. So I wouldn’t mind awaiting it directly. However, users might easily forget to call Start() on it and so I prefer the second form.

How to hibernate async methods (how to serialize Task)

Lucian Wischik, MSFT — Sat, 24 Nov 2012 01:33:00 GMT

Sometimes people ask the question “How can I serialize a Task?” If you try, it throws a SerializationException:

Dim t = TestAsync()

Using stream As New MemoryStream

Dim formatter As New Formatters.Binary.BinaryFormatter

formatter.Serialize(stream, t)
' SerializationException: Type 'Task' is not marked as serializable.

End Using

Async Function TestAsync() AsTask

Await Task.Delay(10)

Console.WriteLine("a")

End Function

It’s the wrong question to ask. A Task is something that conceptually can’t be serialized. Consider as example this task: “Dim t As Task = httpClient.DownloadStringAsync(url)”. The task object embodies two things: (1) task is a promise that the operation will be completed at some time in the future, regardless of whether that operation is being performed locally in an async method or thread, or through some packets hurtling through the internet; (2) a task contains a collection of signed-up behaviors, delegates, that will be invoked once the task transitions to the “completed” state. It doesn’t make sense to talk about serializing either of those two things.

The right question is: “How can I serialize the current state of my progress through this async method?” In this article I’ll show how.

Caveat

Personally, I don’t yet believe that this is a good idea. I’m posting this blog article in case I’m wrong, and in case anyone finds a good design pattern for it.

In the late 1990s I did my PhD on the topic of message-passing as a way to write concurrent programs. It was a bad idea: “await” is better than message-passing. At the time most of my peers were doing PhDs on mobile agents which, I think, are even worse idea. Mobile agents are pieces of code which can serialize their state to disk, be transmitted over the internet, be deserialized on a remote machine, and resume their execution there. Biztalk/XLang also has the notion of long-running interactive processes which can be suspended to disk and resumed later.

I don’t think it’s a good idea to serialize processes/functions/behaviors/agents. That’s because they’re intangible. I think we should figure out the plain-old-data embodiment of the current state of our work, and serialize that data to disk, and deserialize it again all as a passive data-structure rather than an active agent. That way, if someone asks “What is currently in flight?” then in answer we can point them to the data structures.

So please only follow this article if you have a real vision of where it will be practically useful and clean.

Using a serializable async method

Here’s my serializable async method “TestAsync”.

Async Function TestAsync(min As Integer, max As Integer) As Task
  For i = min To max
  Console.WriteLine(i)
  Await Task.Delay(100)
  If i = 5 Then Await Hibernator.Hibernate("a.hib")
  Next
End Function

One of my programs is going to kick off the async method. But half way through, the async method will hibernate itself to disk and terminate abruptly with a “Serialized-to-disk” exception.

Console.WriteLine("INITIATING...")
Try
Await TestAsync(1, 10)
Catch ex As OperationCanceledException
Console.WriteLine("EX" & ex.Message)
End Try

INITIATING ASYNC METHOD...

5
EX Serialized to a.hib

My other program might be running on a different machine, or maybe just a long time later, or maybe it just needed the async method to be snapshotted to disk for resilience against errors. It’s going to resume that same async method, mid-flight, from disk.

Console.WriteLine("RESUMING...")

Dim t As Task = Hibernator.Resume("a.hib")

Await t

Console.WriteLine("done")

RESUMING...

10
done

Q. What makes the async method serializable? A. All its parameters and local variables must themselves be serializable.

Q. Why design it so that hibernation throws a SerializationException from the original method? A. Imagine you stepped into a Star Trek transporter to beam down to the planet’s surface. Would you want the original copy of you in the transporter bay to go on living, while the new copy on the planet’s surface goes on living as well? So there are now two of you? You’d have to be very careful of side-effects, in case the two copies tried to do the same thing at the same time.

Q. Why design it so the async method itself is responsible for hibernating, rather than its caller? A. I think that hibernation is serious business, akin to thread-termination or task-cancellation, which should only be done cooperatively when the async method is ready for it. I’d probably want to extend the above code to use HiberationTokenSource / HibernationToken, by analogy to CancellationTokenSource / CancellationToken.

Implementing serialization of async methods

Here’s my implementation of the Hibernate and Resume functionality. Code first; discussion afterwards.

<Serializable>
Public Class Hibernator : Implements INotifyCompletion, ISerializable
  Private m_ex As Runtime.ExceptionServices.ExceptionDispatchInfo
  Private m_fn As String

  '===================================================================================

  Public Shared Function Hibernate(fn As String) As Hibernator
  ' Intended use: "Await Hibernator.Hibernate(filename)"
  Return New Hibernator With {.m_fn = fn}
  End Function

  Private Sub New()
  ' Private constructor used only by the Hibernate() shared method
  End Sub

  Function GetAwaiter() As Hibernator
  Return Me
  End Function

  Public ReadOnly Property IsCompleted As Boolean
  Get
  Return False ' so that OnCompleted gets called
  End Get
  End Property

  Public Sub OnCompleted(continuation As Action) Implements INotifyCompletion.OnCompleted
  Dim ex0 As Exception = Nothing
  Try
  Dim sm = continuation.Target.GetType().GetField("m_stateMachine",
  BindingFlags.NonPublic Or BindingFlags.Instance).GetValue(continuation.Target)
  Using stream As New FileStream(m_fn, FileMode.Create, FileAccess.Write)
  Dim formatter As New Runtime.Serialization.Formatters.Binary.BinaryFormatter
formatter.Serialize(stream, sm.GetType().Assembly.FullName)
formatter.Serialize(stream, sm.GetType().FullName)
  For Each field In sm.GetType().GetFields(BindingFlags.Public Or
BindingFlags.NonPublic Or BindingFlags.Instance)
If field.Name = "$Builder" OrElse field.Name = "<>t__builder" Then Continue For
  Dim fieldValue = field.GetValue(sm)
  If fieldValue Is Nothing Then Continue For
  If field.Name.Contains("$awaiter") AndAlso
Not Object.ReferenceEquals(fieldValue, Me) Then Continue For
formatter.Serialize(stream, field.Name)
formatter.Serialize(stream, fieldValue)
  Next
  End Using
ex0 = New OperationCanceledException("Serialized to " & m_fn)
  Catch ex1 As Exception
  If File.Exists(m_fn) Then IO.File.Delete(m_fn)
ex0 = ex1
  End Try
m_ex = Runtime.ExceptionServices.ExceptionDispatchInfo.Capture(ex0)
  ' That's so we can save+rethrow the exception with its intended callstack

   continuation()
  End Sub

  Public Sub GetResult()
  If Not m_ex Is Nothing Then m_ex.Throw()
  End Sub

  '===================================================================================

  Public Shared Function [Resume](fn As String) As Task
  Using stream As New IO.FileStream(fn, FileMode.Open, FileAccess.Read)
  Dim formatter As New Runtime.Serialization.Formatters.Binary.BinaryFormatter
  Dim assemblyName = CStr(formatter.Deserialize(stream))
  Dim typeName = CStr(formatter.Deserialize(stream))
  Dim sm = TryCast(System.Activator.CreateInstance(assemblyName, typeName).Unwrap(),
IAsyncStateMachine)

  While stream.Position < stream.Length
  Dim fieldName = CStr(formatter.Deserialize(stream))
  Dim fieldValue = formatter.Deserialize(stream)
  Dim field = sm.GetType().GetField(fieldName, BindingFlags.Public Or
  BindingFlags.NonPublic Or BindingFlags.Instance)
field.SetValue(sm, fieldValue)
  End While

  Dim builder = AsyncTaskMethodBuilder.Create()
  Dim builderField = sm.GetType().GetField("$Builder", BindingFlags.Public Or
  BindingFlags.NonPublic Or BindingFlags.Instance)
  If builderField Is Nothing Then builderField =
sm.GetType().GetField("<>t__builder", BindingFlags.Public Or
BindingFlags.NonPublic Or BindingFlags.Instance)
builderField.SetValue(sm, builder)
builder.Start(sm) ' This invokes MoveNext()
  Dim builder2 = CType(builderField.GetValue(sm), AsyncTaskMethodBuilder)
  Return builder2.Task
  End Using
  End Function

    '===================================================================================

  Private Sub New(info As SerializationInfo, ctx As StreamingContext)
  ' Private constructor used only by the deserializer
  End Sub

  Public Sub GetObjectData(info As SerializationInfo,
context As StreamingContext) Implements ISerializable.GetObjectData
  ' we have nothing to serialize
  End Sub

End Class

NOTE 1: What fields need be serialized? Well, all of them, except for...

The builder field won't be serialized. It must be reconstructed. In VB it's called $Builder. In C# it's called <>t__builder. Note that both names are unutterable in user code.
Most awaiters don't need to be serialized; only the current one is needed. The VB compiler calls its awaiters $awaiterXYZ The C# compiler calls its <>XYZ$awaiterABC. Again, both names are unutterable in user code. Both compilers creates several awaiters: one awaiter of type 'object' for all reference-type awaitables, and one awaiter for each type of value-type.
Null fields don't need to be serialized, and indeed can't.

NOTE 2: Normally it's wrong for OnCompleted to invoke continuation() directly, because you end up with MoveNext invoking OnCompleted invoking MoveNext, which if it went too far would blow the stack. We should really be posting a delegate to the current synchronization context. But here the intended usage is "Await Hibernator.Hibernate(filename)" which will terminate the async method with an uncaught exception, so there's no danger.

NOTE 3: In this design of hibernator, GetResult will be called at two times:

when the user did "Await Hibernator.Hibernate()", and our OnCompleted method saved an exception into m_ex, and we always wish to throw that exception
when the user resumed the aysnc method from hibernation, and the Hibernator was deserialized with all fields null, and we don't want to throw any exception.

NOTE 4: Why does the [Resume] method retrieve the "builder" field a second time? Well, Async state machines are structures that get boxed at their first await. If they get boxed, then all structure-fields within them get boxed as well. Therefore "sm.builder" after the first MoveNext might not be the same "builder" that we created before.

Conclusions

This is a pretty sketchy implementation. It does just enough to make the sample code work. It doesn’t work with Async Subs or Async Task(Of T) Functions.

It also isn't composable: you can't write a wrapper method around a call to Hibernate(). That's because it only serializes its immediate caller.

This code is tightly coupled to the .NET4.5 compilers’ internal implementations of async methods. It relies on the fact that “Await Hibernate()” will invoke Hibernate’s OnCompleted method with a delegate whose Target object has an m_stateMachine field that points to the state machine. It relies on this state-machine having lifted local variables, and a builder of a recognizable name, and recognizable awaiter fields.

We didn’t give these things a clean public API, because we didn’t have any good use-cases for how or why people would want to serialize their async methods. If you folks come up with compelling mainstream scenarios, then we might consider maybe a Microsoft-authored NuGet package which does serialization, or exposing the necessary fields via a clean public API.

Why must async methods return Task?

Lucian Wischik, MSFT — Thu, 22 Nov 2012 01:26:00 GMT

We all know that async methods return Task or Task(Of T):

    Async Function GetNameAsync() As Task(Of String)
        Await Task.Delay(10)
        Return "ernest"
    End Function

Sometimes, advanced users ask for the ability to return different types out of an async method. That’s disallowed: it gives the error message “The Async modifier can only be used on Subs, or on Functions that return Task or Task(Of T)”.

    Async Function GetNameAsync() As ITask(Of String)
        Await Task.Delay(10)
        Return "ernest"
    End Function

In this post I’ll explain why it’s disallowed.

It wasn’t always disallowed. Our first internal prototypes of the async feature allowed you to return any suitable type from an async method, and I’ll explain how. We did some powerful things with it. But in the end there were solid language-design reasons for disallowing it, and I’ll explain those.

Motives for flexible async return types

Why would you ever want to return something other than Task(Of T) from an async method? Here are some (very legitimate) reasons.

The Task type is a reference type, which means I have to pay the cost of at least one heap allocation from every async method. I want to return a structure type to avoid that cost, and reduce the need for garbage collection.
Let’s call such a putative return type “TaskLite(Of T)”
I want to return IAsyncOperation(Of T) directly from my async methods.
This request comes from people writing WinRT components. It’s a special case of the next request...
My library/framework/app uses its own different Task-like type because Task wasn’t suitable for whatever reason. I want to be able to return my own Task-like type from async methods.
I want an interface type ITask(Of T) instead of Task(Of T). That would be a better object-oriented design. ITask(Of T) wouldn’t have any of the blocking synchronous members like Wait() or Result. It would also allow me to provide my own implementations of ITask(Of T).

Early prototype of async had flexible return types

Our first internal prototype of async allowed for arbitrary return types. Here’s how it worked:

    Async Function GetNameAsync() As Taskoid
        Await Task.Delay(10)
        Return "ernest"
    End Function

(1) The compiler needs to know, from the return type “Taskoid”, how to get a builder for it. We wanted to allow extension methods, so we did it with a “fake static method” like this:

Dim builder = CType(Nothing, Taskoid).GetBuilder()

(2) The compiler turns each async method into a compiler-generated stub, plus a compiler-generated state-machine class. Here’s the stub. Observe that it’s the builder who’s completely responsible for figuring out how to produce the ultimate task-like thing that’s returned from the method:

    Function GetNameAsync() As Taskoid
        Dim sm As New GetNameAsync$SM
        sm.builder = CType(Nothing, Taskoid).GetBuilder()
        Return sm.builder.Build(sm)
    End Function

(3) The state machine has a MoveNext method that embodies the user’s original async method body. The way it implements “Await” or “Return” is simply by asking the builder to handle them. Once again, it’s the builder who’s completely responsible for figuring out how the flow of execution around these operators:

    Public Sub MoveNext() Implements IAsyncStateMachine.MoveNext
        Try
            SelectCase iState
                Case 0
                    ' await Task.Delay(10)
                    Awaiter1 = Task.Delay(10).GetAwaiter()
                    iState = 1 : builder.Await(Awaiter1) : Return
                Case 1
                    ' return "ernest"
                    iState = 2 : builder.Return("ernest") : Return
            End Select
        Catch ex As Exception
            builder.Throw(ex)
        End Try
    End Sub

(4) Actually, we also asked the builder to handle the “Yield” statement as well. I’m not going to write out the details of how we implemented various different builders – they consist of small clever tricks within a sea of boilerplate code. What was exciting was all the powerful things we could do with them:

    ' Iterators
    Async Function GetNodes() As IEnumerable(Of Integer)

    ' Async iterators
    Async Function GetNodesAsync() As IAsyncEnumerable(Of Integer)

    ' WinRT operations, using Yield to report progress
    Async Function GetNodesAsync() As IAsyncOperationWithProgress(Of String, Integer)

    ' RX, using multiple Returns to produce events
    Async Function WatchEvents() As IObservable(Of String)

    ' Async methods that implicitly start on a parallel thread
    Async Function ThreadpoolWorkAsync() As ParallelTask

Why it didn’t work: type inference

The early prototype of async allowed flexible builders, but it didn’t support async lambdas nor generic type inference. When we started to add lambdas and type inference, we discovered it was incompatible with the flexible builders.

In the choice between “generics + lambdas + inferences” versus “flexible return types”, we picked the first, and it was definitely the right choice.

Why am I so adamant that it was the right choice? Well, think of things like Task.Run or Task.WhenAll. We pass async lambdas around all over the place. They’re essential. They’re far more common than flexible return types would have been.

In what way are the two options incompatible? Well, let’s start with a simple example:

    Sub f(Of T)(lambda As Func(Of IAsyncOperation(Of T)))

    f(AsyncFunction()
          Return 5
      End Function)

Here you’d expect it to have picked T=Integer, and to have figured out that it should have called “IAsyncOperation(Of Integer).GetBuilder()”. How would it have done that? Let’s spell out the compiler’s thoughts explicitly:

There is a return statement with operand “5”
Therefore, the call to builder.SetResult(5) must compile
<magic>
Therefore, the builder must have had type IAsyncOperationBuilder(Of Integer)
<magic>
Therefore, the stub method must have invoked IAsyncOperation(Of Integer).GetBuilder()
Therefore the lambda must have had type Func(Of IAsyncOperation(Of Integer))
Therefore, T=Integer

Here’s a more difficult example.

    Sub h(Of T)(lambda As Func(Of Unusual(Of T)))

    Class Unusual(Of T) : Implements IAsyncOperation(Of IEnumerable(Of T))

    h(Async Function()
          Return 5
      End Function)

I don’t know what I’d expect it to pick for T in this case. In general it would depend entirely on the vagaries of overload resolution, i.e. what overloads of builder.Return() and builder.Yield() there happen to be, and which ones happen to work with which arguments. It’s entirely possible to have builders where “T” can’t be inferred at all just from those calls to Return/Yield.

There are only a few possible solutions for the “magic” steps:

We could enforce a convention/constraint: every return type from an async method must be a generic type with exactly one generic type parameter, and the corresponding builder’s Return() method must not be overloaded, and it must take exactly one parameter, and the parameter’s type must be the generic type parameter. Verdict: this wouldn’t generalize to Yield, and it wouldn’t work with lots of interesting return types, and this kind of complex and arbitrary restriction is unprecedented and ugly.
We could add an arbitrary “constraint-solving” component to VB and C#. Whenever it encounters a type inference situation, it assembles all the knowledge it has about all overloads and return statements and yield statements, and plugs them into the solver, and sees what answers pop out. Verdict: the behavior of the compiler would now be unpredictable to everyone. It’s true that constraint-solvers do exist in other languages, e.g. F# uses the Hindley-Milner solver for type inference. But there’s no precedent for the complexity of the constraint-solver we’d need here.
We could decide that flexible return types are allowed only for top-level methods or for lambas where their return-types are explicitly declared. Verdict: this would be ugly. If the language has a feature, you should be able to combine it with other features. If we consider how we use lambdas today, we use type inference in almost all of them.

Workarounds for the lack of flexible types

I started this article by outlining some of the scenarios where people want flexible return types. Now that we know they’re impossible, let’s consider the workarounds.

SCENARIO 1. I want to return a value-type from my async methods, to avoid the cost of a heap-allocated reference type. Even if the data is available immediately and I return Task.FromResult(5), say, that still incurs the cost of allocating Task on the heap.

If we’d anticipated the “async” feature five years ago we might have made the “Task” type a structure to start with. But it’s too late now. The best you can do is make your own Task-like type that’s a structure, and return it from a wrapper method. Here below is an example.

    Function GetNameLiteAsync() As TaskLite(Of String)
        If Not m_cachedName Is Nothing Then Return New TaskLite(Of String)(m_cachedName)
        Return New TaskLite(Of String)(GetNameInternalAsync())
    End Function

    Structure TaskLite(Of T) : Implements Runtime.CompilerServices.INotifyCompletion
        Private m_SyncValue As T
        Private m_AsyncValue As Task(Of T)

        Public Sub New(Value As T)
            m_SyncValue = Value
        End Sub

        Public Sub New(AsyncValue As Task(Of T))
            m_AsyncValue = AsyncValue
        End Sub

        Public Function GetAwaiter() As TaskLite(Of T)
            Return Me
        End Function

        Public ReadOnly Property IsCompleted As Boolean
            Get
                Return (m_AsyncValue Is Nothing) OrElse (m_AsyncValue.IsCompleted)
            End Get
        End Property

        Public Sub OnCompleted(continuation As Action) Implements INotifyCompletion.OnCompleted
            m_AsyncValue.GetAwaiter().OnCompleted(continuation)
        End Sub

        Public Function GetResult() As T
            If m_AsyncValue Is Nothing Then Return m_SyncValue
            Return m_AsyncValue.GetAwaiter().GetResult()
        End Function
    End Structure

We might have created TaskLite(Of T) ourselves in the .NET framework, and added it into the compiler, and baked in support for async methods to return either Task(Of T) or TaskLite(Of T). That would have worked, but would have been ugly framework design: every single user would have to worry all of the time about whether to return TaskLite or Task, when in reality it’s not even worth worrying about for the majority of users.

SCENARIOS 2 and 3: I want my async method to return IAsyncOperation(Of T), or some other task-like type, because my existing framework is built around my own task-like type.

The Task type is pluripotent. You can build any other task-like thing out of it. For instance we provide an extension method to turn a Task(Of T) into an IAsyncOperation(Of T). That’s how it is in general: whenever you want to return a different task-like thing, you have to do it via a wrapper method.

    Function GetNameRTAsync() As Windows.Foundation.IAsyncOperation(Of String)
        Dim t As Task(Of String) = GetNameInternalAsync()
        Return System.WindowsRuntimeSystemExtensions.AsAsyncOperation(t)
    End Function

SCENARIO 4: I want my async method to return an ITask(Of T).

This suggested scenario is a positively bad idea. We already as of .NET4 had a single canonical Task type. All the combinators like Task.WhenAll and Task.WhenAny operate on it, as do the Reactive Extensions, and Dataflow, and other libraries.

If we introduced ITask(Of T) as well, then suddenly all those libraries would become useless. They’d need to be rewritten to take ITask arguments rather than Task arguments. And for every async method or library-routine that you write, you’d have to decide whether it should return Task or ITask. That’s a detail that’s not worth worrying about for most programmers, and shouldn’t be forced upon them.

Also, if the compiler compiles an async method whose declared return type was ITask(Of T), it would still have to pick a concrete implementation for that return type. (Likewise, when the compiler compiles an iterator method with return type IEnumerable(Of T), it needs to pick a concrete implementation of it). So allowing an async return method to return ITask(Of T) would never change the underlying fact that it returns an object whose runtime type is Task(Of T).

    Function GetNameIAsync() As ITask(OfString)
        Dim t As Task(Of String) = GetNameInternalAsync()
        Return New TaskWrapper(Of String)(t)
    End Function

It was fascinating to work together with the other members of the VB and C# language design teams, to design the async language feature. We didn’t make any decisions lightly. The decision in this article, about flexible return types, took months of discussion and prototypes until we reached an answer. I’m confident we made the right decision in this case. Next month I’ll discuss why we allowed void-returning async methods. It was altogether more controversial...

When is a non-breaking language fix, breaking?

Lucian Wischik, MSFT — Thu, 19 Jul 2012 22:08:00 GMT

In VS2012 we fixed some method-resolution problems from VS2010. These were method-calls that failed to compile in VS2010, but now compile cleanly in VS2012.

VB:

Dim numbers As IEnumerable(Of Integer) = {1, 2, 3}
Sub f(Of T)(x As T)
Sub f(Of T)(x As IEnumerable(Of T))

f(numbers) ' Should obviously pick the second overload, since it’s more specific (but VS2010 didn't)

C#:

static string mg(this string x) { return ""; }
static void f<T>(System.Func<T> lambda) { }

f("".mg); // Should be able to infer that T=string (but VS2010 couldn't)

The change in VB was a new rule we added to the language spec and compiler, needed for Task.Run(...) to be usable. The change in C# was a bugfix to make the compiler behave how the language spec said it should.

You'd think that these improvements would be safe... Their only effect is that code which used to give compile-time errors in VS2010 will now compile cleanly in VS2012. No one should be adversely impacted, right?

Wrong.

Whenever we improve method-resolution, there exists the theoretical possibility of code that uses overload-resolution and lambdas in a particular way, and which used to work (more by accident than design), and which will now fail to compile. Credit goes to Vladimir Reshetnikov, who pointed out this class of problems a few years ago, and whom we hired to work in the VB/C# language QA team. Although we're normally loathe to take any breaking changes at all, this particular class of breaks seems so convoluted and obscure that we're not as concerned about it.

Module Module1
Dim numbers As IEnumerable ( Of Integer ) = {1, 2, 3}

Sub Main()
   ' VS2010 used to give an ambiguity error for this call.
   ' We fixed the spec+compiler by adding a new tie-breaker rule.
   ' VS2012 now picks the second "f" due to improved overload-resolution rules...
   f(numbers)

   ' But confusingly, even though the improvement meant that normal code
   ' now works which used to give an ambiguity error,
   ' there are also some subtle cases involving lambdas and
   ' "overload-resolution-within-overload-resolution" which used to compile
   ' but which now give an ambiguity error.

   ' VS2010 used to pick the second "g" and the first "f" for this call:
   ' VS2012 now gives an ambiguity error...
   g( Sub (x) f(x))

   ' Explanation: the compiler looks at the two candidates of "g".
   ' With the first candidate, "Sub(x) f(x)" would have x typed as IEnumerable(Of Integer).
   ' In VS2010 this would have meant the call to f(x)
   ' was ambiguous, and so the lambda couldn't be passed to the first candidate for "g",
   ' and so VS2010 rejected the first candidate for "g", and opted for the second (which works).
   ' In VS2012 this call to f(x) is not ambiguous, and so the lambda could happily be
   ' pased to the first candidate for "g" as well as to the second.
   ' That's why VS2012 reports an error on which overload of "g" to pick.
   '
   ' This affects cases when you pass a lambda as an argument to a method
   ' with multiple overloads, and when that lambda itself contains a call with
   ' multiple overloads, and when the interplay of outer and inner overloads
   ' happen to hit one of the cases of improved overload-resolution.
End Sub

Sub f( Of T )(x As T )
End Sub
Sub f( Of T )(x As IEnumerable ( Of T ))
End Sub

Sub g(lambda As Action ( Of IEnumerable ( Of Integer )))
   lambda(numbers)
End Sub
Sub g(lambda As Action ( Of Integer ))
   lambda(1)
End Sub

End Module

static class Program {

static void Main() {
   // VS2010 used to give a compile error because, due to a extension-method bug, it couldn't infer T
   // We fixed the compiler bug.
   // VS2012 now compiles this correctly, inferring T=string.
   f( "" .mg);

   // VS2010 used to produce an invalid executable for the following code, due to the same bug.
   // Our same bugfix made this work.
   // VS2012 now compiles this correctly, producing valid EXE that calls the second overload of "g".
   g( "" .mg);

   // But confusingly, even though the fix meant that normal code
   // now works which used to give a compiler error (or invalid executable),
   // there are also some subtle cases involving lambdas and
   // "overload-resolution-within-overload-resolution" which used to compile
   // but which now give an ambiguity error.

   // VS2010 used to pick the second overload of "h", passing the instance method C.mg to f.
   // VS2012 now gives an ambiguity error
   h(x => f(x.mg));

   // Explanation: The compiler looks at the two candidates of "h".
   // With the first candidate, "x => f(x.mg)" would have x typed as string.
   // In VS2010 this would run foul of the bug, and produced an error, and so VS2010
   // rejected the first candidate for "h", and opted for the second one (which worked).
   // In VS2012 this call to "f(x.mg)" no longer suffers from the bug, and so the
   // two candidates for "h" work fine.
   // That's why VS2012 reports an error for which overload of "h" to pick.

   // This affects cases where you pass a lambda as an argument to a method with
   // multiple overloads, and when that lambda itself contains a call that experienced
   // the bugfix.
}

static string mg( this string x) { return "" ; }
static void f<T>(System. Func <T> lambda) { }

static void g(System. Func < object > lambda) { }
static void g<T>(System. Func <T> lambda) { }

static void h(System. Action < string > action) { }
static void h(System. Action < C > action) { }
}

class C { public string mg() { return "" ; } }

What would you like to know about "how we do language design for VB and C#" ?

Lucian Wischik, MSFT — Thu, 24 May 2012 17:40:49 GMT

On June 6th I'll be speaking at the Norway Developer Conference in Oslo, on the subject "How we do language design at Microsoft (VB/C#)". It will cover topics like how we designed Async, how we run our week-to-week language design process, where we get ideas from, how we recognize and respond to user requests and market needs, and some of our attitudes towards what a language should be.

Afterwards, I'll put the slides up here on this blog.

QUESTION: Are there any areas that you'd like to see discussed? Any questions you'd like answered? If you post questions or topics here as comments on this blog, then I'll take two weeks to think them over, and I'll incorporate the answers into my slides.

Disclaimer: The talk and slides will come solely from my personal experience from the past few years in my job as language PM -- learning the ropes, working with the other language designers, running the VB language design meetings, and working as a VB/C# compiler developer. They certainly won't be Official Microsoft Statements on the subject "How Language Design Must Be Done".

Async Targeting Pack

Lucian Wischik, MSFT — Tue, 24 Apr 2012 18:16:00 GMT

[UPDATE: The async targeting pack has been renamed "Microsoft.Bcl.Async". It now supports async for portable libraries]

We’ve just released an "Async Targeting Pack” [update: replaced by Microsoft.Bcl.Async] via NuGet. It lets you use the async feature of VS11 for applications that target Silverlight5 or .NET4. (You don't need it if you're using VS11 to target .NET45).

We expect all users of VS2010+AsyncCTP to upgrade to VS11. That’s because VS11 implements the async feature fully, and is supported. By contrast the AsyncCTP has many known bugs in its implementation of async, is not supported, and is difficult to install.

We intend not to release any further versions of the AsyncCTP. For the sake of Azure and Windows Phone, however, we are keeping the AsyncCTP for VS2010 online for now [link]. It remains unsupported. Update: the new Microsoft.Bcl.Async works fine for Windows Phone.

I want to put in a note of appreciation to David Grunwald, Omer Mor and Alex Davies who released AsyncBridge while we were putting together the Async Targeting Pack. Great work!

Some reminiscences of the Async CTP

Now that the Async CTP can (almost) finally be laid to rest, here's a logo from it:

Mads Torgersen has two whiteboards in his office that hang from a picture-rail. Right at the very beginning of our first discussions on async, he wrote the word "await" in giant filled-in lettering on the boards and pushed the two halves apart to opposite ends of the wall. I used that to create the logo.

My favorite real-life Async anecdote comes from covert observations of Anders Heijlsberg in the Microsoft cafeteria checkout line. He ordereds burritos more days than not... (1) He walks to the burrito counter, makes an order, and gets a receipt. (2) He queues up at the cash register to pay for the receipt. (3) Then he goes back to the burrito folks who have by now finished preparing his meal. If that's not async in action, I don't know what is!

Dim receipt As Task(Of Burrito) = OrderBurritoAsync()
Pay()
Await receipt

On another occasion, Mads was giving a talk about async to some group within Microsoft. He'd missed breakfast and was eating a banana at the podium, waving it about animatedly as he discussed async control flow. We called it his "async banana". It's an old Microsoft tradition to share chocolates, cakes and cookies with colleagues, so that evening I baked some "Async banana bread" and brought it into the office the next day. First workitem was a bug report from our QA team that Finally bodies were sometimes being executed twice inside async methods. So that evening I baked a second identical "Async banana bread" and brought it into the office as a dupe. (The bugfix on this occasion was so subtle that it left even Jon Skeet scratching his head for weeks!)

We just got word that the AsyncCTP has been downloaded over 50,000 times. That's a huge number of downloads for an experimental preview of a language feature. We're hugely glad to have had people use it, and give their feedback, and shape the final product. But it was just an experimental technology preview, and we're glad to finally have VS11 and the Async Targeting Pack as its replacement.

Talk: Async Part 1 - the message-loop, and the Task type

Lucian Wischik, MSFT — Thu, 29 Mar 2012 00:19:00 GMT

Talk: Async Part 1 - the message-loop, and the Task type

After giving lots of training and talks over the past two years, this is my most recent take on how to explain Async.

The message-loop is fundamental. You can't understand async unless you understand the message-loop. Once you understand how async uses the message-loop, you'll understand everything else.
You can use async to write the sort of distributed algorithms - with cancellation and retries - that the new generation of software needs, but that were too hard to write before.
The Task type and its various library functions like Task.WhenAll are compositional: they let you build your async algorithms out of smaller pieces.

This talk is called "Async Part 1". I've also given a sequel, Async Part 2, that goes into more advanced uses of async.

I gave this talk in C# at NDC2012 in Oslo, Norway on 2012.06.06

Watch video online
Download slides.pptx [2.7mb]
Download script.docx [28k]
Download source_code.zip [640k, requires VS2012 and Win8]

I gave this talk in VB at DevConnections2012 in Las Vegas, USA on 2012.03.26

Download video.mp4 [120mb, and I'm afraid the sound quality at the start is pretty bad]
Download slides.pptx [3.1mb]
Download script.docx [30k]
Download source_code.zip [1.4Mb, requires VS11 and Win8]

I want to call out thanks to Greg Paperin, who came up with the slide ideas for "generations of programming languages" and "message-loop animation".

Talk: What's new in VB11 (VS11 Beta)

Lucian Wischik, MSFT — Wed, 28 Mar 2012 23:21:00 GMT

Talk: What's new in VB11

This talk details some of what's new in VB11. (Just some: there were too many improvements to cover them all in just one talk).

Win8 support
VBCore (=> new platform support for VB on Phone, XNA, MicroFramework, Kinect, Surface...)
Async
CallerMemberInfo
Iterators
View Call Hierarchy
Namespace Global
AnyCpu32bitPreferred allows EnC on x64 machines
Better codespit and pretty-listing
Better performance
Better error-reporting for lambdas, and (not yet in VS11beta but will be) no more 102-error-limit for vbc/msbuild
Fixed language corners in ForEach, ReturnArrayLiterals, OptionalParameterOverloads, GenericOverloadResolution

I gave this talk at DevConnections2012 in Vegas on 2012.93.26

Watch video online
Download source_code.zip [82k, requires VS11]
Download script.docx [29k]
Download slides.pptx [3.4mb]

Similar material elsewhere
- Blog post by Jonathan Aneja - read article "What's new in VB11" on the VB Team Blog
- MSDN article "Breaking Changes in VB11"

AsyncCTP installation problems (and VS11)

Lucian Wischik, MSFT — Sun, 25 Mar 2012 17:01:00 GMT

The AsyncCTP is getting a bit long in the tooth. This post is to explain how, why, and what to do.

Common symptoms:

AsyncCTP fails to install -- i.e. it doesn't create the "Async CTP" folder inside "My Documents", and VS doesn't recognize the async keywords.
Installing VS11 causes AsyncCTP to stop working.
AsyncCTP doesn't work on Windows 8.

Explanation

The AsyncCTP is an update. It updates Visual Studio 2010 with modified versions of the VS IDE compilers that colorize the new keywords. It also updates the .NET4 framework with modified versions of the command-line csc.exe and vbc.exe compilers. But the AsyncCTP was never incorporated into Microsoft's official source code for VS2010 or .NET4, and was never meant to be -- it's just an experimental CTP.

And so, when Microsoft produces official updates to either VS or .NET4, and they touch the same files, then they tend to overwrite the AsyncCTP -- or prevent it from installing in the first place.

Also .NET45, which comes with VS11 and Win8, is an in-place update on top of .NET4, and so replaces the csc.exe and vbc.exe from the AsyncCTP with updated ones. As Stephen Toub explains, the new csc.exe and vbc.exe are incompatible with AsyncCTPLibrary.dll.

What to do

Best option is to upgrade to VS11 Beta and .NET45. It has much better async support (fewer bugs, more features, better debugging, better perf)
Failing that, if you're stuck with VS2010 + AsyncCTP, here are ideas:

If you already have VS2010 + AsyncCTP working, then don't install VS11.
If you have VS2010 SP1 but the AsyncCTP installer doesn't seem to have worked, the problem might be conflicting updates such as
KB2635973, KB2615527, KB2645410
Any update after October 11 is liable to conflict, though. Go to ControlPanel > AddRemovePrograms > WindowsUpdates and uninstall them if present.

Platforms:

Targeting .NET45 with async:
you have to use VS11 Beta, which works fine.
Targeting Silverlight5 or .NET4 with async:
you're currently stuck with VS2010 + AsyncCTP. We're exploring options here to let you use VS11 instead.
Targeting Windows Phone with async:
again you're currently stuck with VS2010 + AsyncCTP.

Other ideas:

If you can set up a VM with only VS2010 SP1 and no updates, then AsyncCTP should install fine.
Then you can copy the "Microsoft Visual Studio Async CTP" folder out of My Documents, to use on other machines.
From the same VM, you could go C:\Windows\Microsoft.Net\Framework and copy the following files. They constitute the entirety of AsyncCTP updates to the .NET4 framework. You could copy these files to a build machine, for instance, and then msbuild would use the AsyncCTP without needing to run the installer. (Caution! This will likely break regular .NET updates!)

csc.exe, vbc.exe, 1033\cscui.dll, 1033\vbc7ui.dll

It's theoretically possible to do the same trick for VS2010, copying just the files that were modified by the AsyncCTP installer. But this is error-prone and more likely to leave VS2010 in a broken state than not.

NB. The AsyncCTP installer has a problem with network drives: if your My Documents is mapped to a network drive, then it won't install.

Thoughts

We understand async has become indispensable for many of your projects. Ultimately, though, we can't support the AsyncCTP -- it's too fragile and has too many bugs. It took us an entire release cycle to fix it, and VS11/.NET45 is the result. The only long-term supportable strategy is for people to migrate over to VS11. As soon as we've figured out the migrate story, we'll let you know.

PS. If you notice other updates which conflict with AsyncCTP, please post in the comments here, and I'll update the instructions.

Async CTP v3 – installation

Lucian Wischik, MSFT — Tue, 01 Nov 2011 00:01:00 GMT

The third version of the Async CTP was released on 31st October 2011.

The reason for releasing v3 is to address installation issues -- a recent Windows Update had conflicted with the Async CTP v2 and stopped it working, or prevented its installation in the first place. The v3 release also contains one small bugfix, to stop intermittent VB IDE crashes.

This post is to help with some common installation issues.

Run Windows Update before installing Async CTP v3, and get all updates (including Optional updates) from before October 2011

*** BUT NONE that came after. Three in particular, KB2635973 and KB2615527 and KB2645410, are known to be incompatible with Async CTP v3.

After installing Async CTP v3, then you can get subsequent updates.

Q. Why do we keep releasing updates of the Async CTP? This is the third version so far:

October 2010 – original Async CTP v1
April 2011 – Async CTP v2 (SP1 Refresh), to make it compatible with SP1 of VS2010 and fix some bugs
October 2011 – Async CTP v3, to make it work with recent product updates

The reason is that Visual Studio and .NET4 keep being updated. The Async CTP is actually a mini-fork of VS2010. Therefore, each time there’s a Windows Update which includes changes to the VB or C# compilers, then the Async CTP gets out of date: the update will break the Async CTP if you have it on your machine, and it’ll block the Async CTP from being installed on top of it. So far these conflicting updates have been coming out about once every six months, and so we’ve had to re-release the Async CTP every six months.

Note: The Async CTP v3 can be installed over the top of Async CTP v2 (SP1 Refresh): you don’t need to uninstall it first.

A quick check that the Async CTP is working

If you suspect that something’s wrong with your installation of the Async CTP, then please check the version number of these four files. (Please check all four, even if you’re not doing C# or VB or Phone development).

C:\Windows\Microsoft.Net\Framework\v4.0.30319\csc.exe
C:\Windows\Microsoft.Net\Framework\v4.0.30319\vbc.exe
C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\msvbide.dll
My Documents\Microsoft Visual Studio Async CTP\Samples\AsyncCtpLibrary_Phone.dll

If you see the following version numbers, then the Async CTP Refresh has been correctly installed:

	csc.exe	vbc.exe	msvbide.dll	AsyncCTPLibrary_Phone.dll
Async CTP v3	4.0.30319.517	10.0.30319.517	4.0.40219.355	1.1.0.0

Q. How to find those version numbers? A. Right-click on each file and choose Properties; the version number is in the "Details" tab.

If the version numbers are incorrect

Here are some other version numbers you might find

	csc.exe	vbc.exe	msvbide.dll	AsyncCTPLibrary_Phone.dlll
VS2010	4.0.30319.1	10.0.30319.1	10.0.30319.1	absent
Async CTP v1	4.0.30319.326	10.0.30319.326	10.0.30319.326	absent
VS2010 SP1	4.0.30319.1	10.0.30319.225 10.0.30319.431	10.0.40219.1	absent
Windows Update or KB2468871		10.0.30319.233 10.0.30319.450
Async CTP v2 (SP1 Refresh)	4.0.30319.440	10.0.30319.440	10.0.40219.15	1.0.0.0
Async CTP v3	4.0.30319.517	10.0.30319.517	10.0.40219.355	1.1.0.0

I want to stress this: you should check the version numbers of all four files. Even if you think you're not going to use C#, say, you should still check all four files including csc.exe. Any discrepancy in any of the files will indicate your machine is in an incorrect state.

If Visual Studio crashes upon opening a VB/C# project

The issue is that VS2010 SP1 had a bug in NGEN. The Async CTP v3 requires that this bug have already been fixed – either by having Async CTP v2 already on the machine, or by running Windows Update and getting all updates including optional ones, or by manually installing KB2468871.

Alternatively, if you’ve already got Async CTP v3 on your machine, you can fix the bug manually: open a Developer Command Prompt, run the following command in it, and have a nice cup of tea and biscuit because it’s going to take up to an hour:

ngen update /force

Uninstallation

To uninstall Async CTP v3

This is easy: Start > ControlPanel > UninstallPrograms > ViewUpdates, search for all updates with “async” in their title, and uninstall all of them. You might see as many as five different entries that need to be uninstalled.

TIP: Although uninstall order doesn’t matter, it goes easier if you uninstall the “Visual Studio Async” updates first and the “.NET Framework Async” updates next. That’s because the uninstalling the latter will require a reboot.

To uninstall Async CTP v2 (SP1 Refresh)

The uninstall of Async CTP v2 has become broken due to a conflicting Windows Update. If you have v2 on your machine and want to install it, then you actually have to install v3 first.

To uninstall Async CTP v1 (pre-SP1 release)

If anyone still has Async CTP v1 from October 2010: we recommend to uninstall this prior to installing SP1.

If you still have problems

Please post a request for help in the Async Discussion Forums. We do monitor those forums frequently, and we may be able to help.

When you ask for help, PLEASE mention all four version numbers. That will help!

A day in the life of compiler bugfixing

Lucian Wischik, MSFT — Mon, 17 Oct 2011 20:31:18 GMT

The VB/C# team is hard at work on the Async feature... You might already have download the Async CTP yourself, or tried out async in the VS11 Developer Preview, and you might wonder: "The feature seems complete already -- so what are they still working on?" This post is about an example bug that I dealt with last week.

Most programmers trust their compilers. If their program's not behaving right, they think their own code is at fault. Even if they narrow it down to what looks like a compiler bug, they still believe it's their own fault -- maybe they think they misunderstand the language, or aren't using it right. It's a big expectation that we in the compiler team have to live up to.

Here's one bug I worked on last week. I discovered this bug because, as part of unrelated performance investigations, I tested out using "Await" in every possible position allowed by the the VB language.

Module Module1
    Sub Main()
        f().Wait()
    End Sub

    Async Function f() As Task
        Dim c As New C
        Dim t = Task.FromResult(c)

        For (Await t).i = 1 To 10
            Console.WriteLine(c.i)
        Next

        ' WHAT I EXPECT: It should print out the numbers 1 to 10
        ' WHAT I GET: it doesn't print out anything at all, and fails PEVerify
    End Function

    Class C
        Public i As Integer
    End Class
End Module

Loop control variables with side-effects

It looks weird to have an expression as the loop control variable. Nevertheless, as a historical legacy, VB allows it -- it stems from the days when the For loop didn't declare its own local variable and so always had to refer to some pre-existing variable. The important question to ask is when and how often this expression gets executed. Alas the VB language specification (c:\Program Files (x86)\Microsoft Visual Studio 10.0\VB\Specifications\1033) doesn't say. So let's resort to experiment, by making the expression have some observable side-effects:

    Dim c As New C
    Function test() As C
        Console.Write("* ")
        Return c
    End Function

    For test().i = 1 To 2
        Console.Write(c.i & " ")
    Next
    ' WHAT I GET: * 1 * * 2 * *


    Dim min = 1
    Dim max = 2
    For test().i = min To max
        Console.Write(c.i & " ")
    Next
    ' WHAT I GET: * * 1 * * 2 * *

This is unexpected! It's unexpected that the compiler executes the expression so many times, and unexpected that the exact number of times varies depending on whether the loop bounds were constants or not.

Looking at the compiler implementation, what it's doing is at the start of each iteration is it assigns to the loop control variable and then reads from the loop control variable to check whether to do another iteration. Both of these tasks involve evaluating the expression. But in the first code snippet, with constants for minimum and maximum, it knows it can skip that check the first time around.

Another strange thing is that the original code used to work fine in the Async CTP, but stopped working in the VS11 Developer Preview. The reason is that the CTP played fast-and-loose with evaluation order. It basically factored out all "Await" expressions from a statement, did them all and assigned the results to temporary variables, and then performed the rest of the statement. This was a quick-and-dirty hack to get the CTP out as soon as possible. But it results in the wrong evaluation order -- for instance, "Console.WriteLine(f() & Await g())" would evaluate g() first, then Await it, then evaluate f().

The VS11 Developer Preview got correct evaluation order in most cases, but it seems to have missed this edge case of For loops.

Fixing the bug?

It looked like it would take me about 5 days to fix this bug -- which is very long, considering that I'd like to be fixing 2 bugs a day. It would take this long because the loop comparison logic is all in the codegen phase of the compiler, well after the "async transformation". It hardly seemed worth the effort -- no one in their right mind would use this strange corner of the compiler. And VB would probably have been a nicer language if it didn't allow any side effects in its loop control variable expressions.

We can't just leave the bug unfixed. We'll have to add a new paragraph to the language spec and a new error to the compiler to say "You can't use Await in a loop control expression".

The decision to add this new error felt like an easy decision. There was no need to bring it to the "VB-Insiders", a set of elite users under Non-Disclosure Agreements with whom we discuss ideas. There was no need even to bring it to a regular VB Language Design Meeting. My call (as the VB Language Lead) was that it was sufficient just to send an email around the compiler team. Everyone agreed to just add the error.

The next step was to finalize the wording of the error message with the User Experience team, who are also in charge of writing MSDN documentation about the language. They have to pick a wording that's similar to the existing error messages, and one that can be translated into different world languages. They settled on this:

BC37060: 'Await' cannot be used in a loop control variable expression

After this, the VB IDE team asked: should we make a "quick-fix" for this error, i.e. a one-click way to automatically fix the user's code if they wrote this? The answer was "definitely not".

There we have it. Bug fixed in half a day, on schedule.

Talk: What's new in VB10 (VS2010)

Lucian Wischik, MSFT — Wed, 25 May 2011 16:13:00 GMT

Talk: What's new in VB10 (VS2010)

This talk details the new VB language features that arrived in VS2010 --

Implicit Line Continuations - http://www.unemployedunderscores.com/
Array Literals
Collection Initializers
Auto-implemented Properties
Multi-line Sub lambdas (which I used for Silverlight async)
XML-literals, XML-to-schema, LINQ [actually this was already in VS2008, but it's so good I repeated it]
Silverlight interop; responding to events
DLR silverlight interop; writing to HTML DOM
No-PIA - embedding Primary Interop Assemblies for Office interop
Co-evolution

I gave this talk at PDC09 in L.A. on 2009.11.18
- I'd just bought an old used convertible, spent two months restoring it, and for its first trip I drove down from Seattle to LA on the coastal road. This was my first ever road trip and it was fantastic! Neal Gafter put me up one night at his place San Jose, but apart from that I slept rough in a pile of leaves by the roadside each night and cooked dinner in a billy-can. I got to PDC on the morning the conference started, completely dishevelled.
- Watch the talk online
- Download source_code.zip [95k, requires VS2010]
- Download script.txt [8k]
- Download slides.ppt [1.8mb] - actually these are completely useless: it was a code-only talk.
Similar material elsewhere
- Interview with me and Lisa Feigenbaum - watch the interview "What's new in VB10" online on Channel9
- Article by Jonathan Aneja - read article "What's new in VB10" online at MSDN

Talk: How to write an [Async] connected app for Windows Phone 7.1

Lucian Wischik, MSFT — Fri, 20 May 2011 22:04:00 GMT

Talk: How to write an [Async] connected app for Windows Phone 7.1

This talk is all about writing a connected app for the Windows Phone. The talk covered:

(1) Azure/WP7 toolkit for the server-side of the phone app. Most good phone apps will need some kind of server-side component, be it for live tile updates or just to form a social community.
(2) XNA for the phone app itself (a Breakout clone) using the new Mango (7.1) phone tools,
(3) Async language feature which makes it much easier to program how the app communicates with the server -- in this case to download high-score table.

The material here is in VB, but it all works equally well with C#.

I gave a VB-specific version of this talk at TechEd 2010 in Atlanta, Georgia on 2011.06.18
- Download slides.pptx [4.8mb]
- Download script.docx [27k]
- Download source_code.zip [115k], requires VS2010+SP1+WindowsPhone7.1+AsyncCTP
- Session page online. Alas they haven't put up a video of it yet.
- To encourage people to give feedback, I donate to charity for each session eval submitted. In all 22 were submitted and so on May 24th the charity "Doctors Without Borders" got $220, half from me, half from Microsoft. Thank you Atlanta!
Sri Narayanan and I gave a C#-and-VB version of this talk at VSLive2011 in Vegas on 2011.04.19
- The talk covered exactly the same material as above.
- The audience in Vegas filled out 14 evals and so on May 20th 2011 the charity "Doctors Without Borders" got $140, half from me, half from Microsoft. That's the kind of win I like to see at Vegas!

Talk: The [Async] Future of VB and C#

Lucian Wischik, MSFT — Fri, 20 May 2011 18:29:00 GMT

Talk: The [Async] Future of VB and C#

I gave a C#-specific version of this talk at DevConnections in Las Vegas, USA on 2011.11.01

This talk is mainly about Async and goes into considerable technical depth. It also covers the new CallerInfo feature, and includes a Roslyn demo.
Download slides.pptx [3mb]
Download script.docx [29k]
Download source_code.zip [42k, requires VS2010 SP1 and Async CTP v3 and Phone Developer Tools]
To encourage people to give feedback, I donate to charity for each session eval submitted. I'm looking forward to see how many evals the DevConnections audience hand in.

I gave this talk at TechDays11 in Basel, Switzerland on 2011.05.02.
- This talk is mainly about Async, with a little about Roslyn. At PDC 2010, Anders Hejlsberg delivered talk FT09 on the same topics
- Watch the talk online
- Download slides.pptx [6.3mb]
- Download script.docx [22k]
- Download source_code.zip [16k, requires VS2010 SP1 and Async CTP]
- The audience at Basel filled out 89 evals and so on May 20th 2011 the charity "Medecins Sans Frontiers" got $890 half from me and half from Microsoft. Thank you, Basel!
I gave a VB-specific version of this talk at TechEd Europe 2010 in Berlin, Germany on 2010.11.11
- This VB-specific talk covered Iterators in VB, and VB Support for Windows Phone 7, as well as Async above. (The presentation of Async above was better than what I did here...)
- Watch the talk online
- Download slides.pptx [8mb]
- Download source_code.zip for iterators [3k, requires VS2010 SP1 and Async CTP]
- The audience in Berlin filled out 29 evals and so on on November 11th 2010 the charity "Medecins Sans Frontiers" got $290 half from me and half from Microsoft. Thank you, Berlin!

[placeholder] How to write a connected VB phone app

Lucian Wischik, MSFT — Wed, 18 May 2011 12:20:00 GMT

This blog article has been moved and enhanced:

http://blogs.msdn.com/b/lucian/archive/2011/05/20/talk-how-to-write-an-async-connected-app-for-windows-phone-7.aspx

Async CTP Refresh - mitigation strategies

Lucian Wischik, MSFT — Mon, 18 Apr 2011 14:13:00 GMT

Async CTP Refresh - mitigations when using the CTP

The preceding four blog posts have outlined the known and unknown bug areas in the CTP. Now it's time to set out some concrete mitigation strategies.

Use F# instead

F# already shipped in VS2010 with Async support. It is tried, tested, high-quality, supported, and flexible. With F# you can go ahead writing Async code without even worrying about the CTP. F# is also a great language to learn in its own right.

Run PEVerify.exe and unit tests

Running PEVerify has caught about half of the codegen bugs we've found. You should definitely use it too.

Hint: quickly launch the "VS Command Prompt (2010)" from the start menu by typing the word "Command" in the search box and picking it from the results.

And as always, run unit tests. Most of the codegen bugs show up quite clearly even with superficial unit testing. The key bugs we found which didn't (especially the race-condition bugs) have been fixed in the CTP Refresh.

Be careful of synchronization context in async Windows Phone apps

There's a known issue on Windows Phone 7 such that any Await statements will always resume on the UI thread - even if you performed the Await on a background thread. In the CTP, the "AsyncBackgroundThread" sample shows a work-around for this issue.

Avoid complicated nested awaits

A "nested await" is an await expression that occurs as a sub-expression of another expression. A good rule of thumb is this: "If I couldn't easily have rewritten my statement by pulling out all awaits onto the preceding line, then the Async CTP might not have done it right either." For instance

// OK: SIMPLE OPERATORS IN VB+C#:
var s = await f() && await g();
// is probably safe (albeit wrong order), since it can be trusted to behave like this:
var $temp1 = await f();
var $temp2 = await g();
var s = $temp1 && $temp2;

' OK: USING CLAUSES IN VB+C#:
Using x = Await g() : ... : End Using
' is probably safe, since it can be trusted to behave like this:
Dim $temp1 = Await g()
Using x = $temp1 : ... : End Using

' USE WITH CARE, AND VERIFY+TEST: LOOPS IN VB:
While Await g() : ... : End While
' Note that it must have been more complicated than just a simple rewrite:
Dim $temp = Await g()
While temp : ... : End While

// DANGER: OTHER COMPLICATED IDIOMS IN VB+C#:
f(await g()).StructField.x = f(await g()).m(ref y, await t);
// Distrust this kind of code!
// * Avoid "await" on the left hand side of an assignment
// * Avoid combining awaits with ByRef arguments
// * Avoid invoking methods on expressions involving awaits
// * Avoid "await" inside array or object initializers

Miscellaneous

Be careful of For loops in VB, and also Select Case statements. Unit-test them thoroughly and PEVerify.

In particular, non-constant "Steps" in For statements don't work. Thanks to MattMc3 for reporting this.