Async CTP Refresh - Design Changes

The big news about the Async CTP Refresh is that it enabled development on SP1 and for Windows Phone 7, and came with a new EULA. But there were also a few design changes...

 

Async is like the zombie virus

I've come to believe that async will be like the "zombie virus" - once it bites one part of your program, you'll be inclined make more of your program async just in case an operation takes time. But normally your async methods will complete immediately -- and we've made some design changes to help the performance of this "fast path".

 

Design change: new "await" pattern for greater efficiency

In the Async CTP Refresh, we changed the "await" pattern to make the fast path more efficient. This will affect anyone who made their own types awaitable. The following code is in VB, but it applies equally to C#.

' What pattern does the compiler use to implement an await expression?
Dim i = Await e

' Old pattern, in first Async CTP

Dim
$temp = e.GetAwaiter()
SAVE_STATE()
If $temp.BeginAwait(AddressOf cont) Then
    Return
End If
cont:
RESTORE_STATE()
Dim i = $temp.EndAwait()

' New pattern, in Async CTP Refresh
 
Dim $temp = e.GetAwaiter()
If Not $temp.IsCompleted Then
    SAVE_STATE()
    $temp.OnCompleted(AddressOf cont)
    Return
    cont:
    RESTORE_STATE()
End If
Dim i = $temp.GetResult()
i = Nothing

In the old pattern, it incurred the cost of SAVE_STATE() even if the task had already completed. We couldn't put SAVE_STATE() inside the "Then" clause, because of the off chance that cont might be executed even before the "Then" clause had executed. In the new pattern we separated it out, so SAVE_STATE() and RESTORE_STATE() are never even executed in the "fast path".

In the old pattern, it also incurred the cost of constructing a delegate (which I've written "Addressof <label>" in the pseudo-code above), even if it wasn't needed. The new pattern will allow us to allocate that delegate lazily, potentially saving a heap allocation in the fast path. However, this is just potential for the future: we don't actually take advantage of it.

In the old pattern, it also left the "awaiter" field hanging around. In the new pattern, we null it out -- the C# equivalent is "default(T)". This is so that the awaiter does not hold onto references for longer than necessary, which could harm garbage collection.

GUIDANCE

 

If you are concerned about garbage collection in a long-running async method, you can “null out” your local variables. This is the same as what you can currently do in long-running non-async methods.

 

Taking advantage of async "fast path"

How might you take advantage of the async fast path? Here's some sample code, this time in C#. It uses the new method TaskEx.FromResult<T>() which was added in the CTP Refresh.

// This will use the "fast path" in the common case where the

// data is already available locally. It will avoid heap allocations

// in that case.

Database db;

while (await db.MoveNextAsync())

{

    Console.WriteLine(db.Current);

}

Thanks to the change, this "while" loop incurs only two small method calls (and no heap allocations) beyond what would be needed in a non-async alternative. The performance boost comes down to this implementation:

 

class Database

{

    private static Task<bool> trueTask = TaskEx.FromResult(true);

    private string[] currentBuffer = null;

    private int currentIndex = -1;

 

    public string Current {get {return currentBuffer[currentIndex];}}

 

    public Task<bool> MoveNextAsync()

    {

        if (currentBuffer != null && currentIndex < currentBuffer.Length-1)

        {

            currentIndex++;

            return trueTask; // avoids the cost of allocating a task

        }

        return MoveNextAsyncInternal();

    }

 

    public async Task<bool> MoveNextAsyncInternal()

    {

        currentBuffer = DownloadNextChunkFromDatabase();

        if (currentBuffer == null) return false;

        currentIndex = 0;

        return true;

    }

}

 

 

 

Costs of async... where even the "fast path" doesn't help

Even if the fast paths are taken everywhere, there are still some inherent costs in async:

' Compare the cost of these two statements.

' The async one has some overhead...

Dim i1 = f1()

Dim i2 = Await f2()

 

Function f1() As Integer

    Return 1

End Function

 

Async Function f2() As Task(Of Integer)

    Return 1

End Function

  • The async call costs THREE extra heap allocations - one for the async state machine, one for its continuation delegate, and one for the resultant Task object.
  • The async call costs several extra method calls to set up those objects and those fields.
  • The async call costs additional IL instructions and an additional try/catch block within the body of the method.
  • The return statement in the async case costs an additional method call.
  • To await f2() costs two extra method calls.

 

So how is it that the previous code managed to avoid most of those costs?

while (await db.MoveNextAsync()) {

    Console.WriteLine(db.Current);

}

Here "MoveNextAsync()" was not actually an async method, i.e. it didn't use the async modifier. And it returned a pre-allocated Task object instead of allocating a new one each time. In this way it bypasses most of the extra async costs. The only extra cost it incurs over and above the synchronous case is two extra method calls.

GUIDANCE

 

Create “chunky async” APIs rather than “fine-grained async” APIs. For instance, create APIs which asynchronously retrieve a batch of rows from the database in one call, rather than just one row at a time.

 

The extra cost of async is negligible compared to the latency of network operations or UI operations. It is only ever worth thinking about in inner-loops or in server code where you’re optimizing for scalability. As always, measure performance before optimizing.

 

Don’t make your code async just for the sake of it. Only do so for a reason, e.g. to avoid blocking the UI, or because the API calls you’re making are async, or to avoid consuming too many threads.

 

As in the above case, the extra costs of consuming a “fast-path” async can be minimized in some situations. The extra costs of producing an async method cannot.

 

In case of the zombie virus, it’s best to prepare an emergency kit beforehand. Include a shovel.

 

 

 

Design change: new exception behavior for "Async Subs"

We made another change because we wanted uniformity in the behavior of exceptions in Async Subs ("void-returning asyncs" in C#).

This hopefully won't affect anyone!

GUIDANCE

 

It’s fine to use async methods that are Subs (void-returning asyncs) for top-level event handlers and the like. It’s okay for these to throw exceptions

 

It’s fine to use async methods that are Task-returning or Task<T>-returning, and have the suffix “Async”, for your normal async methods.

 

As for other acceptable uses of async methods, there are a few niche cases where it makes sense to write “fire-and-forget” Async Subs which the caller is never able to await, but these should not throw exceptions that your program is intended to handle.

For everyone who followed that guidance, the change in behavior of exception-throwing Async Subs won't have any effect. That's because the only exception-throwing Async Subs were the top-level event handlers.

 ' First Async CTP: idiosyncratic exceptions

Async Sub f()
    Throw New Exception("A")
    Await t
    Throw New Exception("B")
    Await TaskEx.Yield()
    Throw New Exception("C")
End Sub

' [A] Always thrown to the caller of f()
' [B] Might be thrown to caller of f(),
' or maybe to the caller of the
' continuation-after-t (usually the UI
' message pump), depending on whether t
' took the fast path or not
' [C] Always thrown by whoever called the
' continuation-after-Yield

 ' Async CTP Refresh: uniform exceptions

Async Sub g()
    Throw New Exception("A")
    Await t
    Throw New Exception("B")
    Await TaskEx.Yield()
    Throw New Exception("C")
End Sub

' [A] Thrown to the caller's Sync.Context
' [B] Thrown to the caller's Sync.Context
' [C] Thrown to the caller's Sync.Context

Implementation in first Async CTP:

  1. At the start of the async method, the compiler implicitly generates a call to
    System.Runtime.CompilerServices.
    VoidAsyncMethodBuilder.Create()
  2. This saves the current SynchronizationContext.Current
  3. It then calls sc.OperationStarted()

  4. When the method completes normally, the compiler implicitly generates a call to
    builder.SetCompleted().
    This calls sc.OperationCompleted()
  5. If the method completed due to an exception, the compiler implicitly generates a call to
    builder.SetCompleted().
    This calls sc.OperationCompleted().
    Next, the compiler lets the exception propagate up the callstack.

Implementation in CTP Refresh:

  1. At the start of the async method, the compiler implicitly generates a call to
    System.Runtime.CompilerServices.
    AsyncVoidMethodBuilder.Create()
  2. This saves the current
    SynchronizationContext.Current
  3. It then calls sc.OperationStarted()

  4. When the method completes normally, the compiler implicitly generates a call to
    builder.SetCompleted().
    This calls sc.OperationCompleted()
  5. If the method completed due to an exception, the compiler implicitly generates a call to
    builder.SetException(ex).
    This calls sc.OperationCompleted().
    It then does sc.Post( () => throw ex ).