We all know that async methods return Task or Task(Of T):
Async Function GetNameAsync() As Task(Of String) Await Task.Delay(10) Return "ernest" End Function
Sometimes, advanced users ask for the ability to return different types out of an async method. That’s disallowed: it gives the error message “The Async modifier can only be used on Subs, or on Functions that return Task or Task(Of T)”.
Async Function GetNameAsync() As ITask(Of String) Await Task.Delay(10) Return "ernest" End Function
In this post I’ll explain why it’s disallowed.
It wasn’t always disallowed. Our first internal prototypes of the async feature allowed you to return any suitable type from an async method, and I’ll explain how. We did some powerful things with it. But in the end there were solid language-design reasons for disallowing it, and I’ll explain those.
Why would you ever want to return something other than Task(Of T) from an async method? Here are some (very legitimate) reasons.
Our first internal prototype of async allowed for arbitrary return types. Here’s how it worked:
Async Function GetNameAsync() As Taskoid Await Task.Delay(10) Return "ernest" End Function
(1) The compiler needs to know, from the return type “Taskoid”, how to get a builder for it. We wanted to allow extension methods, so we did it with a “fake static method” like this:
Dim builder = CType(Nothing, Taskoid).GetBuilder()
(2) The compiler turns each async method into a compiler-generated stub, plus a compiler-generated state-machine class. Here’s the stub. Observe that it’s the builder who’s completely responsible for figuring out how to produce the ultimate task-like thing that’s returned from the method:
Function GetNameAsync() As Taskoid Dim sm As New GetNameAsync$SM sm.builder = CType(Nothing, Taskoid).GetBuilder() Return sm.builder.Build(sm) End Function
(3) The state machine has a MoveNext method that embodies the user’s original async method body. The way it implements “Await” or “Return” is simply by asking the builder to handle them. Once again, it’s the builder who’s completely responsible for figuring out how the flow of execution around these operators:
Public Sub MoveNext() Implements IAsyncStateMachine.MoveNext Try SelectCase iState Case 0 ' await Task.Delay(10) Awaiter1 = Task.Delay(10).GetAwaiter() iState = 1 : builder.Await(Awaiter1) : Return Case 1 ' return "ernest" iState = 2 : builder.Return("ernest") : Return End Select Catch ex As Exception builder.Throw(ex) End Try End Sub
(4) Actually, we also asked the builder to handle the “Yield” statement as well. I’m not going to write out the details of how we implemented various different builders – they consist of small clever tricks within a sea of boilerplate code. What was exciting was all the powerful things we could do with them:
' Iterators Async Function GetNodes() As IEnumerable(Of Integer) ' Async iterators Async Function GetNodesAsync() As IAsyncEnumerable(Of Integer) ' WinRT operations, using Yield to report progress Async Function GetNodesAsync() As IAsyncOperationWithProgress(Of String, Integer) ' RX, using multiple Returns to produce events Async Function WatchEvents() As IObservable(Of String) ' Async methods that implicitly start on a parallel thread Async Function ThreadpoolWorkAsync() As ParallelTask
The early prototype of async allowed flexible builders, but it didn’t support async lambdas nor generic type inference. When we started to add lambdas and type inference, we discovered it was incompatible with the flexible builders.
In the choice between “generics + lambdas + inferences” versus “flexible return types”, we picked the first, and it was definitely the right choice.
Why am I so adamant that it was the right choice? Well, think of things like Task.Run or Task.WhenAll. We pass async lambdas around all over the place. They’re essential. They’re far more common than flexible return types would have been.
In what way are the two options incompatible? Well, let’s start with a simple example:
Sub f(Of T)(lambda As Func(Of IAsyncOperation(Of T))) f(AsyncFunction() Return 5 End Function)
Here you’d expect it to have picked T=Integer, and to have figured out that it should have called “IAsyncOperation(Of Integer).GetBuilder()”. How would it have done that? Let’s spell out the compiler’s thoughts explicitly:
Here’s a more difficult example.
Sub h(Of T)(lambda As Func(Of Unusual(Of T))) Class Unusual(Of T) : Implements IAsyncOperation(Of IEnumerable(Of T)) h(Async Function() Return 5 End Function)
I don’t know what I’d expect it to pick for T in this case. In general it would depend entirely on the vagaries of overload resolution, i.e. what overloads of builder.Return() and builder.Yield() there happen to be, and which ones happen to work with which arguments. It’s entirely possible to have builders where “T” can’t be inferred at all just from those calls to Return/Yield.
There are only a few possible solutions for the “magic” steps:
I started this article by outlining some of the scenarios where people want flexible return types. Now that we know they’re impossible, let’s consider the workarounds.
SCENARIO 1. I want to return a value-type from my async methods, to avoid the cost of a heap-allocated reference type. Even if the data is available immediately and I return Task.FromResult(5), say, that still incurs the cost of allocating Task on the heap.
If we’d anticipated the “async” feature five years ago we might have made the “Task” type a structure to start with. But it’s too late now. The best you can do is make your own Task-like type that’s a structure, and return it from a wrapper method. Here below is an example.
Function GetNameLiteAsync() As TaskLite(Of String) If Not m_cachedName Is Nothing Then Return New TaskLite(Of String)(m_cachedName) Return New TaskLite(Of String)(GetNameInternalAsync()) End Function Structure TaskLite(Of T) : Implements Runtime.CompilerServices.INotifyCompletion Private m_SyncValue As T Private m_AsyncValue As Task(Of T) Public Sub New(Value As T) m_SyncValue = Value End Sub Public Sub New(AsyncValue As Task(Of T)) m_AsyncValue = AsyncValue End Sub Public Function GetAwaiter() As TaskLite(Of T) Return Me End Function Public ReadOnly Property IsCompleted As Boolean Get Return (m_AsyncValue Is Nothing) OrElse (m_AsyncValue.IsCompleted) End Get End Property Public Sub OnCompleted(continuation As Action) Implements INotifyCompletion.OnCompleted m_AsyncValue.GetAwaiter().OnCompleted(continuation) End Sub Public Function GetResult() As T If m_AsyncValue Is Nothing Then Return m_SyncValue Return m_AsyncValue.GetAwaiter().GetResult() End Function End Structure We might have created TaskLite(Of T) ourselves in the .NET framework, and added it into the compiler, and baked in support for async methods to return either Task(Of T) or TaskLite(Of T). That would have worked, but would have been ugly framework design: every single user would have to worry all of the time about whether to return TaskLite or Task, when in reality it’s not even worth worrying about for the majority of users.
SCENARIOS 2 and 3: I want my async method to return IAsyncOperation(Of T), or some other task-like type, because my existing framework is built around my own task-like type.
The Task type is pluripotent. You can build any other task-like thing out of it. For instance we provide an extension method to turn a Task(Of T) into an IAsyncOperation(Of T). That’s how it is in general: whenever you want to return a different task-like thing, you have to do it via a wrapper method.
Function GetNameRTAsync() As Windows.Foundation.IAsyncOperation(Of String) Dim t As Task(Of String) = GetNameInternalAsync() Return System.WindowsRuntimeSystemExtensions.AsAsyncOperation(t) End Function
SCENARIO 4: I want my async method to return an ITask(Of T).
This suggested scenario is a positively bad idea. We already as of .NET4 had a single canonical Task type. All the combinators like Task.WhenAll and Task.WhenAny operate on it, as do the Reactive Extensions, and Dataflow, and other libraries.
If we introduced ITask(Of T) as well, then suddenly all those libraries would become useless. They’d need to be rewritten to take ITask arguments rather than Task arguments. And for every async method or library-routine that you write, you’d have to decide whether it should return Task or ITask. That’s a detail that’s not worth worrying about for most programmers, and shouldn’t be forced upon them.
Also, if the compiler compiles an async method whose declared return type was ITask(Of T), it would still have to pick a concrete implementation for that return type. (Likewise, when the compiler compiles an iterator method with return type IEnumerable(Of T), it needs to pick a concrete implementation of it). So allowing an async return method to return ITask(Of T) would never change the underlying fact that it returns an object whose runtime type is Task(Of T).
Function GetNameIAsync() As ITask(OfString) Dim t As Task(Of String) = GetNameInternalAsync() Return New TaskWrapper(Of String)(t) End Function
It was fascinating to work together with the other members of the VB and C# language design teams, to design the async language feature. We didn’t make any decisions lightly. The decision in this article, about flexible return types, took months of discussion and prototypes until we reached an answer. I’m confident we made the right decision in this case. Next month I’ll discuss why we allowed void-returning async methods. It was altogether more controversial...