Proposal: Allow [AsyncMethodBuilder(...)] on methods

[stephentoub]

EDIT: Proposal added 11/5/2020:
https://github.com/dotnet/csharplang/blob/master/proposals/csharp-10.0/async-method-builders.md

Background

AsyncMethodBuilderAttribute can be put on a type to be used as the return type of an async method, e.g.

public struct AsyncCoolTypeMethodBuilder
{
    public static AsyncCoolTypeMethodBuilder Create();
    ...
}

[AsyncMethodBuilder(typeof(AsyncCoolTypeMethodBuilder))]
public struct CoolType { ... }

public async CoolType SomeMethodAsync() { ... } // will implicitly use AsyncCoolTypeMethodBuilder.Create()

This, however, means that:

• every use of a given return type requires the same builder.
• there's no context available to be passed to the builder.
• if you don't own the type and it's not already attributed, you can't use it as an async return type.

Proposal

Two parts:

1. Allow AsyncMethodBuilderAttribute to be applied to methods. When applied to an async method, it would be used as the builder for that method, overriding any AsyncMethodBuilderAttribute on the return type.
2. Allow the builder's Create method to have arguments. Specifically, it can have arguments that match the parameters to the method to which it's applied, including the implicit this for instance methods. The compiler will then forward the arguments to the method's invocation into the builder's Create.

Ammortized allocation-free async methods

There are four types in .NET with built-in builders: Task, ValueTask, Task<T>, and ValueTask<T>. In .NET Core, significant work has gone in to optimizing async methods with these types, and thus in the majority of uses, each async method invocation will incur at most one allocation of overhead, e.g. a synchronously completing async method returning ValueTask<T> won’t have any additional allocations, and an asynchronously completing async method returning ValueTask<T> will incur one allocation for the underlying state machine object.

However, with this feature, it would be possible to avoid even that allocation, for developers/scenarios where it really mattered. .NET Core 2.1 sees the introduction of IValueTaskSource and IValueTaskSource<T>. Previously ValueTask<T> could be constructed from a T or a Task<T>; now it can also be constructed from an IValueTaskSource<T>. That means a ValueTask<T> can be wrapped around an arbitrary backing implementation, and that backing implementation can be reused or pooled (.NET Core takes advantage of this in a variety of places, for example enabling allocation-free async sends and receives on sockets). However, there is no way currently for an async ValueTask<T> method to utilize a custom IValueTaskSource<T> to back it, because the builder can only be assigned by the developer of the ValueTask<T> type; thus it can’t be customized for other uses.

If a developer could write:

[AsyncMethodBuilder(UseMyCustomValueTaskSourceMethodBuilder)]
public async ValueTask<T> SomeMethodAsync() { … }

then the developer could write their own builder that used their own IValueTaskSource<T> under the covers, enabling them to plug in arbitrary logic and even to pool.

However, such a pool would end up being shared across all uses of SomeMethodAsync. This could be a significant scalability bottleneck. If a pool for this were being managed manually, a developer would be able to make the pool specific to a particular instance rather than shared globally. For example, WebSocket’s ValueTask<…> ReceiveAsync(…) method utilizing this feature would end up hitting a pool shared by all WebSocket instances for this particular WebSocket-derived type, but given WebSocket’s constraints that only one receive can be in flight at a time, WebSocket can have a very efficient “pool” of a single IValueTaskSource<T> that’s reused over and over. To enable that, the compiler could pass the this into UseMyCustomValueTaskSourceMethodBuilder’s Create method, e.g.

internal sealed class ManagedWebSocket : WebSocket
{
    private struct WebSocketReceiveMethodBuilder
    {
        private ManagedWebSocket _webSocket;

        public static WebSocketReceiveMethodBuilder Create(ManagedWebSocket thisRef, Memory<byte> buffer, CancellationToken cancellationToken) =>
            new WebSocketReceiveMethodBuilder { _webSocket = thisRef };

        …
    }

    private IValueTaskSource<ValueWebSocketReceiveResult> _receiveVtsSingleton;

    [AsyncMethodBuilder(typeof(WebSocketReceiveMethodBuilder)]
    public override async ValueTask<ValueWebSocketReceiveResult> ReceiveAsync(Memory<byte> buffer, CancellationToken cancellationToken) { … }
}

The ReceiveAsync implementation can then be written using awaits, and the builder can use _webSocket._receiveVtsSingleton as the cache for the single instance it creates. As is now done in .NET Core for task’s builder, it can create an IValueTaskSource<ValueWebSocketReceiveResult> that stores the state machine onto it as a strongly-typed property, avoiding a separate boxing allocation for it. Thus all state for the async operation becomes reusable across multiple ReceiveAsync invocations, resulting in amortized allocation-free calls.

Related

https://github.com/dotnet/corefx/issues/27445
dotnet/coreclr#16618
dotnet/corefx#27497

LDM Notes

• https://github.com/dotnet/csharplang/blob/main/meetings/2021/LDM-2021-03-29.md
• https://github.com/dotnet/csharplang/blob/main/meetings/2020/LDM-2020-11-11.md

[yaakov-h]

I'm not familiar with IValueTaskSource and it has very few Google search results, but how do you pool a struct? 😕

[stephentoub]

IValueTaskSource<T> is an interface, not a struct.

[benaadams]

I'm not familiar with IValueTaskSource and it has very few Google search results,

Available in .NET Core 2.1 (preview2) https://github.com/dotnet/corefx/issues/27445

[yaakov-h]

Ah, I see. You’re pooling the IValueTaskSource, not the ValueTask.

Interesting - both that feature and this proposal.

[juepiezhongren]

there should be some default builder implementation for dev easy usage.

[quinmars]

The proposal is the is the missing link, how you can let a custom (i.e. not yield using) IAsyncEnumerable implementation benefit from IValueTaskSource. I like that. What worries me a little is that you are forced to create your own builder for every class or at least for every assembly if you do not want to leak internals. Hence wouldn't it be better to explicitly name the DI property/field? So that you could use a async method builder written once by experts. Like:

class ZipAsyncEnumerator : IAsyncEnumerator
{
    IValueTaskSource _valueTaskSource;
    readonly IAsyncEnumerator _e1;
    readonly IAsyncEnumerator _e2;

    [AsyncMethodBuilder(typeof(ManualResetValueTaskSourceMethodBuilder), nameof(_valueTaskSource))]
    public ValueTask<bool> MoveNextAsync()
    {
        return await _e1.MoveNextAsync() && await _e2.MoveNextAsync();
    }

    ...
}

If the AsyncMethodBuilderAttribute wouldn't be sealed one could also define their own shortcut:

class ManualResetValueTaskAttribute : AsyncMethodBuilderAttribute
{
    public ManualResetValueTaskAttribute() : base(typeof(ManualResetValueTaskSourceMethodBuilder), nameof(_valueTaskSource)) {}
}

[mgravell]

I have a working implementation of zero-alloc (not even the state machine) value task with full support for everything that value-task does, because it is value task. If we had this, it could be applied to tons of places without an API change.

(The gotcha is you can't call GetResult() twice, which is technically a semantic change,but fine as an opt-in basis)

So: please please. This would be amazing.

Citation: https://github.com/mgravell/FunWithAwaitables

Right now you need to return a different type, but that type effectively is a ValueTask (it has a direct conversion to ValueTask-T that is free and retains semantics)

[stephentoub]

zero-alloc (not even the state machine)

Presumably ammortized, because you're pooling the objects?

I've had an implementation of that as well, for async ValueTask in general (no attribute), but in addition to the semantic change you mention (undocumented but potentially relied on), there's the perf cost of potentially fine-grained synchronization on the pool, policy around how much to store, etc. It's on my list to revisit for .NET 5.

This attribute feature would be a way to both opt-in to such behavior (as you mention) and potentially do better in specific situations (as in the example highlighted in my description).

[mgravell]

@stephentoub exactly; POC is linked above (I edited it in). Amazing speed and memory improvements. Probably going to make that a proper lib and use it in a ton of places.

[mgravell]

@stephentoub actually there's something else in the above you might like - an implementation of Yield that doesn't allocate. Not quite complete yet - I need to implement a non-generic version of the POC so that I can respect execution/sync context - right now it always uses the thread-pool

[stephentoub]

an implementation of Yield that doesn't allocate

Task.Yield you mean? In .NET Core 3.0 it doesn't allocate either, at least not in the common case.