[Proposal]: Delegate Type Arguments Improvements

[RikkiGibson]

• Speclet: inline
• Discussion: [Proposal]: Delegate Type Arguments Improvements #8674
• Prototype: In Progress in compiler and runtime

Summary

Allow a wider range of type arguments to delegates, such as the following:

void M(Action<ref int> increment) { }
void M(Func<string, out object, bool> tryParseFunc) { }
void M(Func<ref readonly int> getReadonlyInt) { }
void M(Func<int*> getPointer) { }
void M(Func<delegate*<int, void>> getFuncPointer) { }
void M(Action<TypedReference> useTypedReference) { }

delegate T MyDelegate<T>(T t);
void M(MyDelegate<ref string> myDelegate) { }

Motivation

A long-standing guideline in .NET is to use System.Action and System.Func instead of custom delegates when possible. This makes it easier for users to tell at a glance what the delegate's signature is, as well as improving convertibility between delegates created by one component and handed out to another component.

However, there have always been "holes" in System.Action/Func. One of the big ones is that only by-value parameters and returns are supported. If you want to make a delegate from a method with a 'ref' or 'out' parameter, for example, you need to define a custom delegate type for it. The same issue exists for "restricted" types such as ref structs, pointers, function pointers, TypedReference, etc.

// Can't use `Func<string, out object, bool>` here, for example.
delegate bool TryParseFunc(string input, out object value);
bool TryParse(string input, out object value) { /* ... */ }

M(TryParse);
void M(TryParseFunc func)
{
    // ...
}

It would be easier to work with delegates generally if the standard Action/Func supported a wider range of parameter and return types. We could also make it so the compiler does not synthesize delegates for lambda implicit types in many more scenarios.

It would also help lay the groundwork for further features to make using delegate types easier, such as the ability to specify delegate parameter names at the point where the delegate type is used, a la tuple field names. See related discussion.

// the following is not part of this proposal, but this proposal contributes to it:
Func<int, int, bool> compare; // old way
bool (int left, int right) compare; // new way

// we would also want to support:
Func<string, out object, bool> tryParse; // old way
bool (string input, out object parsed) tryParse; // new way

// this has a strong correspondence to the "signature" of a lambda expression with explicit return and parameter types:
var compare = bool (int left, int right) => left == right;

Detailed design

'ref' type arguments

'ref', 'in' and 'out' are permitted as "modifiers" to type arguments to delegates. For example, Action<ref int>. This is encoded using SignatureTypeCode.ByReference, and by using the InAttribute and OutAttribute as custom modifiers on the type arguments, like how such parameters are encoded on function pointers. This explicitly allows for overloading on 'ref' vs non-'ref' just as in conventional signatures.

The compiler and .NET 7 runtime both need to be modified to support this. See the prototype section for more details.

void Method(Action<int> x) { }
void Method(Action<ref int> x) { } // overloads work

The compiler will ensure that the type arguments in use are valid by checking the signature of the delegate after generic substitution. For example, we would give a compile error in the following scenario:

delegate void UseList<T>(List<T> list);
void M(UseList<ref int> useListFunc); // error: 'ref int' cannot be used for type parameter 'T' in 'UseList<T>' because 'ref int' is not a permitted type argument to 'List<T>'

It's expected that some amount of composition will still be possible:

void M(Func<Action<ref int>> makeAction) { } // ok

Other restricted type arguments

Pointers, function pointers, ref structs, and other restricted types would be generally permitted as type arguments to delegates, without the need to introduce any constraints to the delegate type parameters. The checks will occur in a similar fashion as for 'ref' type arguments.

Constraints

It seems like you could also "delay" constraint checks on usage of type parameters in delegate signatures and simply check the constraints when the delegate is used. It doesn't seem that useful, though.

interface MyInterface<T> where T : class { }
delegate void UseInterface<T>(MyInterface<T> interface); // no error?

void M1(UseInterface<string> i1) { } // ok
void M1(UseInterface<int> i1) { } // error?

It feels like this wouldn't actually present a usability benefit over requiring the declaration to specify the constraints. Let's not change the behavior here. If a delegate signature uses a type parameter, the constraints should still be checked at the declaration site of the delegate.

Why only delegates?

It's reasonable to wonder why we would only want to change the rules for delegates here, instead of allowing more interface-only types to participate, such as interfaces which don't use the type parameters in DIMs, or abstract classes which don't use the type parameters in non-abstract members. We might even wonder about whether this would be useful on some members which do have implementations.

Some of the issues with this include:

1. A "just try constructing it and see if it works" approach is much riskier. It's expected to be able to add members to classes without breaking consumers, or to add DIMs to interfaces without breaking consumers. But if these members introduced a new usage of type parameters which causes consumers that passed ref type arguments to start getting errors, that's a problem. Also, the increased degree of indirection induced by a "check the construction" approach could make it more difficult to report meaningful errors on misuse. This means the feature would need to be driven by new type parameter constraints.
2. Several new constraints would need to be introduced. ref would probably need to be a separate constraint from in which would be separate from out. Pointers (where T : pointer?) have different rules for their use than ref structs (where T : ref struct) which have different rules for their use than restricted types. out in particular is something that's really difficult to imagine generalizing in a useful way for members which have implementations, or even completely abstract types. (is there any interface you can imagine in .NET today which would have a sensible usage if it were given an out type argument?)
3. It's not clear what it would mean to use a type parameter with 'ref' or similar constraints in an implementation. It feels like the basic language rules which are required to make full and sensible use of a ref start to crumble.

void M<T>(T input) where T : ref
{
    // What does this mean?
    // If T is a ref type are we actually doing a ref assignment here?
    // In non-generic contexts the language requires e.g. `other = ref input;` for this.
    T other = input;
}

The takeaway from all this is that it is still worthwhile to continue exploring a where T : ref struct constraint which could be used in any type kind, but the general "relaxation" of restricted type arguments described in this proposal probably only really makes sense for delegates.

Prototype

The compiler and runtime teams were able to create a prototype which handles some simple scenarios with 'ref' and 'out' type arguments to 'System.Action'. See the compiler and runtime prototype branches.

Using a combination of compiler and runtime changes, a small end-to-end scenario like the following was found to work as expected (writing "01" to the console):

using System;

C.M1(C.M0);
class C
{
    public static void M0(ref int x)
    {
        x++;
    }
    public static void M1(Action<ref int> action)
    {
        int i = 0;
        Console.Write(i);
        action(ref i);
        Console.Write(i);
    }
}

We were also pleasantly surprised by the fact that IL reading tools like ildasm and ILSpy handled the new encoding relatively gracefully:

We don't anticipate any significant roadblocks with allowing other restricted types as type arguments to delegates.

Thank you to @cston from the compiler team and @davidwrighton from the runtime team for their help in making the end-to-end prototype possible.

Drawbacks

Tools which read metadata will probably need to be updated to support this. The results from ildasm and ILSpy indicate that this might not be a great deal of work. We were able to get both tools to show Action<ref int> or equivalent, but ILSpy showed Action<out int> as Action<ref int>, for example.

We also need to figure out how C++/CLI handle this: for example, will the C++/CLI compiler crash when loading an assembly that contains delegate types like this? If we move forward with this proposal we will need to give heads up so the tool can get updated in a timely manner.

Alternatives

'ref' constraint

It's possible this feature could be implementing by introducing a set of new "constraints" rather than by applying a "check the construction" approach. This is discussed in the Why only delegates section.

Do nothing

We don't gain the benefits outlined in the motivation section.

Unresolved questions

'in' vs 'ref readonly'

The language has a bit of a wrinkle where 'in' is used for readonly by-reference parameters, while 'ref readonly' is used for readonly by-reference returns. This raises a bit of a question on how a Func which both takes and returns readonly references should work, for example.

Func<in int, ref readonly int> func; // ok..?

The question is essentially: which forms would be required when the type parameter is used for a parameter or return respectively? Do we care? Do we pick one form and disallow the other? Do we allow both interchangeably and silently use the "conventional" appearance in the symbol model, etc.

The type parameter could hypothetically be used for both parameter and return, so perhaps it is best to remain flexible and allow either in or ref readonly regardless of how the type parameter is used, or to pick one of in or ref readonly and disallow the other form in type arguments.

delegate T MyDelegate<T>(T input);
void M(MyDelegate<ref readonly int x> myDelegate);

Design meetings

https://github.com/dotnet/csharplang/blob/main/meetings/2021/LDM-2021-10-25.md#delegate-type-argument-improvements
https://github.com/dotnet/csharplang/blob/main/meetings/2022/LDM-2022-02-16.md#delegate-type-arguments-improvements