[API Proposal]: ReadOnlySpan<T> CreateSpan<T>(RuntimeFieldHandle)

[davidwrighton]

Background and motivation

As of C# 8.0 it is possible to embed constant byte data into .NET Assemblies using a syntax like

ReadOnlySpan<byte> someData = new byte[] { 0, 1 };

However, unlike the support for array initialization this isn't possible for other primitive data types. Instead for other data types the ReadOnlySpan is initialized by actually allocating an array. The reason for this lack of support is that there is no current way to express that the constant data is in little endian format, and needs to be translated to the runtime endian format, if the application is run on hardware which utilizes big endian numbers.

This proposal is designed to provide a new api which works in a manner similar to how array initialization via RuntimeHelpers.InitializeArray works. This will allow constant data to be cleanly expressed on all architectures.

During a recent hackathon event, support for this was prototyped, and a simple example such as

    static int[] _intData = new int[] {1, 2, 3, 4, 5, 6, 7, 8, 9, 10 };

    [MethodImpl(MethodImplOptions.NoInlining)]
    static int GetData(int offset)
    {
        return _intData[offset];
    }

    [MethodImpl(MethodImplOptions.NoInlining)]
    static int GetDataROS(int offset)
    {
        ReadOnlySpan<int> intSpan = (ReadOnlySpan<int>)new int[]{1, 2, 3, 4, 5, 6, 7, 8, 9, 10};
        return intSpan[offset];
    }

was shown to be a fair bit faster.

In particular, looped 1.8 billion times GetData took 9783ms and 'GetDataROS' took 8468ms to execute. For an improvement of somewhere around 13.5%.

@jaredpar has looked at the hackathon effort and is in general agreement that the C# team would be willing to implement code generation of this.

@stephentoub has also done some investigation of locations within the BCL where this sort of data would be useful, and there are a significant number of potential optimizations. Individually, these optimizations are generally fairly small, but there are quite a few which might benefit.

API Proposal

namespace System.Runtime.CompilerServices
{
    public static class RuntimeHelpers
    {
        public ReadOnlySpan<T> CreateSpan<T>(RuntimeFieldHandle fldHandle);
    }
}

Requirements:
T must be a primitive constant sized type (byte, sbyte, char, short, ushort, int, uint, long, ulong, float, double)
field must reference a FieldRVA
The RVA associated with field must be aligned on the size of the primitive type T.

API Usage

ReadOnlySpan<int> intSpan = new int[]{1, 2, 3, 4, 5, 6, 7, 8, 9, 10};

Or possibly if the C# team wish to implement...

ReadOnlySpan<int> intSpan = stackalloc int[]{1, 2, 3, 4, 5, 6, 7, 8, 9, 10};

There is no expectation that developers not generating IL would ever use this api directly.

Alternative Designs

It may be simpler for the JIT to work with the following construct. However, I don't believe it is a significant improvement.

namespace System.Runtime.CompilerServices
{
    public static class RuntimeHelpers
    {
        public CreateSpan<T>(RuntimeFieldHandle field, out ReadOnlySpan<T> span);
    }
}

Risks

Since the new api would only be available on new versions of .NET but the syntax is valid for older frameworks, customers writing code may write code expecting the optimized behavior and instead see substantially slower, allocating logic instead. Suggestions have been made that support for this may also entail building a fallback generator for this sort of thing when targeting older versions of the frameworks to avoid performance cliffs.

Tagging subscribers to this area: @GrabYourPitchforks, @dotnet/area-system-memory
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Background and motivation

As of C# 8.0 it is possible to embed constant byte data into .NET Assemblies using a syntax like

ReadOnlySpan<byte> someData = new byte[] { 0, 1 };

However, unlike the support for array initialization this isn't possible for other primitive data types. Instead for other data types the ReadOnlySpan is initialized by actually allocating an array. The reason for this lack of support is that there is no current way to express that the constant data is in little endian format, and needs to be translated to the runtime endian format, if the application is run on hardware which utilizes big endian numbers.

This proposal is designed to provide a new api which works in a manner similar to how array initialization via RuntimeHelpers.InitializeArray works. This will allow constant data to be cleanly expressed on all architectures.

During a recent hackathon event, support for this was prototyped, and a simple example such as

    static int[] _intData = new int[] {1, 2, 3, 4, 5, 6, 7, 8, 9, 10 };

    [MethodImpl(MethodImplOptions.NoInlining)]
    static int GetData(int offset)
    {
        return _intData[offset];
    }

    [MethodImpl(MethodImplOptions.NoInlining)]
    static int GetDataROS(int offset)
    {
        ReadOnlySpan<int> intSpan = (ReadOnlySpan<int>)new int[]{1, 2, 3, 4, 5, 6, 7, 8, 9, 10};
        return intSpan[offset];
    }

was shown to be a fair bit faster.

In particular, looped 1.8 billion times GetData took 9783ms and 'GetDataROS' took 8468ms to execute. For an improvement of somewhere around 13.5%.

@jaredpar has looked at the hackathon effort and is in general agreement that the C# team would be willing to implement code generation of this.

@stephentoub has also done some investigation of locations within the BCL where this sort of data would be useful, and there are a significant number of potential optimizations. Individually, these optimizations are generally fairly small, but there are quite a few which might benefit.

API Proposal

namespace System.Runtime.CompilerServices
{
    public static class RuntimeHelpers
    {
        public ReadOnlySpan<T> CreateSpan<T>(RuntimeFieldHandle field);
    }
}

Requirements:
T must be a primitive constant sized type (byte, sbyte, char, short, ushort, int, uint, long, ulong, float, double)
field must reference a FieldRVA
The RVA associated with field must be aligned on the size of the primitive type T.

API Usage

ReadOnlySpan<int> intSpan = new int[]{1, 2, 3, 4, 5, 6, 7, 8, 9, 10};

Or possibly if the C# team wish to implement...

ReadOnlySpan<int> intSpan = stackalloc int[]{1, 2, 3, 4, 5, 6, 7, 8, 9, 10};

There is no expectation that developers not generating IL would ever use this api directly.

Alternative Designs

It may be simpler for the JIT to work with the following construct. However, I don't believe it is a significant improvement.

namespace System.Runtime.CompilerServices
{
    public static class RuntimeHelpers
    {
        public CreateSpan<T>(RuntimeFieldHandle field, out ReadOnlySpan<T> span);
    }
}

Risks

Since the new api would only be available on new versions of .NET but the syntax is valid for older frameworks, customers writing code may write code expecting the optimized behavior and instead see substantially slower, allocating logic instead. Suggestions have been made that support for this may also entail building a fallback generator for this sort of thing when targeting older versions of the frameworks to avoid performance cliffs.

Author:	davidwrighton
Assignees:	-
Labels:	api-suggestion, area-System.Memory, untriaged
Milestone:	-

[davidwrighton]

@AaronRobinsonMSFT (Who worked on the hackathon on this project)
@svick
@JulieLeeMSFT This requires a new JIT intrinsic, could you suggest an engineer to review the intrinsic and provide an opinion about this?

[stephentoub]

Nice write-up. #24961 (comment) was already approved... is this different?

[jkotas]

Nit: The approved API has fldHandle as the argument name, but it is exactly the same otherwise.

[JulieLeeMSFT]

@echesakovMSFT PTAL.
cc @tannergooding

Nit: The approved API has fldHandle as the argument name, but it is exactly the same otherwise.

@jkotas if it is the exactly same API with a different argument name, should we close this one and merge this thread with #24961?

[jkotas]

Let's keep this one open since it follows the API proposal template and close #24961.

[VSadov]

There is a difference between the two proposed use variants. I think Roslyn should implement optimization for both.

Re:

API Usage

ReadOnlySpan<int> intSpan = new int[]{1, 2, 3, 4, 5, 6, 7, 8, 9, 10};

Or possibly if the C# team wish to implement...

ReadOnlySpan<int> intSpan = stackalloc int[]{1, 2, 3, 4, 5, 6, 7, 8, 9, 10};

---

When optimized, these will result in the same IL, but language treats them differently.

Span local will assume escape scope from its initializer and stackalloc'd spans cannot escape the method scope.
On the other hand new int[] variant has global scope - i.e. can be returned from a method, so user would typically use that to have a less constrained variable.

{
     ReadOnlySpan<int> intSpan = new int[]{1, 2, 3, 4, 5, 6, 7, 8, 9, 10};
     // this is ok
     return intSpan;
}

However there is a scenario where user would want the more restricted stackalloc int[]{1, 2, 3, 4, 5, 6, 7, 8, 9, 10}; variant.
That is when escaping the span is not the goal, but reinitializing the span with another stackalloc is a possibility.

Ex:

ReadOnlySpan<int> intSpan = stackalloc int[]{1, 2, 3};

if (condition)
{
    // if intSpan is globally escapable, this would be a compile error
    intSpan = stackalloc int[] {x, y, z,};
}

stackalloc does not produce garbage, but no-copy initialization is still cheaper, so stackalloc variant should be optimized too.

[davidwrighton]

@VSadov I agree that the various C# lowerings/optimizations are interesting to discuss, but I don't believe that this is the right forum. I believe the discussion about the precise lowering, and when it should happen, etc should be done in the Roslyn or C# language repos.