[Proposal]: ReadOnlySpan initialization from static data

[stephentoub]

ReadOnlySpan initialization from static data

• Proposed
• Prototype: Not Started
• Implementation: Not Started
• Specification: Not Started

Summary

Provide a syntax for initializing a ReadOnlySpan<T> from constant data and with guaranteed zero allocation.

Motivation

dotnet/roslyn#24621 added compiler support that translates:

ReadOnlySpan<byte> data = new byte[] { const, values };

into non-allocating code that blits the binary data into the assembly data section and creates a span that points directly to that data. The same optimization is done for:

static readonly ReadOnlySpan<byte> Data => new byte[] { const, values };

We now rely on this all over the place in dotnet/runtime and elsewhere, as it provides a very efficient means for accessing a collection of constant values with minimal overhead and in a way the JIT is able to optimize consumption of very well.

However, there are multiple problems with this:

1. The optimization only applies to byte-sized primitive T values, namely byte, sbyte, and bool. Specify any other type, and you fall off a massive cliff, as code you were hoping to be allocation-free now allocates a new array on every access.
2. It's easy to accidentally fall off a similar cliff if at least one of the values turns out to be non-const (or becomes non-const), e.g. if a const value referred to in the initialization is changed elsewhere from a const to a static readonly.
3. The syntax is confusing, as it looks like it's allocating, and PRs that optimize code from:

private static readonly byte[] s_bytes = new byte[] { ... };

to

private static ReadOnlySpan<byte> Bytes => new byte[] { ... };

are often met with confusion and misinformation.

Detailed design

Add dedicated syntax for creating spans without allocating that:

1. Doesn't visually look like it's allocating (i.e. avoid use of 'new').
2. Provides validation with errors if the data isn't provably constant.

As the following syntax fails to compile today:

ReadOnlySpan<byte> data = { 1, 2, 3 };

and

static ReadOnlySpan<byte> Data => { 1, 2, 3 };

they could be co-opted for this purpose.

Opening up this syntax via the removal of new T[] doesn't prevent the optimization from being applied by the compiler when new T[] is used, but it would guarantee a non-allocating implementation when the new T[] isn't used.

This could also tie in with params Span<T>: the syntax for the local variant could blit the data into the assembly if possible, or else fall back to the same implementation it would use for a params Span<T> method argument (assuming that params syntax itself doesn't itself fall back to heap allocation).

Implementation-wise, the compiler would use RVA statics whenever possible and fall back to static readonly arrays otherwise.

Drawbacks

TBD

Alternatives

TBD

Unresolved questions

Related to this, we have prototyped in both the runtime and the C# compiler support for extending this optimization to more than just byte-sized primitives. The difficulty with other primitives is endianness, and it can be addressed in a manner similar to array initialization: the runtime exposes a helper that either returns the original pointer, or sets up a cache of a copy of the data reversed based on the current endianness. There are also multiple fallback code generation options available for if that API isn't available. Such improvements to the compiler are related but separate from the improvements to the language syntax for this existing optimization.

Design meetings

• https://github.com/dotnet/csharplang/blob/main/meetings/2022/LDM-2022-09-21.md#readonlyspan-initialization-from-static-data
• https://github.com/dotnet/csharplang/blob/main/meetings/2023/LDM-2023-10-09.md#readonlyspan-initialization-from-static-data

cc: @jaredpar

[bernd5]

As specified in https://github.com/dotnet/csharplang/blob/3c8559f186d4c5df5d1299b0eaa4e139ae130ab6/spec/arrays.md#array-creation the syntax localOrField = { 1, 2, 3 }; is just a "shortcut" for localOrField = new int[]{ 1, 2, 3 };

So I think the optimization should be done regardless how the array initialization is written syntactically. Currently we have these forms AFAIK:

• new Type[]{ value1, value2}
• new[]{ value1, value2}
• { value1, value2 }

To force this const-array-behaviour we could maybe introduce a new intrinsic Function like System.Array.CreateConstArray(1, 2, 3).

BTW: it would be nice if we could write:

static ReadOnlySpan<byte> Data => { 1, 2, 3 };

or

static ReadOnlySpan<byte> Data { return { 1, 2, 3 }; }

[stephentoub]

So I think the optimization should be done regardless how the array initialization is written syntactically.

As I noted, the compiler would be free to do so... it already does so. But one of the key aspects here is the compiler preventing you from shooting yourself in the foot, guaranteeing that the syntax is non-allocating (at least beyond any initialization required, e.g. in the case of a big endian machine reading the assembly little endian data), and it would be a breaking change if the existing usage that allocates stopped compiling.

[jaredpar]

Provides validation with warnings or errors if the expression would allocate

This feels like the best approach to me. Essentially we could standardize on = { ... } as being the non-allocating form. Then we could warn in any situation where it would cause an allocation:

• The contents of the { ... } were non-const
• The target of the expression was Span<byte>
• The target of the expression was ReadOnlySpan<T> where T is not byte, bool or sbyte. This of course would change if we implemented the optimization for non-byte sized types.

The one part that still bothers me is arrays. That syntax works for arrays today but has none of the safe guards. I've thought a bit about adding a disabled warning here that could be enabled by developers who hit this a lot but I'm having trouble convincing myself that meets the bar. Given that this only works on fields and locals today, maybe just knowing it doesn't work on arrays is enough for developers.

BTW: it would be nice if we could write:
static ReadOnlySpan Data => { 1, 2, 3 };

That is asking for { ... } to be a more general expression where today it's limited to field and local declarations. It's not un-doable but a bit of a bigger ask.

If we took this then I'd feel a bit more strongly about finding a way to extend this warning to cases where it crossed into arrays. Take for example the code M({1, 2, 3}) which would be legal if we generalized this to expressions. Whether or not it allocates depends on overload resolution.

[The-Futurist]

I was just doing an experiment and stumbled upon what seems to be a bug:

Surely the 'buffer' is - implicitly - fixed in the sense that the GC is never going to relocate it if its inside a ref struct?

[stephentoub]

#1792

[CyrusNajmabadi]

@Korporal see #1792

--

Beaten by @stephentoub :)

[JeroenBos]

Not exactly the same but related discussion: #955, more along the lines of

const ReadOnlySpan<byte> data = new byte[] { const, values };

[tannergooding]

I particularly like the idea of putting const somewhere in the expression to indicate that the result must be considered "constant" (and therefore non-allocating, etc).

I feel like this could be more generally extended in the future to handle things like constexpr or unmanaged constants (the ability to define constants for blittable data types whose layout will never change, like Guid or Vector2/3/4, etc).

• Unmanaged Constants #688 proposes unmanaged constant support utilizing the same functionality as constant arrays of primitives and likewise a helper method for fixing up endianness. The actual data declarations fully support such declarations being structured, so the only real language requirement is some constraint/expectation/contract that the layout of a type won't change or that such layout changes (e.g. packing differences) can be handled by the runtime

[jaredpar]

I'd thought a bit about const here but discarded it for the following reasons.

Firstly if we allowed const here then we need to consider all the places in which const expressions are legal: attribute arguments, optional parameters, folding, etc ... For ReadOnlySpan<byte> I haven't dug deeply into all of these but at a high level I think there are answers for these questions. Several of them involve a significant amount of work compared to the initial suggestion of re-using an existing syntax for initialization.

One of the other suggestions we are looking at though is expanding this optimization to types which are not byte sized. For example ReadOnlySpan<char>, ReadOnlySpan<int>, etc ... That brings endianess concerns into the conversation and at the moment I don't think there are great answers for how we work those into our existing const semantics. That becomes a significant work item for us to dig into.

Secondly const only solves the issue for fields and locals. That doesn't cover all the cases where is desire to have the non-allocating syntax. Reading into the issue you can see the desire exists to essentially see this at the expression level hence const doesn't solve it.

That's not to say I don't want const ReadOnly<byte> to work in the future (i'd actually like it). I just don't think it's necessarily the best solution for the particular problem being stated here.

[svick]

@jaredpar What if the new const expressions were completely separate from the existing constant expressions and const fields/variables? Syntactically, it could look something like:

ReadOnlySpan<byte> data = const byte[] { 1, 2, 3 };

(This may be what @tannergooding meant by "putting const somewhere in the expression".)

I think this is nice, because:

• it more clearly expresses that it's the expression that's changing, not the field or variable,
• it doesn't have to work for attributes etc.,
• it works everywhere you can have expressions,
• it's consistent with new and stackalloc: the keyword indicates where you're allocating from.

It's also problematic, because it would be very confusing if const meant two related, but very different things. Though that could be solved by using a different keyword (constalloc? constexpr? static?).

[jaredpar]

@svick

What if the new const expressions were completely separate from the existing constant expressions and const fields/variables?

There isn't inherently anything wrong with this. But it's introducing a new expression form which is going to have a higher bar than re-using existing expression form in more places.

it doesn't have to work for attributes etc.,

It would be really weird if it didn't though. For example it would be weird if I could say const i = const 42 but not [My(const 1)]. I can't introducing this without introducing the complete picture.

It's also problematic, because it would be very confusing if const meant two related, but very different things.

This doesn't bother me. It's close enough to use for both concepts. Or said differently it's not different enough that I would warrant a new keyword for it.

[naine]

It would be really weird if it didn't though. For example it would be weird if I could say const i = const 42 but not [My(const 1)]. I can't introducing this without introducing the complete picture.

It doesn't need to be supported for all constant expressions, only array initializer expressions:

ReadOnlySpan<byte> span = new byte[] { 1, 2, 3 };        // creates span from System.Array (unless optimizable)
ReadOnlySpan<byte> span = stackalloc byte[] { 1, 2, 3 }; // creates span from stack array
ReadOnlySpan<byte> span = const byte[] { 1, 2, 3 };      // creates span from assembly data section

[jaredpar]

It doesn't need to be supported for all constant expressions, only array initializer expressions:

That seems very counterintuitive. If const is meant to explicitly identify expressions that are const why should it be limited to a subset of const expressions?

Yes I agree that const 42 (or any other literal) is a bit silly but what about const A.B? Lacking const that could be a side effecting property but with const prefix I've guaranteed I'm accessing a constant here. Or even const A + B as another example.

I can't see introducing const as a way to identify const array expressions but exclude it from all the other expression that are constant.