API Proposal: Add type to avoid boxing .NET intrinsic types

[JeremyKuhne]

Background and Motivation

Currently there is no way to pass around a heterogeneous set of .NET value types without boxing them into objects or creating a custom wrapper struct. To facilitate low allocation exchange of value types we should provide a struct that allows passing the most common value types without boxing and still allows storing within other types (including arrays) or on the heap when needed.

ASP.NET and Azure SDK have both expressed a need for this functionality for scenarios such as logging.

This following is an evolved proposal based on feedback from various sources. The original proposal is included below. Key changes were to make this a smaller type, alignment with object semantics, support of any object, and more focused non-boxing support.

Proposed API

public readonly struct Value
{
    public Value(object? value);
    public Value(byte value);
    public Value(byte? value);
    public Value(sbyte value);
    public Value(sbyte? value);
    public Value(bool value);
    public Value(bool? value);
    public Value(char value);
    public Value(char? value);
    public Value(short value);
    public Value(short? value);
    public Value(int value);
    public Value(int? value);
    public Value(long value);
    public Value(long? value);
    public Value(ushort value);
    public Value(ushort? value);
    public Value(uint value);
    public Value(uint? value);
    public Value(ulong value);
    public Value(ulong? value);
    public Value(float value);
    public Value(float? value);
    public Value(double value);
    public Value(double? value);
    public Value(DateTimeOffset value);         // Boxes with offsets that don't fall between 1800 and 2250
    public Value(DateTimeOffset? value);        // Boxes with offsets that don't fall between 1800 and 2250
    public Value(DateTime value);
    public Value(DateTime? value);
    public Value(ArraySegment<byte> segment);
    public Value(ArraySegment<char> segment);
    // No decimal as it always boxes


    public Type? Type { get; }                      // Type or null if the Value represents null
    public static Value Create<T>(T value);
    public unsafe bool TryGetValue<T>(out T value); // Fastest extraction
    public T As<T>();                               // Throws InvalidCastException if not supported

    // For each type of constructor except `object`:
    public static implicit operator Value(int value) => new(value);
    public static explicit operator int(in Value value) => value.As<int>();
}

Fully working prototype

Usage Examples

public static void Foo(Value value)
{
    Type? type = value.Type;
    if (type == typeof(int))
    {
        int @int = value.As<int>();

        // Any casts that would work with object work with Value

        int? nullable = value.As<int?>();

        object o = value.As<object>();
    }

    if (value.TryGetValue(out long @long))
    {
        // TryGetValue follows the same casting rules as "As"
    }

    // Enums are not boxed if passed through the Create method
    Value dayValue = Value.Create(DayOfWeek.Friday);

    // Does not box (until Now is > 2250)
    Value localTime = DateTimeOffset.Now;
    localTime = Value.Create(DateTimeOffset.Now);
    localTime = new(DateTimeOffset.Now);

    // ArraySegment<char> and ArraySegment<byte> are supported
    Value segment = new ArraySegment<byte>(new byte[2]);

    // Any type can go into value, however. Unsupported types will box as they do with object.
    Value otherSegment = new(new ArraySegment<int>(new int[1]));
}

Details

Goals

• Takes any value
• Can be stored anywhere
• Follows object semantics (can't box a nullable, for example)
• Type is 128 bits on 64bit platforms
• Internal data is opaque
• Does not box intrinsics (outside of decimal)
• Does not box nullable intrinsics
• Does not box DateTime or most DateTimeOffset values (1800-2250 for local times supported)

Other benefits

• As is can be implemented out-of-box

Other Possible Names

• Variant
• ValueObject
• ???

Original Proposal

Currently there is no way to pass around a heterogeneous set of .NET value types without boxing them into objects or creating a custom wrapper struct. To facilitate low allocation exchange of value types we should provide a struct that allows passing the information without heap allocations. The canonical example of where this would be useful is in `String.Format`.

Related proposals and sample PRs

• C#: Efficient Params and String Formatting
• CoreFX: https://github.com/dotnet/corefx/issues/28379
• Non-allocating string format prototype: Protoype for nonallocating string formatting corefxlab#2595

Goals

1. Support intrinsic value types (int, float, etc.)
2. Support most common value types used in formatting (DateTime)
3. Have high performance
4. Balance struct size against type usage frequency
5. Facilitate "raw" removal of value type data (you want to force cast to int, fine)
6. Provide a mechanism for passing a small collection of Variants via the stack
7. Allow all types by falling back to boxing
8. Support low allocation interpolated strings

Non Goals

1. Support all value types without boxing
2. Make it work as well on .NET Framework as it does on Core (presuming it's possible in the final design)

Nice to Have

1. Usable on .NET Framework (currently does)

General Approach

Variant is a struct that contains an object pointer and a "union" struct that allows stashing of arbitrary blittable (i.e. where unmanaged) value types that are within a specific size constraint.

Sample Usage

// Consuming method
public void Foo(ReadOnlySpan<Variant> data)
{
     foreach (Variant item in data)
     {
         switch (item.Type)
         {
             case VariantType.Int32:
             //   ...
         }
     }
}

// Calling method
public void Bar()
{
     var data = Variant.Create(42, true, "Wow");
     Foo(data.ToSpan());

     // Only needed if running on .NET Framework
     data.KeepAlive();
}

Surface Area

namespace System
{
    /// <summary>
    /// <see cref="Variant"/> is a wrapper that avoids boxing common value types.
    /// </summary>
    public readonly struct Variant
    {
        public readonly VariantType Type;

        /// <summary>
        /// Get the value as an object if the value is stored as an object.
        /// </summary>
        /// <param name="value">The value, if an object, or null.</param>
        /// <returns>True if the value is actually an object.</returns>
        public bool TryGetValue(out object value);

        /// <summary>
        /// Get the value as the requested type <typeparamref name="T"/> if actually stored as that type.
        /// </summary>
        /// <param name="value">The value if stored as (T), or default.</param>
        /// <returns>True if the <see cref="Variant"/> is of the requested type.</returns>
        public unsafe bool TryGetValue<T>(out T value) where T : unmanaged;

        // We have explicit constructors for each of the supported types for performance
        // and to restrict Variant to "safe" types. Allowing any struct that would fit
        // into the Union would expose users to issues where bad struct state could cause
        // hard failures like buffer overruns etc.
        public Variant(bool value);
        public Variant(byte value); 
        public Variant(sbyte value);
        public Variant(short value);
        public Variant(ushort value);
        public Variant(int value);
        public Variant(uint value);
        public Variant(long value);
        public Variant(ulong value);
        public Variant(float value);
        public Variant(double value);
        public Variant(decimal value);
        public Variant(DateTime value);
        public Variant(DateTimeOffset value);
        public Variant(Guid value);
        public Variant(object value);

        /// <summary>
        /// Get the value as an object, boxing if necessary.
        /// </summary>
        public object Box();

        // Idea is that you can cast to whatever supported type you want if you're explicit.
        // Worst case is you get default or nonsense values.

        public static explicit operator bool(in Variant variant);
        public static explicit operator byte(in Variant variant);
        public static explicit operator char(in Variant variant);
        public static explicit operator DateTime(in Variant variant);
        public static explicit operator DateTimeOffset(in Variant variant);
        public static explicit operator decimal(in Variant variant);
        public static explicit operator double(in Variant variant);
        public static explicit operator Guid(in Variant variant);
        public static explicit operator short(in Variant variant);
        public static explicit operator int(in Variant variant);
        public static explicit operator long(in Variant variant);
        public static explicit operator sbyte(in Variant variant);
        public static explicit operator float(in Variant variant);
        public static explicit operator TimeSpan(in Variant variant);
        public static explicit operator ushort(in Variant variant);
        public static explicit operator uint(in Variant variant);
        public static explicit operator ulong(in Variant variant);

        public static implicit operator Variant(bool value);
        public static implicit operator Variant(byte value);
        public static implicit operator Variant(char value);
        public static implicit operator Variant(DateTime value);
        public static implicit operator Variant(DateTimeOffset value);
        public static implicit operator Variant(decimal value);
        public static implicit operator Variant(double value);
        public static implicit operator Variant(Guid value);
        public static implicit operator Variant(short value);
        public static implicit operator Variant(int value);
        public static implicit operator Variant(long value);
        public static implicit operator Variant(sbyte value);
        public static implicit operator Variant(float value);
        public static implicit operator Variant(TimeSpan value);
        public static implicit operator Variant(ushort value);
        public static implicit operator Variant(uint value);
        public static implicit operator Variant(ulong value);

        // Common object types
        public static implicit operator Variant(string value);

        public static Variant Create(in Variant variant) => variant;
        public static Variant2 Create(in Variant first, in Variant second) => new Variant2(in first, in second);
        public static Variant3 Create(in Variant first, in Variant second, in Variant third) => new Variant3(in first, in second, in third);
    }

    // Here we could use values where we leverage bit flags to categorize quickly (such as integer values, floating point, etc.)
    public enum VariantType
    {
        Object,
        Byte,
        SByte,
        Char,
        Boolean,
        Int16,
        UInt16,
        Int32,
        UInt32,
        Int64,
        UInt64,
        DateTime,
        DateTimeOffset,
        TimeSpan,
        Single,
        Double,
        Decimal,
        Guid
    }

    // This is an "advanced" pattern we can use to create stack based spans of Variant. Would also create at least a Variant3.
    public readonly struct Variant2
    {
        public readonly Variant First;
        public readonly Variant Second;

        public Variant2(in Variant first, in Variant second);

        // This is for keeping objects rooted on .NET Framework once turned into a Span (similar to GC.KeepAlive(), but avoiding boxing).
        [MethodImpl(MethodImplOptions.NoInlining)]        
        public void KeepAlive();

        public ReadOnlySpan<Variant> ToSpan();
    }
}

FAQ

Why "Variant"?

• It does perform a function "similar" to OLE/COM Variant so the term "fits". Other name suggestions are welcome.

Why isn't Variant a ref struct?

• Primarily because you can't create a Span of ref structs.
• We also want to give the ability to store arrays of these on the heap when needed

What about variadic argument support (__arglist, ArgIterator, etc.)?

• Short answer: not sufficient. Referred to as "Vararg" in the CLI specification, the current implemenation is primarily for C++/CLI. It isn't supported on Core yet and would require significant investment to support scenarios here reliably and to support non-Windows environments. This would put any solution based on this way out and may make down level support impossible.

What about TypedReference and __makeref, etc.?

• TypedReference is a ref struct (see above). Variant gives us more implementation flexibility, doesn't rely on undocumented keywords, and is actually faster. (Simple test of wrapping/unwrapping an int it is roughly 10-12% faster depending on inlining.)

Why not support anything that fits?

• We could in theory, but there would be safety concerns with getting the data back out. To support high performance usage we want to allow hard casts of value data.

How about enums?

• This one may be worth it and is technically doable. Still investigating...

cc: @jaredpar, @vancem, @danmosemsft, @jkotas, @davidwrighton, @stephentoub

[benaadams]

Would this be a 16 byte (Guid/Decimal) + enum sized struct? (24 bytes with padding on x64)

[jkotas]

TypedReference is a ref struct (see above). Variant gives us more implementation flexibility, doesn't rely on undocumented keywords, and is actually faster.

These all can be fixed, without too much work. TypedReference has been neglected, but that does not mean it is a useless type. (Some of this is described in https://github.com/dotnet/corefx/issues/29736.)

I think fixing TypedReference would be a better choice than introducing a new Variant type, if everything else is equal.

Allow all types by falling back to boxing

I think the design should allow all types without falling back to boxing.

Work on .NET Framework

This should be a non-goal. It is fine if the winning design that we pick happens to work on .NET Framework, but trying to make it work on .NET Framework should be an explicit non-goal. We have made a contious design to not restrict our design choices to what works on .NET Framework.

[JeremyKuhne]

Would this be a 16 byte (Guid/Decimal) + enum sized struct? (24 bytes with padding on x64)

Goal is 24 bytes. We've looked at a lot of different ways of packing that in. A pointer and 16 bytes of data. It might involve some contortions or dropping down to 12 bytes of data.

but that does not mean it is a useless type.

Not trying to infer it is useless, just not appropriate in this case. I'm not sure how you'd make it a non-ref struct or make as fast as something targeted at key types.

This should be a non-goal.

Fair enough, I've changed it to nice-to-have. There are, however, real business needs for mitigating formatting inefficiencies on .NET Framework.

I think the design should allow all types without falling back to boxing.

I think we should have some design that does this but I don't think we can provide a solution that solves everything for all scenarios well. Having multiple approaches doesn't seem like a terrible thing to me, particularly given that we could make this sort of solution available much much sooner than full varargs support.

[stephentoub]

I think we should have some design that does this but I don't think we can provide a solution that solves everything for all scenarios well. Having multiple approaches doesn't seem like a terrible thing to me, particularly given that we could make this sort of solution available much much sooner than full varargs support.

FWIW, this approach feels very limited to me, in that I see supporting every value type as a key scenario. I would rather see, for example, a simple unsafe annotation/attribute that would let the API tell the JIT that it promises wholeheartedly an argument won't escape, and then add an overload that takes a [UnsafeWontEscape] params ReadOnlySpan<object> args, where the JIT would stack-allocate the boxes for any value types provided. Just an example.

[JeremyKuhne]

FWIW, this approach feels very limited to me, in that I see supporting every value type as a key scenario. I would rather see, for example, a simple unsafe annotation/attribute that would let the API tell the JIT that it promises wholeheartedly an argument won't escape, and then add an overload that takes a [UnsafeWontEscape] params ReadOnlySpan<object> args, where the JIT would stack-allocate the boxes for any value types provided. Just an example.

To be super clear, I don't see this as a solves-all-boxing solution. I absolutely think we can benefit from broader approaches, but I have a concern about being efficient with core types. Being able to quickly tell that you have an int and extract it is super valuable I think. Certainly for the String.Format case, for example. :)

[jkotas]

Being able to quickly tell that you have an int and extract it is super valuable I think.

Depends on how the actual formatting is implemented. If you can dispatch a virtual formatting method, ability to switch over a primitive type does not seem super valuable.

[UnsafeWontEscape] params ReadOnlySpan<object> args

Something like this would work too. It is pretty similar to ReadOnlySpan<TypedReference> on the surface, with different tradeoffs and low-level building blocks.

[stephentoub]

It is pretty similar to ReadOnlySpan on the surface, with different tradeoffs and low-level building blocks.

I'd be fine with that as well if it was similarly seamless to a caller.

[jaredpar]

Rather than an attribute and a promise I'd like to leverage the type system if possible here. 😄

What if instead we added a JIT intrinsic that "boxes" value types into a ref struct named Boxed. This type would have just enough information to allow manipulation of the boxed value:

ref struct Boxed {
  Type GetBoxedType();
  T GetBoxedValue<T>();
}

The JIT could choose to make this a heap or stack allocation depending on the scenario. The important part is that it would move the boxing operation into a type whose lifetime we need to carefully monitor. The compiler will do it for us.

That doesn't completely solve the problem because you can't have ReadOnlySpan<Boxed> as a ref struct can't be a generic argument. That's not because of a fundamental limitation of the type system but more because we didn't have a motivating scenario. Suppose this scenario was enough and we went through the work in C# to allow it. Then we could have the signature of the method be params ReadOnlySpan<Boxed>. No promises needed here, the compiler will be happy to make developers do the right thing 😉

[stephentoub]

That also sounds reasonable.

(Though the [UnsafeWontEscape] approach could also work on the existing APIs: we just attribute the existing object arguments in the existing methods, and apps just get better.)

[jkotas]

How would struct Boxed differ from existing TypedReference (with extra methods added to make it useful)?

Either way, it sounds reasonable too.

[vancem]

I do think that if our goal is just to solve the parameter passing problem, something based on references (which can work uniformly on all types) is worth thinking about (this is Jan's TypedReference approach).

However that does leave out the ability to have something that can represent anything (but all primitives efficiently (without extra allocation)) that you can put into objects (which is what Variant is).

I think the fact that we don't have a standard 'Variant' type in the framework is rather unfortunate. Ultimately it is an 'obvious' type to have in the system (even if ultimately you solve the parameter passing issue with some magic stack allocated array of types references).

I also am concernd that we are solving a 'simple' problem (passing prameters) with a more complex one (tricky refernece based classes whose safety is at best subtle).

I think we should have a Variant class, it is straightforward, and does solve some immediate problems without having to design a rather advanced feature (that probably would not make V3.0.

For what it is worth...

[jkotas]

we don't have a standard 'Variant' type in the framework is rather unfortunate.

I agree with that and the Variant proposal would look reasonable to me if the Variant was optimized for primitive types only. The proposal makes it optimized for primitive types and set of value types that we think are important for logging today. It does not feel like a design that will survive over time. I suspect that there will be need to optimize more types, but it won't be possible to extend the design to fit them.

[vancem]

Note that generally speaking, a Variant is a chunk of memory that holds things in-line and a pointer that allows you to hold 'anything'.

Semantically it is always the case that a variant can hold 'anything', so that is nice in that the there is not a 'sematic' cliff, only a performance cliff (thus as long as the new types that we might want to add in the future are not perf critical things are OK. I note that the list that really are perf-critical are pretty small and likely to not change over time (int, string, second tier are long, and maybe DateTime(Offset)). So I don't think we are taking a huge risk there.

And there are things you can do 'after the fact' Lets assume we only alotted 16 bytes for in-line data but we wanted something bigger. If there is any 'skew' to the values (this would for most types, but not for random number generated IDs), you could at least store the 'likely' values inline and box the rest. It would probably be OK, and frankly it really is probably the right tradeoff (it would be surprising to me that a new type in the future so dominated the perf landscape over existing types that it was the right call to make the struct bigger to allow it to be stored inline). That has NEVER happened so far.

Indeed from a cost-benefit point of view, we really should be skewing things to the int and string case becasue these are so much more likely to dominate hot paths. We certainly don't want this to be bigger than 3 pointers, and it would be nice to get it down to 2 (but that does require heroics for any 8 byte sized things (long, double, datetime ...), so I think we are probably doing 3.

But it does feel like a 'stable' design (5 years from now we would not feel like we made a mistake), sure bugger types will be slow, but I don't think would want to make the type bigger even if we could. It would be the wrong tradeoff.

So, I think Variant does have a reasonablys table design point, that can stand the test of time.

From my point of view, I would prefer that the implementation be tuned for overwhelmingly likely case of int an string). My ideal implementation would be a 8 bytes of inline-data / discriminator, and 1 object pointer. This is a pro

[stephentoub]

One of the main use cases this is being proposed for is around string interpolation and string formatting.

I realize there are other uses cases, so not necessarily instead of a something Variant-like, but specifically to address the case of string interpolation, I had another thought on an approach….

Today, you can define a method like:

AppendFormat(FormattableString s);

and use that as the target of string interpolation, e.g.

AppendFormat($”My type is {GetType()}.  My value is {_value:x}.”);

Imagine we had a pattern (or an interface, though that adds challenge for ref structs) the compiler could recognize where a type could expose a method of the form:

AppendFormat(object value, ReadOnlySpan<char> format);

The type could expose additional overloads as well, and the compiler would use normal overload resolution when determining which method to call, but the above would be sufficient to allow string interpolation to be used with the type in the new way. We could add this method to StringBuilder, for example, along with additional overloads for efficiency, e.g.

public class StringBuilder
{
    public void AppendFormat(object value, ReadOnlySpan<char> format);
    public void AppendFormat(int value, ReadOnlySpan<char> format);
    public void AppendFormat(long value, ReadOnlySpan<char> format);
    public void AppendFormat(ReadOnlySpan<char> value, ReadOnlySpan<char> format);
    … // etc.
}

We could also define new types (as could anyone), as long as they implemented this pattern, e.g.

public ref struct ValueStringBuilder
{
    public ValueStringBuilder(Span<char> initialBuffer);
    
    public void AppendFormat(FormattableString s);
    public void AppendFormat(object value, ReadOnlySpan<char> format);
    public void AppendFormat(int value, ReadOnlySpan<char> format);
    public void Appendformat(long value, ReadOnlySpan<char> format);
    public void AppendFormat(ReadOnlySpan<char> value, ReadOnlySpan<char> format);
    … // etc.

    public Span<char> Value { get; }
}

Now, when you call:

ValueStringBuilder vsb = …;
vsb.AppendFormat($”My type is {GetType()}.  My value is {_value:x}.”);

rather than generating what it would generate today if this took a FormattableString:

vsb.AppendFormat(FormattableStringFactory.Create("My type is {0}. My value is {1:x}.”, new object[] { GetType(), (object)_value }));

or if it took a string:

vsb.AppendFormat(string.Format("My type is {0}. My value is {1:x}.”, GetType(), (object)_value));

it would instead generate:

vsb.AppendFormat(“My type is “, default);
vsb.AppendFormat(GetType(), default);
vsb.AppendFormat(“. My value is “, default);
vsb.AppendFormat(_value, “x”);
vsb.AppendFormat(".", default);

There are more calls here, but most of the parsing is done at compile time rather than at run time, and a type can expose overloads to allow any type T to avoid boxing, including one that takes a generic T if so desired.