JIT struct work planned for .NET 9

[jakobbotsch]

This issue captures the planned work items for .NET 9 with respect to struct improvements. This list is expected to change throughout the release cycle according to ongoing planning and discussions, with possible additions and subtractions to the scope.

Expanding the scope of physical promotion

.NET 8 added physical promotion, which removes many of the limitations of the existing promotion pass. Physical promotion has the following pros over the regular promotion pass:

• It can partially promote only the struct fields that are used in the function
• It can promote structs with arbitrary number of fields
• It supports recursive promotion of structs
• Physically promoted fields are always normalize-on-load
• Physically promoted fields can always be enregistered; it does not have the notion of dependent promotion

Long term, we would like physical promotion to replace the regular promotion pass entirely. We do NOT expect this to happen in .NET 9. However, we do expect to make progress towards this goal in .NET 9.

ABI handling

The main limitation that stops physical promotion from replacing regular promotion is currently that it lacks the support around ABI boundaries that regular promotion has. This comes out of the fact that our multireg support is tied very directly into the existing promotion mechanism.
Thus, we expect to work on some of the following items in .NET 9:

• ('24) Support producing FIELD_LIST in physical promotion for call arguments, allowing physically promoted arguments to stay in registers (JIT: Transform multi-reg args to FIELD_LIST in physical promotion #104370)
• ('24) Support producing FIELD_LIST in physical promotion for returned values, allowing physically promoted return values to stay in registers (JIT: More elaborate store forwarding patterns with physical promotion #86665)
• ('24) Support FIELD_LIST for returns in the backend
• (Stretch) Allow struct fields of physically promoted parameters to stay in registers (experiment: JIT: [experiment] Add an explicit IR representation for parameter definitions #92026)
• (Stretch) Allow struct fields of physically promoted call returns to stay in registers

The FIELD_LIST items are expected to be easier than the last two items (this is related to the fact that we do not have a good way of representing multiply-defined things in JIT IR).

Reducing scope of dependent promotion

We also expect to switch some cases where we can predict dependent promotion to be handled by physical promotion instead. This is generally expected to have good CQ benefits as it allows fields to stay in registers.

• ('24) Handle retbuf definitions via physical promotion (JIT: Skip old promotion for retbuf defined locals #104439)
• ('24) Handle structs with significant padding via physical promotion (JIT: Disable old promotion for structs with significant padding #104438)

Miscellaneous work items

• JIT: Propagate LCL_ADDR nodes during local morph #102808
• JIT: Support retbuf optimization for non 'lvIsTemp' locals #104467

Moved out of .NET 9

Struct copy propagation

The JIT has to create a temporary when constructing structs on the off chance that the constructor observes address of this. However, practically no struct constructor depends on this, and we are typically able to prove this after inlining. In many cases, however, we still end up with subpar CQ due to the copy.

This work item is about introducing a pass to get rid of copies in the common case where we inlined and proved them to be unnecessary.

• [RyuJIT] Introduce a general reverse copy prop for struct temps #8887

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

Issue Details

---

This issue captures the planned work items for .NET 9 with respect to struct improvements. This list is expected to change throughout the release cycle according to ongoing planning and discussions, with possible additions and subtractions to the scope.

Expanding the scope of physical promotion

.NET 8 added physical promotion, which removes many of the limitations of the existing promotion pass. Physical promotion has the following pros over the regular promotion pass:

• It can partially promote only the struct fields that are used in the function
• It can promote structs with arbitrary number of fields
• It supports recursive promotion of structs
• Physically promoted fields are always normalize-on-load
• Physically promoted fields can always be enregistered; it does not have the notion of dependent promotion

Long term, we would like physical promotion to replace the regular promotion pass entirely. We do NOT expect this to happen in .NET 9. However, we do expect to make progress towards this goal in .NET 9.

ABI handling

The main limitation that stops physical promotion from replacing regular promotion is currently that it lacks the support around ABI boundaries that regular promotion has. This comes out of the fact that our multireg support is tied very directly into the existing promotion mechanism.
Thus, we expect to work on some of the following items in .NET 9:

• (Q4) Support producing FIELD_LIST in physical promotion for call arguments
• ('24) Support producing FIELD_LIST in physical promotion for returned values
• ('24) Support FIELD_LIST for returns in the backend
• (Stretch) Allow struct fields of physically promoted parameters to stay in registers (experiment: JIT: [experiment] Add an explicit IR representation for parameter definitions #92026)
• (Stretch) Allow struct fields of physically promoted call returns to stay in registers

The FIELD_LIST items are expected to be easier than the last two items (this is related to the fact that we do not have a good way of representing multiply-defined things in JIT IR).

Reducing scope of dependent promotion

We also expect to switch some cases where we can predict dependent promotion to be handled by physical promotion instead. This is generally expected to have good CQ benefits as it allows fields to stay in registers.

• ('24) Handle retbuf definitions via physical promotion
• ('24) Handle structs with holes via physical promotion

To accomplish the first item we will likely reorder regular promotion and local morph. This means introducing another walk over locals somewhere after regular promotion to replace LCL_FLDs that refer to regular promoted fields. However, it also means we can avoid promoting address exposed structs.

Struct copy propagation

The JIT has to create a temporary when constructing structs on the off chance that the constructor observes address of this. However, practically no struct constructor depends on this, and we are typically able to prove this after inlining. In many cases, however, we still end up with subpar CQ due to the copy.

This work item is about introducing a pass to get rid of copies in the common case where we inlined and proved them to be unnecessary.

• ('24) [RyuJIT] Introduce a general reverse copy prop for struct temps #8887

Miscellaneous work items

• (Q4) JIT: Support retbuf optimization for non 'lvIsTemp' locals #93071

Author:	jakobbotsch
Assignees:	-
Labels:	area-CodeGen-coreclr
Milestone:	-

[jakobbotsch]

A couple of user examples where dependent promotion harms CQ:

using System.Runtime.CompilerServices;
using System.Runtime.Intrinsics;

public static class Tests
{
    public static MyVec2 TestAdd1(MyVec2 v1, MyVec2 v2) => MyVec2.Add1(v1, v2);
    public static MyVec2 TestAdd2(MyVec2 v1, MyVec2 v2) => MyVec2.Add2(v1, v2);
}

public struct MyVec2
{
    public double X, Y;

    [MethodImpl(MethodImplOptions.AggressiveInlining)]
    public static MyVec2 Add1(MyVec2 left, MyVec2 right)
    {
        var tmp = Unsafe.As<MyVec2, Vector128<double>>(ref left) + Unsafe.As<MyVec2, Vector128<double>>(ref right);
        return Unsafe.As<Vector128<double>, MyVec2>(ref tmp);
    }

    [MethodImpl(MethodImplOptions.AggressiveInlining)]
    public static MyVec2 Add2(MyVec2 left, MyVec2 right)
    {
        var tmp = Unsafe.BitCast<MyVec2, Vector128<double>>(left) + Unsafe.BitCast<MyVec2, Vector128<double>>(right);
        return Unsafe.BitCast<Vector128<double>, MyVec2>(tmp);
    }
}

CQ of Add1 is worse than Add2 because Add2 introduces some natural copies that leads to avoided dependent promotion.

using System;
using System.Runtime.CompilerServices;
using System.Numerics;
using System.Runtime.Intrinsics;


public static class Tests
{
    [MethodImpl(MethodImplOptions.AggressiveInlining | MethodImplOptions.AggressiveOptimization)]
    private static Vector2<double> ByRefOperatorAdd(Vector2<double> v1, Vector2<double> v2)
    {
        return v1 + v2;
    }
    
    [MethodImpl(MethodImplOptions.AggressiveInlining | MethodImplOptions.AggressiveOptimization)]
    private static Vector2<double> ByRefMethodAdd(Vector2<double> v1, Vector2<double> v2)
    {
        return Vector2<double>.Add(in v1, in v2);
    }
    
    [MethodImpl(MethodImplOptions.AggressiveInlining | MethodImplOptions.AggressiveOptimization)]
    private static Vector2<double> ByValMethodAdd(Vector2<double> v1, Vector2<double> v2)
    {
        return Vector2<double>.Add2(v1, v2);
    }
    
    [MethodImpl(MethodImplOptions.AggressiveInlining | MethodImplOptions.AggressiveOptimization)]
    private static Vector2<double> ByRefMethodAdd2(Vector2<double> v1, Vector2<double> v2)
    {
        return Vector2<double>.Add3(in v1, in v2);
    }
    
    [MethodImpl(MethodImplOptions.AggressiveInlining | MethodImplOptions.AggressiveOptimization)]
    private static Vector2<double> ByValMethodAdd2(Vector2<double> v1, Vector2<double> v2)
    {
        return Vector2<double>.Add4(v1, v2);
    }
}

public static class Extensions
{
    [MethodImpl(MethodImplOptions.AggressiveInlining | MethodImplOptions.AggressiveOptimization)]
    public static ref readonly Vector128<T> AsVector128<T>(in this Vector2<T> vector) where T : unmanaged, INumber<T>
    {
        return ref Unsafe.As<Vector2<T>, Vector128<T>>(ref Unsafe.AsRef(in vector));
    }
    
    [MethodImpl(MethodImplOptions.AggressiveInlining | MethodImplOptions.AggressiveOptimization)]
    public static ref readonly Vector2<T> AsVector2<T>(in this Vector128<T> vector) where T : unmanaged, INumber<T>
    {
        return ref Unsafe.As<Vector128<T>, Vector2<T>>(ref Unsafe.AsRef(in vector));
    }
}

public readonly struct Vector2<T> where T : unmanaged, INumber<T>
{
    [MethodImpl(MethodImplOptions.AggressiveInlining | MethodImplOptions.AggressiveOptimization)]
    public static Vector2<T> operator +(in Vector2<T> left, in Vector2<T> right) => Add(in left, in right);
    
	public T X
    {
		[MethodImpl(MethodImplOptions.AggressiveInlining | MethodImplOptions.AggressiveOptimization)]
		get;
	}
    
	public T Y
    {
		[MethodImpl(MethodImplOptions.AggressiveInlining | MethodImplOptions.AggressiveOptimization)]
		get;
	}
    
    public Vector2(T x, T y)
    {
        X = x;
        Y = y;
    }
    
    [MethodImpl(MethodImplOptions.AggressiveInlining | MethodImplOptions.AggressiveOptimization)]
    public static Vector2<T> Add(in Vector2<T> left, in Vector2<T> right)
    {
        unsafe
        {
            // Total size = 128 bits
            if (sizeof(Vector2<T>) == 16)
            {
                if (Vector128<T>.IsSupported && Vector128.IsHardwareAccelerated)
                    return (left.AsVector128() + right.AsVector128()).AsVector2();
            }
        }

        return new(left.X + right.X, left.Y + right.Y);
    }
    
    [MethodImpl(MethodImplOptions.AggressiveInlining | MethodImplOptions.AggressiveOptimization)]
    public static Vector2<T> Add2(Vector2<T> left, Vector2<T> right)
    {
        unsafe
        {
            // Total size = 128 bits
            if (sizeof(Vector2<T>) == 16)
            {
                if (Vector128<T>.IsSupported && Vector128.IsHardwareAccelerated)
                    return (left.AsVector128() + right.AsVector128()).AsVector2();
            }
        }

        return new(left.X + right.X, left.Y + right.Y);
    }
    
    [MethodImpl(MethodImplOptions.AggressiveInlining | MethodImplOptions.AggressiveOptimization)]
    public static Vector2<T> Add3(in Vector2<T> left, in Vector2<T> right)
    {
        return new(left.X + right.X, left.Y + right.Y);
    }
    
    [MethodImpl(MethodImplOptions.AggressiveInlining | MethodImplOptions.AggressiveOptimization)]
    public static Vector2<T> Add4(Vector2<T> left, Vector2<T> right)
    {
        return new(left.X + right.X, left.Y + right.Y);
    }
}

ByRefMethodAdd is worse than ByValMethodAdd for the same reasons as above.

Both examples produce identical codegen between the by-ref/by-val versions under DOTNET_JitStressModeNames=STRESS_NO_OLD_PROMOTION (allowing physical promotion to take over). The work item here is figuring out how to detect the dependent promotion early enough to let physical promotion handle the scenario.

[JulieLeeMSFT]

Closing as .NET 9 work is complete.