JIT: Assertion failed 'ins != INS_invalid' during 'Generate code'

[amanasifkhalid]

On Windows x64:

// Found by Antigen
// Reduced from 30.05 KB to 2.35 KB.


using System;
using System.Collections.Generic;
using System.Runtime.CompilerServices;
using System.Runtime.Intrinsics;
using System.Runtime.Intrinsics.Arm;
using System.Runtime.Intrinsics.X86;
using System.Numerics;
public class TestClass
{
    public struct S1
    {
    }
    public struct S2
    {
    }
    static Vector128<byte> s_v128_byte_17 = Vector128.Create((byte)74, 2, 1, 5, 1, 1, 74, 74, 2, 1, 1, 2, 5, 1, 74, 2);
    static Vector512<int> s_v512_int_41 = Vector512.Create(5);
    static Vector512<long> s_v512_long_43 = Vector512.CreateScalar((long)4);
    static Vector512<double> s_v512_double_46 = Vector512.Create(4, -4.971830985915493, -0.9789473684210527, -1.956043956043956, 2, 74.25, 1.0533333333333332, 4.033898305084746);
    static S1 s_s1_60 = new S1();
    static S2 s_s2_61 = new S2();
    float float_71 = -0.9876543f;
    Vector128<double> v128_double_86 = Vector128.Create(-0.9791666666666666);
    Vector512<ushort> v512_ushort_100 = Vector512.CreateScalar((ushort)74);
    Vector512<int> v512_int_101 = Vector512<int>.Zero;
    Vector512<long> v512_long_103 = Vector512.CreateScalar((long)5);
    private static List<string> toPrint = new List<string>();
    public Vector512<long> Method1(S1 p_s1_122, out float p_float_123, bool p_bool_124, out Vector512<int> p_v512_int_125, ref Vector512<ushort> p_v512_ushort_126, Vector128<byte> p_v128_byte_127, Vector128<double> p_v128_double_128, ref S2 p_s2_129)
    {
        unchecked
        {
            p_float_123 = 15+4;
            p_v512_int_125 = v512_int_101 = Vector512<int>.AllBitsSet * s_v512_int_41+ v512_int_101 | v512_int_101| v512_int_101& Vector512.AsInt32(s_v512_double_46 ^ Vector512<double>.AllBitsSet);
            return (15|4)* (s_v512_long_43 += 15*4* s_v512_long_43)^ v512_long_103 * s_v512_long_43+ (s_v512_long_43 += 15*4* s_v512_long_43);
        }
    }
    public void Method0()
    {
        unchecked
        {
            v512_long_103 = Method1(s_s1_60, out float_71, 15==4, out v512_int_101, ref v512_ushort_100, s_v128_byte_17 -= Vector128<byte>.AllBitsSet & s_v128_byte_17 ^ s_v128_byte_17, v128_double_86 += v128_double_86 & v128_double_86, ref s_s2_61);
            return;
        }
    }
    public static void Main(string[] args)
{}
    public static int Antigen()
    {
        new TestClass().Method0();
        return string.Join(Environment.NewLine, toPrint).GetHashCode();
    }
}
/*
Environment:

set DOTNET_JitStressProcedureSplitting=1
set DOTNET_TieredPGO_InstrumentedTierAlwaysOptimized=1
set DOTNET_JitRandomEdgeCounts=1
set DOTNET_JitCheckSynthesizedCounts=1
set DOTNET_TieredCompilation=0
set DOTNET_PreferredVectorBitWidth=512
set DOTNET_JitStressRegs=1
set DOTNET_JitThrowOnAssertionFailure=1
set DOTNET_LegacyExceptionHandling=1

Debug: 1335962276

Release: 0
JIT assert failed:
Assertion failed 'ins != INS_invalid' in 'TestClass:Method1(TestClass+S1,byref,ubyte,byref,byref,System.Runtime.Intrinsics.Vector128`1[ubyte],System.Runtime.Intrinsics.Vector128`1[double],byref):System.Runtime.Intrinsics.Vector512`1[long]:this' during 'Generate code' (IL size 189; hash 0x309db00f; FullOpts)

    File: D:\a\_work\1\s\src\coreclr\jit\hwintrinsiccodegenxarch.cpp Line: 389


*/

cc @dotnet/jit-contrib

[dotnet-policy-service]

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

[AndyAyersMS]

I'll take a look.

[AndyAyersMS]

Seems like it requires AVX512 ... also if it requires DOTNET_JitStressProcedureSplitting=1 then will move to 10.0.

[AndyAyersMS]

Only required config setting is DOTNET_TieredCompilation=0. So possibly a 9.0 issue.

[AndyAyersMS]

Generating: N041 (  9, 17) [000024] ----G------                   t24 = *  HWINTRINSIC simd64 int AndNot REG mm2 <l:$2d0, c:$2cf>

Assert failure(PID 6024 [0x00001788], Thread: 24824 [0x60f8]): Assertion failed 'ins != INS_invalid' in 'TestClass:Method1(TestClass+S1,byref,ubyte,byref,byref,System.Runtime.Intrinsics.Vector128`1[ubyte],System.Runtime.Intrinsics.Vector128`1[double],byref):System.Runtime.Intrinsics.Vector512`1[long]:this' during 'Generate code' (IL size 189; hash 0x309db00f; FullOpts)

we are trying to generate NI_AVX512DQ_AndNot

[AndyAyersMS]

Just upstream we have

N019 (  7, 15) [000023] ----G------                   t23 = *  HWINTRINSIC simd64 double Xor <l:$3c2, c:$3c3>
                                                            /--*  t361   simd64 
                                                            +--*  t23    simd64 
N020 (  9, 17) [000024] ----G------                   t24 = *  HWINTRINSIC simd64 int And <l:$2d0, c:$2cf>
                                                            /--*  t16    simd64 
                                                            +--*  t24    simd64 

in source this is

 ... v512_int_101& Vector512.AsInt32(s_v512_double_46 ^ Vector512<double>.AllBitsSet)

and it seems like the Vector512.AsInt32 effect is perhaps lost. Lower coalesces these to AndNot as if simd base type was double (taken from the Xor), and later we blow up looking for an instruction that can handle int.

[AndyAyersMS]

One possible fix is to pick up the simdBaseType from the surviving And node...

@tannergooding is this too simplistic?

[tannergooding]

and it seems like the Vector512.AsInt32 effect is perhaps lost

AsInt32 is a type of bitcast and is dropped as part of importation.

@tannergooding is this too simplistic?

There's a bit of a tradeoff here in how it impacts codegen and other downstream optimizations (like whether an embedded maski or embedded broadcast can be used).

I think for .NET 9 using the base type from userIntrin is the simpler option and the least likely to cause issues. I'll log an issue tracking a "better" fix that takes into account the optimizations for "future".