Assertion failed 'IsCompatibleType(cseLclVarTyp, expTyp)'

[BruceForstall]

Failure in libraries System.Numerics.Vectors.Tests: release libraries, checked coreclr with:

set COMPlus_JitStress=1
set COMPlus_TieredCompilation=0

Windows x64

E.g.,

build -s libs+libs.tests -runtimeConfiguration checked -c Release -arch x64 /p:ArchiveTests=true

(the /p:ArchiveTests=true is required to build the RunTests.cmd wrapper scripts)

C:\gh\runtime\artifacts\bin\System.Numerics.Vectors.Tests\net5.0-Release>call .\RunTests.cmd -r C:\gh\runtime\artifacts\bin\testhost\net5.0-Windows_NT-Release-x64

Assert failure(PID 42620 [0x0000a67c], Thread: 42556 [0xa63c]): Assertion failed 'IsCompatibleType(cseLclVarTyp, expTyp)' in 'System.Numerics.Tests.Matrix3x2Tests:Matrix3x2CreateRotationCenterTest():this' during 'Optimize Valnum CSEs' (IL size 118)

    File: C:\gh\runtime\src\coreclr\src\jit\optcse.cpp Line: 2768
    Image: C:\gh\runtime\artifacts\bin\testhost\net5.0-Windows_NT-Release-x64\dotnet.exe

@briansull @tannergooding @CarolEidt @echesakovMSFT

[briansull]

I did:

build -s libs+libs.tests -runtimeConfiguration checked -c Release -arch x64 /p:ArchiveTests=true
_<takes a long time>_
cd artifacts\bin\System.Numerics.Vectors.Tests\net5.0-Release
call .\RunTests.cmd -r D:\gh\runtime\artifacts\bin\testhost\net5.0-Windows_NT-Release-x64
<this uses the release runtime located here:> artifacts\bin\testhost\net5.0-Windows_NT-Release-x64\shared\Microsoft.NETCore.App\5.0.0
I then copied the artifacts\tests\coreclr\Windows_NT.x64.Checked\Tests\Core_Root\clrjit.dll into this directory

[briansull]

I also edited src\libraries\System.Numerics.Vectors\Tests\Matrix3x2Tests.cs
and removed this line that was disabling the test case.

[SkipOnCoreClr("https://github.com/dotnet/runtime/issues/36587", RuntimeTestModes.JitStress)]

[briansull]

It didn't repro for me and I gathered a dump of the method which shows the random JitStress was active