JIT/Regression/JitBlue/Runtime_34587 crashes Crossgen2 compilation with an assertion failure

[trylek]

OS: all (observed on Windows and Linux)
Architecture: all (observed on x86 and x64, arm runs are somewhat flaky these days)
Example run: https://dev.azure.com/dnceng/public/_build/results?buildId=1552238&view=ms.vss-test-web.build-test-results-tab

Diagnostics:

  Starting:    JIT.Regression.XUnitWrapper (parallel test collections = on, max threads = 4)
    JIT\Regression\JitBlue\Runtime_34587\Runtime_34587\Runtime_34587.cmd [FAIL]
      Could Not Find C:\h\w\A2A20899\w\C2860A5B\e\JIT\Regression\JitBlue\Runtime_34587\Runtime_34587\IL-CG2\composite-r2r.dll
      Could Not Find C:\h\w\A2A20899\w\C2860A5B\e\JIT\Regression\JitBlue\Runtime_34587\Runtime_34587\Runtime_34587.dll.rsp
      Process terminated. Assertion failed.
         at ILCompiler.DependencyAnalysis.SortableDependencyNode.ObjectNodeComparer.Compare(DependencyNodeCore`1 x1, DependencyNodeCore`1 y1)
         at ILCompiler.Sorting.Implementation.MergeSortCore`5.Merge(T[] localCopyOfHalfOfArray, TDataStructure arrayToSort, Int32 index, Int32 halfLen, Int32 length, TComparer comparer)
         at ILCompiler.Sorting.Implementation.MergeSortCore`5.MergeSortHelper(TDataStructure arrayToSort, Int32 index, Int32 length, TComparer comparer, T[] scratchSpace)
         at ILCompiler.Sorting.Implementation.MergeSortCore`5.ParallelSortApi(TDataStructure arrayToSort, TComparer comparer)
         at ILCompiler.MergeSortApi.MergeSortAllowDuplicates[T](List`1 listToSort, IComparer`1 comparer)
         at ILCompiler.DependencyAnalysisFramework.DependencyAnalyzer`2.ComputeMarkedNodes()
         at ILCompiler.ReadyToRunCodegenCompilation.Compile(String outputFile)
         at ILCompiler.Program.RunSingleCompilation(Dictionary`2 inFilePaths, InstructionSetSupport instructionSetSupport, String compositeRootPath, Dictionary`2 unrootedInputFilePaths, HashSet`1 versionBubbleModulesHash, CompilerTypeSystemContext typeSystemContext)
         at ILCompiler.Program.Run(String[] args)
         at ILCompiler.Program.Main(String[] args)
      
      Return code:      1

This got broken after @tannergooding's change

4f89b79

but the change itself isn't to blame, it just uncovered a pre-existing bug in Crossgen2 regarding unstable ordering - we create two instances of ReadyToRunInstructionSetSupportSignature, one of which is the "baseline instruction set support" and the other is a PerMethodInstructionSetSupportFixup (for the method [Runtime_34587]Runtime_34587.g__ValidateVector256|1_11()) - both of which have the same key, "X86Base+Sse+Sse2,". The reason we need the instruction set fixup is the second loop (over _actualInstructionSetUnsupported) where the conditional block in the loop body is entered for the instruction set "ARM64_Rdm_Arm64" (while building for Windows x64 release):

runtime/src/coreclr/tools/Common/JitInterface/CorInfoImpl.cs

Line 450 in 2692407

needPerMethodInstructionSetFixup = true;

I have consulted @davidwrighton w.r.t. triage of this issue and we are in agreement that the proximate bug is in Crossgen2 that shouldn't generate a per-method instruction set fixup considering the instruction set is the same as the baseline. It's however unclear whether the root cause is truly in Crossgen2 or whether it might be caused by erroneous, imprecise or misinterpreted JIT reporting by means of the notifyInstructionSetUsage JitInterface API.

Thanks

Tomas

/cc @dotnet/crossgen-contrib @dotnet/jit-contrib

[MichalStrehovsky]

Duplicate of #63716?

[trylek]

For sure, thanks @MichalStrehovsky for pointing that out. I'm about to close the item 63716 as a duplicate of this one because I believe this item contains more triage details.

[tannergooding]

@trylek anything needed from me here or that I can assist with?

[trylek]

@tannergooding - I don't think so, your change just hit a previously unseen code path in Crossgen2. I think it would be ideal if David Wrighton could look into the issue as he originally implemented the instruction set manipulation in Crossgen2 so he probably has the best context regarding the exact semantics.