Mono Pri0 Runtime Tests Run Linux arm64 release leg failing

[kunalspathak]

Looks like Mono Pri0 Runtime Tests Run Linux arm64 release leg started failing recently in JIT.HardwareIntrinsics test. See these errors. I noticed this failure in at least 4 PRs. #37942, #37912, #37941 and #37934.

@tannergooding made a change today to split the tests under ARM HWIntrinsic in #37859. Is that related? I don't see this leg ran as part of this PR, so may be that's why it was not caught earlier.

[Dotnet-GitSync-Bot]

I couldn't figure out the best area label to add to this issue. Please help me learn by adding exactly one area label.

[kunalspathak]

@tannergooding , @BruceForstall , @danmosemsft

[tannergooding]

I'd be very surprised if splitting the project into 5 suddenly started causing Mono to fail...

[benaadams]

Mono Linux arm64 Release @ (Ubuntu.1804.Arm64.Open)

/root/helix/work/workitem/JIT/HardwareIntrinsics/Arm/AdvSimd/AdvSimd_Part0_ro/AdvSimd_Part0_ro.sh: line 317: 650

/root/helix/work/workitem/JIT/HardwareIntrinsics/Arm/AdvSimd/AdvSimd_Part4_ro/AdvSimd_Part4_ro.sh: line 317: 1031 Segmentation fault $LAUNCHER $ExePath "${CLRTestExecutionArguments[@]}"

Return code: 1
Raw output file: /root/helix/work/workitem/JIT/HardwareIntrinsics/Reports/JIT.HardwareIntrinsics/Arm/AdvSimd/AdvSimd_Part4_ro/AdvSimd_Part4_ro.output.txt
Raw output:
BEGIN EXECUTION
/root/helix/work/correlation/corerun AdvSimd_Part4_ro.dll ''
Supported ISAs:
 AdvSimd: False
 Aes: False
 ArmBase: False
 Crc32: False
 Sha1: False
 Sha256: False

Beginning test case ShiftRightLogicalNarrowingLower.Vector64.UInt32.1 at 6/16/2020 4:21:48 AM
Random seed: 20010415; set environment variable CORECLR_SEED to this value to repro

Beginning scenario: RunUnsupportedScenario
Beginning scenario: RunBasicScenario_UnsafeRead
Expected: 100
Actual: 139
END EXECUTION - FAILED
Test Harness Exitcode is : 1
To run the test:
> set CORE_ROOT=/root/helix/work/correlation
> /root/helix/work/workitem/JIT/HardwareIntrinsics/Arm/AdvSimd/AdvSimd_Part4_ro/AdvSimd_Part4_ro.sh
Expected: True
Actual: False

at JIT_HardwareIntrinsics._Arm_AdvSimd_AdvSimd_Part4_ro_AdvSimd_Part4_ro_._Arm_AdvSimd_AdvSimd_Part4_ro_AdvSimd_Part4_ro_sh()
 in /__w/1/s/artifacts/tests/coreclr/Linux.arm64.Release/TestWrappers/JIT.HardwareIntrinsics/JIT.HardwareIntrinsics.XUnitWrapper.cs:line 1401

[tannergooding]

Ah, they are timing out: https://helix.dot.net/api/2019-06-17/jobs/5ce10ff8-b722-4533-8535-485f9336e94a/workitems/JIT.HardwareIntrinsics/files/console.b6522eeb.log

cmdLine:/root/helix/work/workitem/JIT/HardwareIntrinsics/Arm/AdvSimd/AdvSimd_Part2_ro/AdvSimd_Part2_ro.sh Timed Out

I wonder why splitting them is causing a timeout now... Maybe Mono was special casing the previous test?

[tannergooding]

Yes. They were excluded here: https://github.com/dotnet/runtime/blob/master/src/coreclr/tests/issues.targets#L2575

This just needs to be updated to handle the new names, will have an update shortly.

[safern]

I don't see this leg ran as part of this PR, so may be that's why it was not caught earlier.

I put up a PR to fix this, the condition for that leg was wrong: #37985

EDIT: @naricc did the same thing: #37916