range_r work item: fail: Non-success instrumentation exit code: 101, expected: 0

[sbomer]

This test is failing in rolling builds (runtime-extra-platforms) on Mono Android x64 Release @ Ubuntu.1804.Amd64.Android.29.Open. range_d Work Item and range_r Work Item are both failing similarly. The helix logs aren't too helpful:

[10:41:16] info: Running instrumentation class net.dot.MonoRunner took 2.423214 seconds
[10:41:16] dbug: Exit code: 0
                 Standard Output:
                 INSTRUMENTATION_RESULT: return-code=101
                 INSTRUMENTATION_CODE: 101
                 
                 
[10:41:16] info: Instrumentation finished normally with exit code 101
[10:41:19] dbug: Executing command: '/datadisks/disk1/work/A720097B/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.22101.2/runtimes/any/native/adb/linux/adb -s emulator-5556 logcat -d '
[10:41:20] info: Wrote current ADB log to /datadisks/disk1/work/A720097B/w/A03B08E0/uploads/adb-logcat-net.dot.range_r-net.dot.MonoRunner.log
[10:41:20] fail: Non-success instrumentation exit code: 101, expected: 0
[10:41:20] info: Attempting to remove apk 'net.dot.range_r': 
[10:41:20] dbug: Executing command: '/datadisks/disk1/work/A720097B/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.22101.2/runtimes/any/native/adb/linux/adb -s emulator-5556 uninstall net.dot.range_r'
[10:41:20] info: Successfully uninstalled net.dot.range_r.
[10:41:20] dbug: Saving diagnostics data to '/datadisks/disk1/work/A720097B/w/A03B08E0/e/diagnostics.json'
XHarness exit code: 1 (TESTS_FAILED)

Do we need to look at the adb logs (does anyone know how to access them)? @radical who would be the best person to look into this?

Runfo Tracking Issue: range_r work item

Build	Definition	Kind	Run Name	Console	Core Dump	Test Results	Run Client
1597265	runtime-extra-platforms	PR 60879	Mono Android x64 Release @ Ubuntu.1804.Amd64.Android.29.Open	console.log			runclient.py
1597254	runtime-extra-platforms	PR 64806	Mono Android x64 Release @ Ubuntu.1804.Amd64.Android.29.Open	console.log			runclient.py
1596968	runtime-extra-platforms	PR 64330	Mono Android x64 Release @ Ubuntu.1804.Amd64.Android.29.Open	console.log			runclient.py
1596825	runtime-extra-platforms	PR 64567	Mono Android x64 Release @ Ubuntu.1804.Amd64.Android.29.Open	console.log			runclient.py
1596825	runtime-extra-platforms	PR 64567	Mono Android x64 Release @ Ubuntu.1804.Amd64.Android.29.Open	console.log			runclient.py

Build Result Summary

Day Hit Count	Week Hit Count	Month Hit Count
4	4	4

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[radical]

cc @fanyang-mono @steveisok

Tagging subscribers to 'arch-android': @steveisok, @akoeplinger
See info in area-owners.md if you want to be subscribed.

Issue Details

---

This test is failing in rolling builds (runtime-extra-platforms) on Mono Android x64 Release @ Ubuntu.1804.Amd64.Android.29.Open. range_d Work Item and range_r Work Item are both failing similarly. The helix logs aren't too helpful:

[10:41:16] info: Running instrumentation class net.dot.MonoRunner took 2.423214 seconds
[10:41:16] dbug: Exit code: 0
                 Standard Output:
                 INSTRUMENTATION_RESULT: return-code=101
                 INSTRUMENTATION_CODE: 101
                 
                 
[10:41:16] info: Instrumentation finished normally with exit code 101
[10:41:19] dbug: Executing command: '/datadisks/disk1/work/A720097B/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.22101.2/runtimes/any/native/adb/linux/adb -s emulator-5556 logcat -d '
[10:41:20] info: Wrote current ADB log to /datadisks/disk1/work/A720097B/w/A03B08E0/uploads/adb-logcat-net.dot.range_r-net.dot.MonoRunner.log
[10:41:20] fail: Non-success instrumentation exit code: 101, expected: 0
[10:41:20] info: Attempting to remove apk 'net.dot.range_r': 
[10:41:20] dbug: Executing command: '/datadisks/disk1/work/A720097B/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.22101.2/runtimes/any/native/adb/linux/adb -s emulator-5556 uninstall net.dot.range_r'
[10:41:20] info: Successfully uninstalled net.dot.range_r.
[10:41:20] dbug: Saving diagnostics data to '/datadisks/disk1/work/A720097B/w/A03B08E0/e/diagnostics.json'
XHarness exit code: 1 (TESTS_FAILED)

Do we need to look at the adb logs (does anyone know how to access them)? @radical who would be the best person to look into this?

Author:	sbomer
Assignees:	-
Labels:	os-android, untriaged
Milestone:	-

[steveisok]

@fanyang-mono This is a runtime test failure, right?

[elinor-fung]

Recent adb log: https://helixre8s23ayyeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-heads-main-a460fb376262457b8d/range_d/1/adb-logcat-net.dot.range_d-net.dot.MonoRunner.log?sv=2019-07-07&se=2022-02-27T10%3A24%3A36Z&sr=c&sp=rl&sig=N3g9E4zzBWoG%2BpO%2FLhYuhC%2FCJqxy3B%2FM79XNNx8wKLw%3D

02-07 10:41:16.954  9249  9268 I DOTNET  : System.ArgumentException: DOCSDIR should not be empty
02-07 10:41:16.954  9249  9268 I DOTNET  :    at XHarnessRunnerLibrary.RunnerEntryPoint.AndroidEntryPoint.get_TestsResultsFinalPath() in /__w/1/s/src/tests/Common/XHarnessRunnerLibrary/RunnerEntryPoint.cs:line 103
02-07 10:41:16.954  9249  9268 I DOTNET  :    at Microsoft.DotNet.XHarness.TestRunners.Common.AndroidApplicationEntryPointBase.RunAsync()
02-07 10:41:16.954  9249  9268 I DOTNET  :    at XHarnessRunnerLibrary.RunnerEntryPoint.RunTests(Func`2 runTestsCallback, String assemblyName, String filter) in /__w/1/s/src/tests/Common/XHarnessRunnerLibrary/RunnerEntryPoint.cs:line 33
02-07 10:41:16.954  9249  9268 I DOTNET  :    at Program.<Main>$(String[] args) in /__w/1/s/src/tests/JIT/Methodical/Arrays/XUnitWrapperGenerator/XUnitWrapperGenerator.XUnitWrapperGenerator/XHarnessRunner.g.cs:line 3
02-07 10:41:16.955  9249  9268 D DOTNET  : Exit code: 101.
02-07 10:41:16.956  9249  9268 I DOTNET  : MonoRunner finished, return-code=101

It seems like #64744 updated the runner for libraries tests (src/libraries/Common/tests/AndroidTestRunner/AndroidTestRunner.cs) to stop using DOCSDIR. Do we need the equivalent change for runtime tests? cc @akoeplinger

[akoeplinger]

Interesting, I wasn't aware that the runtime tests are not using the shared runner code. Sent a fix: #64950