Skip to content

[Android] Run CoreCLR library tests on Android #114148

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 56 commits into from
May 21, 2025
Merged

[Android] Run CoreCLR library tests on Android #114148

merged 56 commits into from
May 21, 2025

Conversation

kotlarmilos
Copy link
Member

@kotlarmilos kotlarmilos commented Apr 2, 2025 •
edited
Loading

Description

This PR enables running CoreCLR library tests on the Android emulators and devices and adds a script for local testing on Windows.

Changes

  • Disabled library tests that require assemblies on disk
  • Disabled library tests that are failing on CoreCLR Android
  • Re-enabled passing tests on Android
  • Added a script for local testing on Windows

Disabled tests fall into two categories.

Globalization or other mobile‐platform issues

These tests have been disabled because they depend on invariant‐culture behaviors or platform APIs that aren’t fully supported on mobile targets:

System.Runtime\tests\System.Globalization.Tests\Invariant\Invariant.Tests.csproj
System.Net.Http\tests\FunctionalTests\System.Net.Http.Functional.Tests.csproj
System.Diagnostics.DiagnosticSource\tests\TestWithConfigSwitches\System.Diagnostics.DiagnosticSource.Switches.Tests.csproj
System.Reflection.MetadataLoadContext\tests\System.Reflection.MetadataLoadContext.Tests.csproj
System.Runtime\tests\System.Resources.ResourceManager.Tests\System.Resources.ResourceManager.Tests.csproj
System.Runtime\tests\System.Runtime.Tests\InvariantTimezone\System.Runtime.InvariantTimezone.Tests.csproj
System.Runtime.Loader\tests\System.Runtime.Loader.Tests.csproj
System.Runtime\tests\System.Text.Encoding.Tests\System.Text.Encoding.Tests.csproj
System.Runtime\tests\System.IO.Tests\System.IO.Tests.csproj
Microsoft.Bcl.Cryptography\tests\Microsoft.Bcl.Cryptography.Tests.csproj
System.Net.Security\tests\FunctionalTests\System.Net.Security.Tests.csproj
System.Net.WebSockets\tests\System.Net.WebSockets.Tests.csproj

CoreCLR Android-specific failures

The following disabled tests are specific to CoreCLR Android:

Microsoft.VisualBasic.Tests.ErrObjectTests.Clear
MonoTests.System.Configuration.mapped_ExeConfiguration_null
MonoTests.System.Configuration.OpenExeConfiguration1_UserLevel_None
System.Threading.Tests.MutexTests - add a process-wide implementation of named mutex to CoreCLR on Mono to better compatibility with Mono
System.IO.Pipelines.Tests.CompleteWithLargeWriteThrows

Closes #50871 #50874 #50923 #49937 #114148

@kotlarmilos kotlarmilos added this to the 10.0.0 milestone Apr 2, 2025
@kotlarmilos kotlarmilos self-assigned this Apr 2, 2025
@kotlarmilos kotlarmilos added NO-MERGE The PR is not ready for merge yet (see discussion for detailed reasons) NO-REVIEW Experimental/testing PR, do NOT review it labels Apr 2, 2025
This was referenced Apr 2, 2025
Closed
Closed
Open
Open
Open
Open
Open
@kotlarmilos
Copy link
Member Author

/azp run runtime-extra-platforms

@azure-pipelines Azure Pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

This was referenced Apr 4, 2025
Open
Open
Open
Open
Open
Open
Closed
Closed
Closed
Closed
Closed
Closed
@build-analysis build-analysis bot mentioned this pull request May 15, 2025
Open
3 tasks
@kotlarmilos
Copy link
Member Author

/azp run runtime-android,runtime-androidemulator

@azure-pipelines Azure Pipelines
Copy link

Azure Pipelines successfully started running 2 pipeline(s).

@kotlarmilos
Copy link
Member Author

/azp run runtime-android,runtime-androidemulator

@azure-pipelines Azure Pipelines
Copy link

Azure Pipelines successfully started running 2 pipeline(s).

This was referenced May 18, 2025
Open
Open
@kotlarmilos kotlarmilos linked an issue May 19, 2025 that may be closed by this pull request
[Android][mono] Missing libSystem.Security.Cryptography.Native.Android.so causes JIT.Static.Test crashes across all architectures #115692
Closed
@kotlarmilos
Copy link
Member Author

/azp run runtime-android,runtime-androidemulator

@azure-pipelines Azure Pipelines
Copy link

Azure Pipelines successfully started running 2 pipeline(s).

@kotlarmilos
Copy link
Member Author

This PR is ready to be merged.

/cc: @steveisok Build machines run out of disk space without symbol stripping. Enabling symbol stripping makes the app non-debuggable and breaks adb shell run-as access to internal storage where test results are stored. For test infrastructure, we need both: debuggable builds and stripped binaries. I created tracking issue that will be done as follow-up: #115717

@kotlarmilos
Copy link
Member Author

/azp run runtime-android,runtime-androidemulator

@azure-pipelines Azure Pipelines
Copy link

Azure Pipelines successfully started running 2 pipeline(s).

@kotlarmilos
Copy link
Member Author

/azp run runtime-android,runtime-androidemulator

@azure-pipelines Azure Pipelines
Copy link

Azure Pipelines successfully started running 2 pipeline(s).

@kotlarmilos
Copy link
Member Author

/azp run runtime-android,runtime-androidemulator

@azure-pipelines Azure Pipelines
Copy link

Azure Pipelines successfully started running 2 pipeline(s).

@build-analysis build-analysis bot mentioned this pull request May 21, 2025
Open
@kotlarmilos kotlarmilos merged commit f9ad7c9 into dotnet:main May 21, 2025
165 of 168 checks passed
@kotlarmilos
Copy link
Member Author

Follow-up work tracked in #114951 and #115717.

@jkotas jkotas mentioned this pull request May 24, 2025
Closed
SimaTian pushed a commit that referenced this pull request May 27, 2025
@kotlarmilos @steveisok @SimaTian
[Android] Run CoreCLR library tests on Android (#114148)
854a119 
* Enable running library tests

* Add jobs to extra-platforms

* Fix System.Runtime tests

* Fix arm64 android coreclr tests

* Add TestUtilities project reference

* Add support for local android test execution on windows

* Disable failing tests on Android devices

* Disable failing tests on Android devices

* Disable failing tests

* Remove unnecessary using System directive

* Disable failing tests

* Disable failing tests

* Fix typo

* Disable failing tests

* Disable ConfigSwitchIsHonored test

* Disable failing tests

* Fix typo

* Enable System.Runtime.Loader.Tests tests

* Enable System.Text.RegularExpressions tests

* Disable failing tests

* Try run disabled tests

* Fix build

* Disable failing tests

* Enable tests that are passing

* Fix typo

* Fix ActiveIssue annotation

* Disable failing tests

* Add comment

* Disable failing tests

* Fix test annotations

* Update ProjectExclusions tracking issue

* Disable failing tests on Android platform

* Skip MutexTests on CoreCLR Android

* Disable tests on CoreCLR Android

* Fix ActiveIssue annotation

* Disable failing tests

* Disable failing tests

* Exclude System.Net.Security.Tests from project builds

* Exclude System.Net.Security.Tests from project builds

* Disable CoreCLR tests on Mono

* Disable System.Net.WebSockets.Tests due to disk space limitation

* Exclude debug symbols

* Update StripDebugSymbols property

* Disable failing tests

* Fix build

* Disable tests

* Disable failing tests

* Test run

* Use Debug config to ensure android:debuggable=true

* Fix StripDebugSymbols

* Fix formatting

* Test run

---------

Co-authored-by: Steve Pfister <stpfiste@microsoft.com>
@kotlarmilos kotlarmilos mentioned this pull request May 28, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jkotas jkotas jkotas left review comments

@matouskozak matouskozak matouskozak left review comments

Copilot code review Copilot Copilot left review comments

@ivanpovazan ivanpovazan ivanpovazan approved these changes

@steveisok steveisok Awaiting requested review from steveisok steveisok is a code owner

@akoeplinger akoeplinger Awaiting requested review from akoeplinger

@vitek-karas vitek-karas Awaiting requested review from vitek-karas vitek-karas is a code owner

Assignees

@kotlarmilos kotlarmilos

Labels
area-Infrastructure-coreclr os-android
Projects
Runtime Infra
Status: Done
Milestone
10.0.0
6 participants
@kotlarmilos @am11 @steveisok @jkotas @ivanpovazan @matouskozak