Add AOT tests for NativeAOT CLI entry point - #54297
Conversation
Add test project dotnet-aot.Tests with 44 unit and integration tests covering the NativeAOT CLI entry point added in PR dotnet#54002. Refactoring for testability: - Extract DotnetRootResolver from dn/Program.cs with injectable deps - Extract NativeEntryPoint.ExecuteCore from [UnmanagedCallersOnly] Execute - Create AotSourceFiles.props shared source file list Test categories (44 tests): - AOT Parser (10): Parse/invoke --version, --info, default, errors - Path Resolution (13): DOTNET_ROOT env vars, arch-specific, walk-up, hostfxr discovery, version selection, cross-platform - Decision Logic (11): DOTNET_CLI_ENABLEAOT gating, fallback behavior, env var format variants, AppContext/hostfxr setup - Integration/E2E (6): Run actual dn binary (Assert.Skip if not built) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Adds a new test project dotnet-aot.Tests covering the NativeAOT CLI entry point from #54002, together with a small source refactor that extracts DotnetRootResolver and NativeEntryPoint.ExecuteCore for testability and introduces a shared AotSourceFiles.props import.
Changes:
- Extract
DotnetRootResolverfromdn/Program.csandExecuteCorefromNativeEntryPointto enable isolated unit testing with injected dependencies. - Share AOT compile-include list via
AotSourceFiles.propsimported by bothdotnet-aot.csprojand the new test project. - Add 44 unit/integration tests covering AOT parser, path resolution,
DOTNET_CLI_ENABLEAOTdecision logic, and end-to-enddninvocation (withAssert.Skipwhen binary unavailable).
Reviewed changes
Copilot reviewed 14 out of 14 changed files in this pull request and generated 12 comments.
Show a summary per file
| File | Description |
|---|---|
| src/Cli/dn/DotnetRootResolver.cs | New helper with injectable env/FS dependencies extracted from Program.cs. |
| src/Cli/dn/Program.cs | Delegates ResolveDotnetRoot/ResolveHostfxrPath to DotnetRootResolver. |
| src/Cli/dotnet-aot/NativeEntryPoint.cs | Splits native marshalling from new internal ExecuteCore. |
| src/Cli/dotnet-aot/AotSourceFiles.props | New shared MSBuild import for common AOT source files. |
| src/Cli/dotnet-aot/dotnet-aot.csproj | Imports the new props instead of inline <Compile> items. |
| sdk.slnx, cli.slnf | Register the new test project. |
| test/dotnet-aot.Tests/dotnet-aot.Tests.csproj | New test project, defines CLI_AOT, links shared sources. |
| test/dotnet-aot.Tests/AssemblyInfo.cs | Disables parallel execution at assembly level. |
| test/dotnet-aot.Tests/TestManagedHost.cs | Stub ManagedHost to satisfy NativeEntryPoint references. |
| test/dotnet-aot.Tests/AotParserTests.cs | Unit tests for --version/--info/default usage AOT parser. |
| test/dotnet-aot.Tests/DotnetRootResolverTests.cs | Unit tests for root and hostfxr resolution. |
| test/dotnet-aot.Tests/NativeEntryPointTests.cs | Tests for ExecuteCore decision logic and env var gating. |
| test/dotnet-aot.Tests/AotIntegrationTests.cs | End-to-end tests that exec the built dn binary. |
…s, cleanup
- Fix potential deadlock in AotIntegrationTests.RunDn by reading
stdout/stderr asynchronously before WaitForExit
- Rewrite DotnetRootResolverTests to use Path.Combine instead of
hardcoded Windows-style paths for cross-platform CI compatibility
- Remove [Collection("AOT")] attributes from AotParserTests and
NativeEntryPointTests (assembly-level DisableTestParallelization
already handles serialization)
- Remove unused IsDnAvailable helper, call FindDnPath directly
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
JeremyKuhne
left a comment
There was a problem hiding this comment.
We're going to want to set this up to run AOT'ed. That will require publishing and running the published test executable. This PR should probably have at least a script to publish and run the tests and show the results. We can follow up later with integration into the CIs.
Add the ability to publish and run dotnet-aot tests as a NativeAOT binary. - Add conditional PublishAotTests property to the test project that enables NativeAOT publishing with the necessary ILC warning suppressions - Add AotTestEntryPoint.cs: standalone smoke test runner (13 tests) that bypasses xUnit's runner which relies on Assembly.Location (empty in AOT) - Add run-aot-tests.ps1 and run-aot-tests.sh scripts that publish and run the native test binary end-to-end The AOT smoke tests cover Parser, NativeEntryPoint.ExecuteCore decision logic, and DotnetRootResolver path resolution. Normal dotnet test still uses the full xUnit test suite (44 tests) unchanged. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Added NativeAOT publish and run support in the latest commit (632b50b).
What was done:
- Added a conditional
PublishAotTests=trueMSBuild property to the test project that enables NativeAOT publishing - Added
AotTestEntryPoint.cs: a standalone AOT smoke test runner with 13 tests covering Parser,NativeEntryPoint.ExecuteCoredecision logic, and \DotnetRootResolver\ path resolution - Added
un-aot-tests.ps1andun-aot-tests.shscripts that auto-detect the RID, publish as NativeAOT, and run the native binary end-to-end
Usage:
./test/dotnet-aot.Tests/run-aot-tests.ps1
./test/dotnet-aot.Tests/run-aot-tests.sh
Why a custom test runner instead of xUnit: Both xUnit v3's in-process console runner and the Microsoft.Testing.Platform (MTP) runner crash when running as AOT-published binaries because they rely on Assembly.Location, which returns empty in NativeAOT single-file apps. Specifically:
- xUnit console runner:
error: assembly not found:(empty path fromAssembly.Location) - MTP runner:
System.ArgumentException: The path is empty.inTestPlatformTestFramework.RunAsync
The standalone smoke test runner bypasses this limitation by directly exercising the test scenarios without going through xUnit's discovery infrastructure. Once xUnit v3 adds an Assembly.Location fallback (e.g., using AppContext.BaseDirectory), we can switch back to their runner and remove the custom entry point.
Normal dotnet test (44 xUnit tests) continues to work unchanged.
|
@NikolaMilosavljevic AOT should work if you follow the guidance here: https://xunit.net/docs/getting-started/v3/native-aot |
Replace the custom AotTestEntryPoint smoke test runner with proper xUnit v3 AOT packages (xunit.v3.core.aot.mtp-v2 + xunit.v3.assert.aot) that use source generators for test discovery instead of reflection. Changes: - eng/XUnitV3/XUnitV3.targets: Add conditional AOT package switching when PublishAotTests=true (swaps xunit.v3.core → xunit.v3.core.aot.mtp-v2, xunit.v3.assert → xunit.v3.assert.aot) - dotnet-aot.Tests.csproj: Enable MTP runner for AOT, exclude transitive reflection-mode xUnit packages that conflict with AOT variants - NativeEntryPointTests.cs: Replace IDisposable with WithEnvRestore helper (xUnit v3 AOT source generator does not support IDisposable on test classes) - Delete AotTestEntryPoint.cs (custom runner no longer needed) All 44 tests pass when AOT-published (38 succeed, 6 integration tests skipped as expected). Normal dotnet build remains unaffected. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
NikolaMilosavljevic
left a comment
There was a problem hiding this comment.
Switched from the custom smoke test runner to proper xUnit v3 AOT packages per your suggestion.
What changed:
eng/XUnitV3/XUnitV3.targetsnow conditionally swaps inxunit.v3.core.aot.mtp-v2+xunit.v3.assert.aotwhenPublishAotTests=true(instead of the reflection-basedxunit.v3.core+xunit.v3.assert)- Deleted the custom
AotTestEntryPoint.csrunner — xUnit's AOT source generators handle test discovery - Had to exclude transitive reflection-mode xUnit packages (
xunit.v3.extensibility.core,xunit.v3.assert,xunit.v3.common) that flow fromMicrosoft.NET.TestFrameworkand conflict with the AOT variants - Replaced
IDisposableonNativeEntryPointTestswith aWithEnvRestorehelper — the xUnit v3 AOT source generator doesn't supportIDisposableon test classes (generates invalid async lambda code)
Results: All 44 tests pass when AOT-published (38 succeed, 6 integration tests skipped as expected). Normal dotnet build unaffected (0 warnings, 0 errors).
- Fix bash script: disable set -e around test execution so failure message is printed before exiting - Include Configuration/RID in publish directory to prevent stale binary execution across different configurations - Move publish dir assignment after RID auto-detection (PS1) - Narrow xUnit AOT package switching from PublishAotTests (global) to UseXUnitAotPackages (project opt-in) to avoid accidentally affecting other test projects Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
NikolaMilosavljevic
left a comment
There was a problem hiding this comment.
Switched to xUnit v3 AOT packages as you suggested -- the custom smoke test runner is gone.
Commit 1: Switch to xUnit v3 AOT packages
Replaced the custom AotTestEntryPoint.cs runner with proper xUnit v3 AOT packages (xunit.v3.core.aot.mtp-v2 + xunit.v3.assert.aot). These use source generators for test discovery instead of reflection, so all 40 standard xUnit tests ([Fact], [Theory], Assert.*, etc.) now work natively under AOT.
The main challenge was transitive package conflicts: Microsoft.NET.TestFramework flows reflection-mode xUnit packages (xunit.v3.extensibility.core, xunit.v3.assert, xunit.v3.common) that ship DLLs conflicting with the AOT variants. Fixed by adding ExcludeAssets="all" overrides for those packages when AOT-publishing.
Also had to replace IDisposable on NativeEntryPointTests with a WithEnvRestore helper -- the xUnit v3 AOT source generator generates invalid async lambda code for disposable test classes (likely a pre-release bug in v3 pre.81).
Commit 2: Review fixes
- Fixed a bug in
run-aot-tests.shwhereset -ewould cause the script to exit before printing the failure message when tests fail. - Added Configuration/RID to the publish output directory (
artifacts/aot-tests/Debug/win-x64/) to prevent accidentally running stale binaries from a different configuration or RID when using--no-build. - Narrowed the xUnit AOT package switching in
eng/XUnitV3/XUnitV3.targetsfrom keying onPublishAotTests(which could be passed globally and accidentally affect other test projects) to a project-specificUseXUnitAotPackagesproperty that onlydotnet-aot.Tests.csprojsets.
Results: 44 tests total -- 38 pass, 6 integration tests skipped (expected, they need the dn binary in the SDK layout). Normal dotnet build unaffected (0 warnings, 0 errors).
|
@JeremyKuhne - I've addressed issues you raised. All checks are passing and this is ready for re-review. |
JeremyKuhne
left a comment
There was a problem hiding this comment.
Thanks @NikolaMilosavljevic! Can you follow up with the xunit issue by looking for an existing tracking issue and opening one if it does not exist?
NikolaMilosavljevic
left a comment
There was a problem hiding this comment.
Filed xunit/xunit#3584 -- the xUnit v3 AOT source generator emits invalid async lambda code when a test class implements IDisposable, causing CS4010. No pre-existing issue was found. We're working around it with a WithEnvRestore helper using try/finally instead of IDisposable.
The merge with main (PR dotnet#54297) introduced AotSourceFiles.props, a shared source list imported by both dotnet-aot.csproj and the new dotnet-aot.Tests.csproj. The merge resolution left duplicate Compile items in dotnet-aot.csproj for files that are now provided by the props import (CommandLineInfo.cs, Parser.cs, EnvironmentVariableNames.cs), which would cause duplicate-item build failures. Since the shared Parser.cs references the 'sdk check' command, the test project (which compiles Parser.cs via the props) also needs the command's sources, resources, and package dependency. Move those into AotSourceFiles.props so both consumers stay in sync, and remove the now-duplicated entries from dotnet-aot.csproj. Also move the host-provided DotnetRoot state from Program.cs into the shared NativeEntryPoint.cs, because NativeEntryPoint.cs is compiled by the test project but Program.cs is not. This keeps the test project building without pulling in Program.cs (which defines a Main entry point). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add tests for NativeAOT CLI entry point (#54002)
Adds the
dotnet-aot.Testsproject with 44 unit and integration tests covering the NativeAOT CLI entry point introduced in #54002.Source refactoring for testability
DotnetRootResolver— movesResolveDotnetRoot()andResolveHostfxrPath()fromdn/Program.csinto a shared static class with injectable dependencies (environment, file system), enabling isolated unit testing.NativeEntryPoint.ExecuteCore— separates core execution logic from[UnmanagedCallersOnly] Executenative marshalling, making it callable from managed test code.AotSourceFiles.props— shared MSBuild import listing common source files (CommandLineInfo.cs,Parser.cs,EnvironmentVariableNames.cs), imported by bothdotnet-aot.csprojand the test project to keep file lists in sync.Test coverage (44 tests)
--version,--info, default command, unknown commands, error handlingDOTNET_ROOTenv vars, arch-specific variants (x64/x86/ARM64), walk-up directory search, hostfxr discovery, version selection, cross-platform pathsDOTNET_CLI_ENABLEAOTgating, managed fallback behavior, env var format variants (true/1/yes),AppContext/hostfxr setupdnbinary end-to-end (Assert.Skipif not built with NativeAOT)Results
dnbinary), 0 failed