Skip to content

Add AOT tests for NativeAOT CLI entry point - #54297

Merged
NikolaMilosavljevic merged 6 commits into
dotnet:mainfrom
NikolaMilosavljevic:aot.tests
May 28, 2026
Merged

Add AOT tests for NativeAOT CLI entry point#54297
NikolaMilosavljevic merged 6 commits into
dotnet:mainfrom
NikolaMilosavljevic:aot.tests

Conversation

@NikolaMilosavljevic

Copy link
Copy Markdown
Contributor

Add tests for NativeAOT CLI entry point (#54002)

Adds the dotnet-aot.Tests project with 44 unit and integration tests covering the NativeAOT CLI entry point introduced in #54002.

Source refactoring for testability

  • Extract DotnetRootResolver — moves ResolveDotnetRoot() and ResolveHostfxrPath() from dn/Program.cs into a shared static class with injectable dependencies (environment, file system), enabling isolated unit testing.
  • Extract NativeEntryPoint.ExecuteCore — separates core execution logic from [UnmanagedCallersOnly] Execute native marshalling, making it callable from managed test code.
  • Create AotSourceFiles.props — shared MSBuild import listing common source files (CommandLineInfo.cs, Parser.cs, EnvironmentVariableNames.cs), imported by both dotnet-aot.csproj and the test project to keep file lists in sync.

Test coverage (44 tests)

Category Tests What's covered
AOT Parser 10 --version, --info, default command, unknown commands, error handling
Path Resolution 13 DOTNET_ROOT env vars, arch-specific variants (x64/x86/ARM64), walk-up directory search, hostfxr discovery, version selection, cross-platform paths
Decision Logic 11 DOTNET_CLI_ENABLEAOT gating, managed fallback behavior, env var format variants (true/1/yes), AppContext/hostfxr setup
Integration/E2E 6 Run actual dn binary end-to-end (Assert.Skip if not built with NativeAOT)

Results

  • 38 passed, 6 skipped (integration tests requiring published dn binary), 0 failed

Add test project dotnet-aot.Tests with 44 unit and integration tests
covering the NativeAOT CLI entry point added in PR dotnet#54002.

Refactoring for testability:
- Extract DotnetRootResolver from dn/Program.cs with injectable deps
- Extract NativeEntryPoint.ExecuteCore from [UnmanagedCallersOnly] Execute
- Create AotSourceFiles.props shared source file list

Test categories (44 tests):
- AOT Parser (10): Parse/invoke --version, --info, default, errors
- Path Resolution (13): DOTNET_ROOT env vars, arch-specific, walk-up,
  hostfxr discovery, version selection, cross-platform
- Decision Logic (11): DOTNET_CLI_ENABLEAOT gating, fallback behavior,
  env var format variants, AppContext/hostfxr setup
- Integration/E2E (6): Run actual dn binary (Assert.Skip if not built)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new test project dotnet-aot.Tests covering the NativeAOT CLI entry point from #54002, together with a small source refactor that extracts DotnetRootResolver and NativeEntryPoint.ExecuteCore for testability and introduces a shared AotSourceFiles.props import.

Changes:

  • Extract DotnetRootResolver from dn/Program.cs and ExecuteCore from NativeEntryPoint to enable isolated unit testing with injected dependencies.
  • Share AOT compile-include list via AotSourceFiles.props imported by both dotnet-aot.csproj and the new test project.
  • Add 44 unit/integration tests covering AOT parser, path resolution, DOTNET_CLI_ENABLEAOT decision logic, and end-to-end dn invocation (with Assert.Skip when binary unavailable).

Reviewed changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 12 comments.

Show a summary per file
File Description
src/Cli/dn/DotnetRootResolver.cs New helper with injectable env/FS dependencies extracted from Program.cs.
src/Cli/dn/Program.cs Delegates ResolveDotnetRoot/ResolveHostfxrPath to DotnetRootResolver.
src/Cli/dotnet-aot/NativeEntryPoint.cs Splits native marshalling from new internal ExecuteCore.
src/Cli/dotnet-aot/AotSourceFiles.props New shared MSBuild import for common AOT source files.
src/Cli/dotnet-aot/dotnet-aot.csproj Imports the new props instead of inline <Compile> items.
sdk.slnx, cli.slnf Register the new test project.
test/dotnet-aot.Tests/dotnet-aot.Tests.csproj New test project, defines CLI_AOT, links shared sources.
test/dotnet-aot.Tests/AssemblyInfo.cs Disables parallel execution at assembly level.
test/dotnet-aot.Tests/TestManagedHost.cs Stub ManagedHost to satisfy NativeEntryPoint references.
test/dotnet-aot.Tests/AotParserTests.cs Unit tests for --version/--info/default usage AOT parser.
test/dotnet-aot.Tests/DotnetRootResolverTests.cs Unit tests for root and hostfxr resolution.
test/dotnet-aot.Tests/NativeEntryPointTests.cs Tests for ExecuteCore decision logic and env var gating.
test/dotnet-aot.Tests/AotIntegrationTests.cs End-to-end tests that exec the built dn binary.

Comment thread test/dotnet-aot.Tests/AotIntegrationTests.cs Outdated
Comment thread test/dotnet-aot.Tests/AotIntegrationTests.cs Outdated
Comment thread test/dotnet-aot.Tests/AotIntegrationTests.cs
Comment thread test/dotnet-aot.Tests/AotIntegrationTests.cs
Comment thread test/dotnet-aot.Tests/NativeEntryPointTests.cs Outdated
Comment thread test/dotnet-aot.Tests/NativeEntryPointTests.cs Outdated
Comment thread test/dotnet-aot.Tests/DotnetRootResolverTests.cs
Comment thread test/dotnet-aot.Tests/DotnetRootResolverTests.cs
Comment thread test/dotnet-aot.Tests/AotParserTests.cs
Comment thread test/dotnet-aot.Tests/dotnet-aot.Tests.csproj
NikolaMilosavljevic and others added 2 commits May 13, 2026 15:20
…s, cleanup

- Fix potential deadlock in AotIntegrationTests.RunDn by reading
  stdout/stderr asynchronously before WaitForExit
- Rewrite DotnetRootResolverTests to use Path.Combine instead of
  hardcoded Windows-style paths for cross-platform CI compatibility
- Remove [Collection("AOT")] attributes from AotParserTests and
  NativeEntryPointTests (assembly-level DisableTestParallelization
  already handles serialization)
- Remove unused IsDnAvailable helper, call FindDnPath directly

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

@JeremyKuhne JeremyKuhne left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're going to want to set this up to run AOT'ed. That will require publishing and running the published test executable. This PR should probably have at least a script to publish and run the tests and show the results. We can follow up later with integration into the CIs.

Comment thread test/dotnet-aot.Tests/dotnet-aot.Tests.csproj
Add the ability to publish and run dotnet-aot tests as a NativeAOT binary.

- Add conditional PublishAotTests property to the test project that enables
  NativeAOT publishing with the necessary ILC warning suppressions
- Add AotTestEntryPoint.cs: standalone smoke test runner (13 tests) that
  bypasses xUnit's runner which relies on Assembly.Location (empty in AOT)
- Add run-aot-tests.ps1 and run-aot-tests.sh scripts that publish and run
  the native test binary end-to-end

The AOT smoke tests cover Parser, NativeEntryPoint.ExecuteCore decision logic,
and DotnetRootResolver path resolution. Normal dotnet test still uses the full
xUnit test suite (44 tests) unchanged.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

@NikolaMilosavljevic NikolaMilosavljevic left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added NativeAOT publish and run support in the latest commit (632b50b).

What was done:

  • Added a conditional PublishAotTests=true MSBuild property to the test project that enables NativeAOT publishing
  • Added AotTestEntryPoint.cs: a standalone AOT smoke test runner with 13 tests covering Parser, NativeEntryPoint.ExecuteCore decision logic, and \DotnetRootResolver\ path resolution
  • Added un-aot-tests.ps1 and un-aot-tests.sh scripts that auto-detect the RID, publish as NativeAOT, and run the native binary end-to-end

Usage:

./test/dotnet-aot.Tests/run-aot-tests.ps1
./test/dotnet-aot.Tests/run-aot-tests.sh

Why a custom test runner instead of xUnit: Both xUnit v3's in-process console runner and the Microsoft.Testing.Platform (MTP) runner crash when running as AOT-published binaries because they rely on Assembly.Location, which returns empty in NativeAOT single-file apps. Specifically:

  • xUnit console runner: error: assembly not found: (empty path from Assembly.Location)
  • MTP runner: System.ArgumentException: The path is empty. in TestPlatformTestFramework.RunAsync

The standalone smoke test runner bypasses this limitation by directly exercising the test scenarios without going through xUnit's discovery infrastructure. Once xUnit v3 adds an Assembly.Location fallback (e.g., using AppContext.BaseDirectory), we can switch back to their runner and remove the custom entry point.

Normal dotnet test (44 xUnit tests) continues to work unchanged.

@JeremyKuhne

Copy link
Copy Markdown
Member

@NikolaMilosavljevic AOT should work if you follow the guidance here: https://xunit.net/docs/getting-started/v3/native-aot

Replace the custom AotTestEntryPoint smoke test runner with proper xUnit v3
AOT packages (xunit.v3.core.aot.mtp-v2 + xunit.v3.assert.aot) that use
source generators for test discovery instead of reflection.

Changes:
- eng/XUnitV3/XUnitV3.targets: Add conditional AOT package switching when
  PublishAotTests=true (swaps xunit.v3.core → xunit.v3.core.aot.mtp-v2,
  xunit.v3.assert → xunit.v3.assert.aot)
- dotnet-aot.Tests.csproj: Enable MTP runner for AOT, exclude transitive
  reflection-mode xUnit packages that conflict with AOT variants
- NativeEntryPointTests.cs: Replace IDisposable with WithEnvRestore helper
  (xUnit v3 AOT source generator does not support IDisposable on test classes)
- Delete AotTestEntryPoint.cs (custom runner no longer needed)

All 44 tests pass when AOT-published (38 succeed, 6 integration tests
skipped as expected). Normal dotnet build remains unaffected.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

@NikolaMilosavljevic NikolaMilosavljevic left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Switched from the custom smoke test runner to proper xUnit v3 AOT packages per your suggestion.

What changed:

  • eng/XUnitV3/XUnitV3.targets now conditionally swaps in xunit.v3.core.aot.mtp-v2 + xunit.v3.assert.aot when PublishAotTests=true (instead of the reflection-based xunit.v3.core + xunit.v3.assert)
  • Deleted the custom AotTestEntryPoint.cs runner — xUnit's AOT source generators handle test discovery
  • Had to exclude transitive reflection-mode xUnit packages (xunit.v3.extensibility.core, xunit.v3.assert, xunit.v3.common) that flow from Microsoft.NET.TestFramework and conflict with the AOT variants
  • Replaced IDisposable on NativeEntryPointTests with a WithEnvRestore helper — the xUnit v3 AOT source generator doesn't support IDisposable on test classes (generates invalid async lambda code)

Results: All 44 tests pass when AOT-published (38 succeed, 6 integration tests skipped as expected). Normal dotnet build unaffected (0 warnings, 0 errors).

- Fix bash script: disable set -e around test execution so failure
  message is printed before exiting
- Include Configuration/RID in publish directory to prevent stale
  binary execution across different configurations
- Move publish dir assignment after RID auto-detection (PS1)
- Narrow xUnit AOT package switching from PublishAotTests (global) to
  UseXUnitAotPackages (project opt-in) to avoid accidentally affecting
  other test projects

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

@NikolaMilosavljevic NikolaMilosavljevic left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Switched to xUnit v3 AOT packages as you suggested -- the custom smoke test runner is gone.

Commit 1: Switch to xUnit v3 AOT packages

Replaced the custom AotTestEntryPoint.cs runner with proper xUnit v3 AOT packages (xunit.v3.core.aot.mtp-v2 + xunit.v3.assert.aot). These use source generators for test discovery instead of reflection, so all 40 standard xUnit tests ([Fact], [Theory], Assert.*, etc.) now work natively under AOT.

The main challenge was transitive package conflicts: Microsoft.NET.TestFramework flows reflection-mode xUnit packages (xunit.v3.extensibility.core, xunit.v3.assert, xunit.v3.common) that ship DLLs conflicting with the AOT variants. Fixed by adding ExcludeAssets="all" overrides for those packages when AOT-publishing.

Also had to replace IDisposable on NativeEntryPointTests with a WithEnvRestore helper -- the xUnit v3 AOT source generator generates invalid async lambda code for disposable test classes (likely a pre-release bug in v3 pre.81).

Commit 2: Review fixes

  • Fixed a bug in run-aot-tests.sh where set -e would cause the script to exit before printing the failure message when tests fail.
  • Added Configuration/RID to the publish output directory (artifacts/aot-tests/Debug/win-x64/) to prevent accidentally running stale binaries from a different configuration or RID when using --no-build.
  • Narrowed the xUnit AOT package switching in eng/XUnitV3/XUnitV3.targets from keying on PublishAotTests (which could be passed globally and accidentally affect other test projects) to a project-specific UseXUnitAotPackages property that only dotnet-aot.Tests.csproj sets.

Results: 44 tests total -- 38 pass, 6 integration tests skipped (expected, they need the dn binary in the SDK layout). Normal dotnet build unaffected (0 warnings, 0 errors).

@NikolaMilosavljevic

Copy link
Copy Markdown
Contributor Author

@JeremyKuhne - I've addressed issues you raised. All checks are passing and this is ready for re-review.

@JeremyKuhne JeremyKuhne left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @NikolaMilosavljevic! Can you follow up with the xunit issue by looking for an existing tracking issue and opening one if it does not exist?

@NikolaMilosavljevic NikolaMilosavljevic left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Filed xunit/xunit#3584 -- the xUnit v3 AOT source generator emits invalid async lambda code when a test class implements IDisposable, causing CS4010. No pre-existing issue was found. We're working around it with a WithEnvRestore helper using try/finally instead of IDisposable.

@NikolaMilosavljevic
NikolaMilosavljevic merged commit 5aa354d into dotnet:main May 28, 2026
25 checks passed
NikolaMilosavljevic added a commit to NikolaMilosavljevic/sdk that referenced this pull request May 29, 2026
The merge with main (PR dotnet#54297) introduced AotSourceFiles.props, a shared
source list imported by both dotnet-aot.csproj and the new dotnet-aot.Tests.csproj.
The merge resolution left duplicate Compile items in dotnet-aot.csproj for files
that are now provided by the props import (CommandLineInfo.cs, Parser.cs,
EnvironmentVariableNames.cs), which would cause duplicate-item build failures.

Since the shared Parser.cs references the 'sdk check' command, the test project
(which compiles Parser.cs via the props) also needs the command's sources,
resources, and package dependency. Move those into AotSourceFiles.props so both
consumers stay in sync, and remove the now-duplicated entries from dotnet-aot.csproj.

Also move the host-provided DotnetRoot state from Program.cs into the shared
NativeEntryPoint.cs, because NativeEntryPoint.cs is compiled by the test project
but Program.cs is not. This keeps the test project building without pulling in
Program.cs (which defines a Main entry point).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@dotnet-milestone-bot dotnet-milestone-bot Bot added this to the 11.0-preview6 milestone May 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants