Skip to content

Add logic to muxer to invoke AOT-ed SDK#126596

Open
elinor-fung wants to merge 5 commits intodotnet:mainfrom
elinor-fung:launch-aot-sdk
Open

Add logic to muxer to invoke AOT-ed SDK#126596
elinor-fung wants to merge 5 commits intodotnet:mainfrom
elinor-fung:launch-aot-sdk

Conversation

@elinor-fung
Copy link
Copy Markdown
Member

@elinor-fung elinor-fung commented Apr 7, 2026

Fixes #126171

Updates the muxer to look for and natively invoke a \dotnet-aot\ shared library when it exists in the resolved SDK directory. When present, the muxer loads the library, resolves the \dotnet_execute\ entry point, and calls it with the host path, dotnet root, SDK directory, hostfxr path, and user arguments. If the AOT library is not available or doesn't have the expected entry point, the muxer falls back to the existing managed SDK path.

Changes

Native host

  • Add try_invoke_aot_sdk helper in fx_muxer_t::handle_cli that attempts to load dotnet-aot from the resolved SDK directory
  • Fall through to the existing dotnet.dll path if the AOT library is absent

Tests

  • Add mockaotsdk native test library that implements the dotnet_execute entry point and prints received arguments
  • Add DispatchToAotSdk test in SDKLookup.cs validating the muxer dispatches to the AOT SDK with the correct arguments

cc @JeremyKuhne @dotnet/appmodel @AaronRobinsonMSFT

elinor-fung and others added 2 commits April 1, 2026 18:14
When the dotnet muxer resolves an SDK, it now checks for a dotnet-aot
native library in the SDK directory before falling back to the managed
dotnet.dll path. If the library exists and exports a dotnet_execute
entry point, the muxer loads it and calls it with the host path, dotnet
root, SDK directory, hostfxr path, and user arguments.

The AOT library handles its own fallback to the managed runtime path
when needed. If the library cannot be loaded or the entry point is
missing, the muxer gracefully falls back to the existing managed SDK
path.

Fixes dotnet#126171

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add DispatchToAotSdk test to verify the muxer correctly loads and
invokes the dotnet-aot library from the resolved SDK directory. The
test uses a mock AOT SDK shared library that prints the received
arguments for validation.

- Add mockaotsdk native test library with dotnet_execute entry point
- Add DotNetAot entry to Binaries.cs for test discovery
- Add DispatchToAotSdk test in SDKLookup.cs
- Move 'Using AOT-ed SDK' trace to just before invocation and remove
  redundant failure trace on load_library

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings April 7, 2026 02:18
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@dotnet-policy-service
Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @agocke, @jeffschwMSFT, @elinor-fung
See info in area-owners.md if you want to be subscribed.

@elinor-fung elinor-fung closed this Apr 7, 2026
@elinor-fung elinor-fung reopened this Apr 7, 2026
Copy link
Copy Markdown
Member

@agocke agocke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, few nits

Comment thread src/native/corehost/fxr/fx_muxer.cpp Outdated
if (!pal::load_library(&sdk_aot_path, &aot_dll))
return false;

typedef int (*dotnet_execute_fn)(
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have docs somewhere on this signature?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated one of the docs in runtime.
@JeremyKuhne Are there any sdk docs that would make sense to link to here?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@elinor-fung I don't have any links for you yet. The signature in the issue was copied directly from my wip. I'll be putting a PR up next week and I can add a link/links here after I commit. Copied again from my local changes:

    [UnmanagedCallersOnly(EntryPoint = "dotnet_execute")]
    static int Execute(
        nint hostPathPtr,      // const char_t* host_path
        nint dotnetRootPtr,    // const char_t* dotnet_root
        nint sdkDirPtr,        // const char_t* sdk_dir
        nint hostfxrPathPtr,   // const char_t* hostfxr_path
        int argc,              // int argc (user args, no dotnet exe)
        nint argvPtr)          // const char_t** argv

Comment thread src/native/corehost/fxr/fx_muxer.cpp
Copy link
Copy Markdown
Member

@JeremyKuhne JeremyKuhne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.

Copilot AI review requested due to automatic review settings April 17, 2026 20:33
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

Comment thread src/native/corehost/fxr/fx_muxer.cpp
Comment thread docs/design/features/sharedfx-lookup.md Outdated
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings April 17, 2026 22:22
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Comment on lines +85 to +129
pal::dll_t aot_dll = nullptr;
if (!pal::load_library(&sdk_aot_path, &aot_dll))
return false;

// See docs/design/features/sharedfx-lookup.md#sdk-search
typedef int (__cdecl *dotnet_execute_fn)(
const pal::char_t* host_path,
const pal::char_t* dotnet_root,
const pal::char_t* sdk_dir,
const pal::char_t* hostfxr_path,
int argc,
const pal::char_t** argv);

auto dotnet_execute = reinterpret_cast<dotnet_execute_fn>(pal::get_symbol(aot_dll, "dotnet_execute"));
if (dotnet_execute == nullptr)
{
trace::info(_X("AOT-ed SDK [%s] does not contain 'dotnet_execute' entry point."), sdk_aot_path.c_str());
pal::unload_library(aot_dll);
return false;
}

pal::string_t hostfxr_path;
if (!pal::get_own_module_path(&hostfxr_path))
{
trace::info(_X("Failed to determine hostfxr path."));
pal::unload_library(aot_dll);
return false;
}

const pal::char_t* dotnet_root = sdk_root.empty()
? host_info.dotnet_root.c_str()
: sdk_root.c_str();

trace::info(_X("Using AOT-ed SDK=[%s]"), sdk_aot_path.c_str());

*exit_code = dotnet_execute(
host_info.host_path.c_str(),
dotnet_root,
sdk_dir.c_str(),
hostfxr_path.c_str(),
argc - 1, // skip 'dotnet' (first argument)
argv + 1);

return true;
}
Copy link

Copilot AI Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

try_invoke_aot_sdk unloads the library on the failure paths, but on the success path it returns without calling pal::unload_library(aot_dll). In long-lived hosts that call hostfxr_main multiple times, this can accumulate loaded modules / memory and can keep the SDK’s dotnet-aot file in use. Consider ensuring the handle is released on the success path too (ideally via a small RAII wrapper so all early-returns are covered).

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

Add logic to muxer to invoke AOT'ed CLI

4 participants