Support RID-specific .NET Tool packages #48575

dsplaisted · 2025-04-18T12:45:06Z

Work in progress, currently supports RID-specific packages for global tools.

Create a new abstract base class called ToolPackageDownloaderBase. Refactor #ToolPackageDownloader to derive from ToolPackageDownloaderBase, and move the implementation into the base class.

baronfel

Left a few notes. I think this is good to go into preview as it is. There are a few follow-up questions and scenarios I want to cover:

I think we need to be able to support both RID-agnostic and RID-specific tools at install-time (and when packaging) to make sure that older SDKs can continue to install tools that start opting in to RID-specific mode
I don't want users to have to make MSBuild Items - we already know that users don't understand/think about Items, and they're used to dealing with RIDs as properties
I really want to get to a single-command invocation when possible (i.e. when NativeAOT isn't happening) instead of having to do N+1 invocations, where N is the number of RIDs to publish for.

baronfel · 2025-06-06T14:53:28Z

src/Cli/dotnet/ToolPackage/ToolPackageDownloaderBase.cs

+        if (toolConfiguration.RidSpecificPackages?.Any() == true)
+        {
+            var runtimeGraph = JsonRuntimeFormat.ReadRuntimeGraph(_runtimeJsonPath);
+            var bestRuntimeIdentifier = Microsoft.NET.Build.Tasks.NuGetUtils.GetBestMatchingRid(runtimeGraph, RuntimeInformation.RuntimeIdentifier, toolConfiguration.RidSpecificPackages.Keys, out bool wasInGraph);
+            if (bestRuntimeIdentifier == null)
+            {
+                throw new ToolPackageException(string.Format(CliStrings.ToolUnsupportedRuntimeIdentifier, RuntimeInformation.RuntimeIdentifier,
+                    string.Join(" ", toolConfiguration.RidSpecificPackages.Keys)));
+            }
+
+            var resolvedPackage = toolConfiguration.RidSpecificPackages[bestRuntimeIdentifier];
+
+            if (!IsPackageInstalled(new PackageId(resolvedPackage.Id), resolvedPackage.Version, packageDownloadDir.Value))
+            {
+                DownloadAndExtractPackage(new PackageId(resolvedPackage.Id), nugetPackageDownloader, packageDownloadDir.Value, resolvedPackage.Version, packageSourceLocation, includeUnlisted: true);
+            }
+
+            CreateAssetFile(new PackageId(resolvedPackage.Id), resolvedPackage.Version, packageDownloadDir, Path.Combine(assetFileDirectory.Value, ToolPackageInstance.RidSpecificPackageAssetsFileName), _runtimeJsonPath, targetFramework);
+        }


Effectively, this means that as soon as you have any RID-specific packages, you must have RID-specific packages for all platforms you intend to support. This seems limiting - ideally we'd be able to support a RID-agnostic package and a number of RID-specific packages, and in this way older SDKs can use the RID-agnostic package, and newer SDKs can use a RID-specific package when available, and then fallback to the RID-agnostic package if there's no best-fit in the RID-specific packages.

I think it's going to be tricky to support old SDKs. Older SDKs aren't going to be able to understand a newer package / tool configuration XML format. So I think the only way that would work is if the primary tool package still contained the RID-agnostic tool implementation. But then for newer SDKs you're downloading the whole RID-agnostic implementation just to figure out that you'd prefer a RID-specific package and then go download that.

I think it's probably OK if this is a new feature and if tools want to take advantage of it they won't work with older SDKs. There is a format version in the tool configuration XML, so older SDKs will get a warning message saying they don't support the package. (I would have made it an error probably, but for older SDKs we're stuck with existing behavior.)

baronfel · 2025-06-06T14:58:36Z

src/Tasks/Microsoft.NET.Build.Tasks/targets/Microsoft.NET.PackTool.targets

+                      '@(ToolPackageRuntimeIdentifier)' != '' And
+                      '$(RuntimeIdentifier)' == ''">
+
+    <!-- If we are building the primary package for a tool with RID-specific packages, we don't include any of the build output in the package, so we don't need to run Build.


After talking with @nkolev92, we can potentially skip the need for this RemoveTargetFromList Task entirely if we set IncludeBuildOutput to false for the outer RID-specific tool package (see here). If we do this then even if the Build target is called no content from it will be auto-included in the package, which seems cleaner to me than modifying the Target list in this way.

IncludeBuildOutput is already set to false since the tool targets explicitly add the publish output to the package. I had to disable build from running because otherwise either PublishAot would cause a default RuntimeIdentifier to be set, and if I disabled that then the build would error out because PublishAot needs a RuntimeIdentifier.

baronfel · 2025-06-06T15:01:10Z

test/Microsoft.DotNet.PackageInstall.Tests/TestToolBuilder.cs

+    //
+    //  The main thing this class can't handle is if the way the .NET SDK builds packages changes.  In CI runs, we should use a clean test execution folder each time (I think!), so this shouldn't be an issue.
+    //  For local testing, you may need to delete the artifacts\tmp\Debug\TestTools folder if the SDK changes in a way that affects the built package.
+    public class TestToolBuilder


This is great, thanks :)

joeloff · 2025-06-06T15:45:12Z

src/Cli/dotnet/Commands/Tool/Install/ToolInstallLocalInstaller.cs

        try
        {
+            //  NOTE: The manifest file may or may not be under a .config folder.  If it is, we will use
+            //  that directory as the root config directory.  This should be OK, as usually there won't be


nit: Extra space in the comment

Do you mean after the period? I often put two spaces after the end of a sentence as that's what I learned when I was learning to type (on a mechanical typewriter!).

joeloff · 2025-06-06T15:50:56Z

src/Cli/dotnet/ShellShim/ShellShimRepository.cs

        if (OperatingSystem.IsWindows())
        {
-            return _shimsDirectory.WithFile(commandName.Value + ".exe");
+            if (toolCommand.Runner == "dotnet")


Do we need to support Powershell at all?

Good point, but testing just now in powershell it seems like it will resolve .cmd files that are in the path:

PS C:\Users\Daniel> ToolTest Hello, Tool! PS C:\Users\Daniel> (Get-Command ToolTest).Path C:\Users\Daniel\.dotnet\tools\ToolTest.cmd

So it looks like we don't need to do anything more.

joeloff · 2025-06-06T15:59:53Z

src/Cli/dotnet/ToolPackage/ToolPackageDownloader.cs

-        bool isGlobalTool = false,
-        RestoreActionConfig restoreActionConfig = null)
+
+    // The following methods are copied from the LockFileUtils class in Nuget.Client


Does this risk missing out on changes to the Nuget.Client?

Yes, it's possible. In practice it's not a whole lot of code. Also it's code that was already copied into the SDK repo when we switched from running Restore on a temporary project to using the NuGet package download APIs. This PR just refactors and moves the code around.

joeloff · 2025-06-06T16:13:53Z

src/Tasks/Microsoft.NET.Build.Tasks/GenerateToolsSettingsFile.cs

+
+            if (commandRunner != "dotnet")
+            {
+                formatVersion = 2;


Can this be linked back to the other formatVersion in the CLI or does that not make sense, something like const int LatestToolFormatVersion = 2; ? Also, should we check for specific command runners

I didn't do something like that since the version written depends on the features you use, so if you don't use new features we'll use the old format version and be backwards compatible.

I have now just added an error if you set the runner to something besides "dotnet" or "executable".

test/Microsoft.DotNet.PackageInstall.Tests/TestToolBuilder.cs

nagilson

Thanks for responding to my changes. I would like to do more testing, but I know that we'd like to get this in quickly. I agree with @baronfel's suggestions about a future improvement PR. In the future, it'd be nice to have more telem to understand how users are interfacing with/using the feature. Excited for this change and it's a big feature! 🙌

dsplaisted · 2025-06-06T17:00:44Z

I believe @dsplaisted wanted @rainersigwald to look over these MSBuild workarounds

Yes, I had forgotten to ping Rainer about that.

Also @nkolev92 @dotnet/nuget-team this is a pretty big PR, but you might want to look at the build logic parts of it.

dsplaisted · 2025-06-06T18:43:18Z

/azp run dotnet-sdk-public-ci

azure-pipelines · 2025-06-06T18:43:31Z

Azure Pipelines successfully started running 1 pipeline(s).

zivkan · 2025-06-06T21:38:30Z

test/Microsoft.DotNet.PackageInstall.Tests/TestToolBuilder.cs

+            return packageOutputPath;
+        }
+
+        public void RemovePackageFromGlobalPackages(ITestOutputHelper log, string packageId, string version)


In case you're not already aware, the global packages folder can be changed via nuget.config, with the globalPackagesFolder config. NuGet does this in the NuGet.Client repo for restore tests, and deletes the entire directory after each test. But considering this PR is about tools, I'm not sure if dotnet tools loads nuget.config files, and if so, if it takes into account the CWD, or if it looks at the user nuget.config only.

I believe we redirect the global packages folder to a local folder, but we may not clean it each time for perf reasons. So this method uses the NuGet locals command to resolve the correct global packages folder in the current context (and tools do take into account NuGet.config configuration from the current working directory).

nkolev92 · 2025-06-06T23:22:45Z

src/Cli/dotnet/NugetPackageDownloader/PackageSourceLocation.cs

                if (!Uri.IsWellFormedUriString(feed, UriKind.Absolute) && !Path.IsPathRooted(feed))
                {
-                    localFeedsThatIsRooted[index] = Path.GetFullPath(feed);
+                    localFeedsThatIsRooted[index] = Path.GetFullPath(feed, basePath);


Is this order correct?

Yes, the second parameter is what replaces the current directory if you didn't pass it in.

src/Tasks/Common/Resources/xlf/Strings.zh-Hans.xlf~RF3afe8107.TMP

dotnet-policy-service · 2025-08-11T14:42:32Z

Added needs-breaking-change-doc-created label because this PR has the breaking-change label.

When you commit this breaking change:

Create and link to this PR and the issue a matching issue in the dotnet/docs repo using the breaking change documentation template, then remove this needs-breaking-change-doc-created label.
Ask a committer to mail the .NET SDK Breaking Change Notification email list.

You can refer to the .NET SDK breaking change guidelines

baronfel · 2025-08-11T14:42:44Z

Added the breaking change label based on feedback from #49840 - we need to have a breaking change notice to the .NET 10 release notes that dotnet pack of tools changed from the Build target to the Publish target regardless of multi-RID support or not.

ghost added Area-Tools untriaged Request triage from a team member labels Apr 18, 2025

build-analysis bot mentioned this pull request Apr 18, 2025

Template engine tests hitting known parallelism issue in GetVisualStudioInstances #44878

Open

dsplaisted force-pushed the rid-specific-tools branch from 3475e6a to 4359417 Compare May 21, 2025 17:13

build-analysis bot mentioned this pull request May 27, 2025

Throttling of Docker Downloads dotnet/arcade#15659

Open

3 tasks

marcpopMSFT mentioned this pull request May 27, 2025

How to author 'dotnet tool' packages that are CPU-architecture (or OS) specific? #24016

Closed

dsplaisted force-pushed the rid-specific-tools branch 2 times, most recently from 79fc176 to a3a3217 Compare June 1, 2025 14:54

nagilson requested a review from Copilot June 3, 2025 20:33

This comment was marked as outdated.

Sign in to view

This was referenced Jun 3, 2025

The active test run was aborted. Reason: Test host process crashed dotnet/dnceng#451

Closed

dotnet-watch tests failing #40006

Open

[dotnet watch] Cannot open project Lib.fsproj because extension fsproj is not associated with a language #46378

Closed

dsplaisted added 17 commits June 3, 2025 20:09

Start of support for packing tools with RID-specific packages

b3e952b

Support packing NativeAot .NET Tools

e3bac63

Start of support for downloading RID-specific tool packages

839eae4

Copilot WIP

54611eb

Copilot WIP

2b316c3

Refactoring

be491e8

Refactoring global tool installation

eb7bd6e

Refactoring local tool installation

0db06fb

Download RID-specific tool package in ToolPackageDownloader

8fff9de

Handle RID-specific packages for global tools

2b7d6fc

Fix handling tool assets files with RID-less targets

c658286

Fix packing tool shims

2825c91

Use transaction helper API for tool install and uninstall

e54cbc7

Rename RestoredCommand to ToolCommand

654cc87

Copilot edits: Create ToolPackageDownloaderBase

49c1f3f

Create a new abstract base class called ToolPackageDownloaderBase. Refactor #ToolPackageDownloader to derive from ToolPackageDownloaderBase, and move the implementation into the base class.

Fix copilot: Duplicate ToolPackageDownnloaderBase

96b8f68

Make methods non-virtual

f5be4e9

dsplaisted added 4 commits June 5, 2025 21:51

Try to fix test in CI

b6b1c58

Localize exception message

b76fd91

Fix exception message

efcdd4a

Address / remove TODOs

fac0117

baronfel approved these changes Jun 6, 2025

View reviewed changes

joeloff reviewed Jun 6, 2025

View reviewed changes

test/Microsoft.DotNet.PackageInstall.Tests/TestToolBuilder.cs Outdated Show resolved Hide resolved

joeloff approved these changes Jun 6, 2025

View reviewed changes

nagilson approved these changes Jun 6, 2025

View reviewed changes

Apply code review feedback

922242b

Improve tests

ce1bba1

zivkan reviewed Jun 6, 2025

View reviewed changes

nkolev92 reviewed Jun 6, 2025

View reviewed changes

dsplaisted merged commit 47d0bba into dotnet:main Jun 6, 2025
21 of 30 checks passed

baronfel mentioned this pull request Jun 12, 2025

[New article]: dotnet tools support RID-specific and AOT packages dotnet/docs#46801

Closed

PathogenDavid reviewed Jun 19, 2025

View reviewed changes

src/Tasks/Common/Resources/xlf/Strings.zh-Hans.xlf~RF3afe8107.TMP Show resolved Hide resolved

baronfel added breaking-change Using this label will notify dotnet/compat and trigger a request to file a compat bug needs-breaking-change-doc-created labels Aug 11, 2025

baronfel mentioned this pull request Aug 11, 2025

[Breaking change]: .NET Tool packaging now invokes Publish instead of Build dotnet/docs#47916

Closed

3 tasks

Copilot AI mentioned this pull request Aug 11, 2025

Document .NET Tool packaging breaking change for .NET 10 Preview 6 dotnet/docs#47925

Merged

baronfel removed the needs-breaking-change-doc-created label Aug 21, 2025

Copilot AI mentioned this pull request Nov 12, 2025

Add article for RID-specific, self-contained and AOT .NET tools dotnet/docs#49874

Merged

Support RID-specific .NET Tool packages #48575

Support RID-specific .NET Tool packages #48575

Uh oh!

Conversation

dsplaisted commented Apr 18, 2025

Uh oh!

This comment was marked as outdated.

Uh oh!

baronfel left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

nagilson left a comment

Choose a reason for hiding this comment

Uh oh!

dsplaisted commented Jun 6, 2025

Uh oh!

dsplaisted commented Jun 6, 2025

Uh oh!

azure-pipelines bot commented Jun 6, 2025

Uh oh!

zivkan Jun 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

dotnet-policy-service bot commented Aug 11, 2025 • edited by baronfel Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

baronfel commented Aug 11, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

zivkan Jun 6, 2025 •

edited

Loading

dotnet-policy-service bot commented Aug 11, 2025 •

edited by baronfel

Loading