Description
RID Plan
We worked on a plan to tame the RID graph last year. It got close to resolution, but not quite far enough. We also didn't have time to fund it as part of .NET 7. We started talking about it again earlier this year. We have defined targeted updates to the earlier plan that we think will get across the finish line.
The goal remains the same, which is to stop updating the RID graph and to stop reading it in all scenarios, by default. Eventually, we want to delete it. The RID graph represents a poor design point of the product, particularly in terms of how it supports Linux, aiming to act as a persisted database of historical and current operating system releases. We're also not doing a great job of keeping it updated to satisfy its intended function.
Quick context
The RID graph is used to two primary spots:
- By
dotnet build
ordotnet publish
to select the best-match RID-specific assets (primarily from NuGet packages) to generate a RID-specific app. - By the host to select RID-specific assets from a portable app layout.
Put another way, given RID-specific assets in an app NuGet graph, those assets are either selected during build or deferred to runtime.
In both cases, the target RID of the app or the environment often doesn't match the RID of the assets. That's intentional and fine, however resolving that difference requires a best-match algorithm. Today, the RID graph serves as an authoritative database for that algorithm to use.
For example, a portable app might be running on a Linux Mint 19 machine, yet carry assets (for a given dependency) that separately target Ubuntu 18.04 and portable Linux. The host knows (via the RID graph) that the Ubuntu 18.04 asset is compatible with Linux Mint 19 and a better match than the portable Linux asset. It chooses the Ubuntu 18.04 asset.
Open issues
The following issues are the primary ones blocking progress:
- We planned to have an algorithmic approach. Should it include concrete RIDs, like
ubuntu.22.04
? - Source build users tend to include concrete RIDs, like
rhel.8
. Should we ensure that distro-specifc source-build builds and the portable Linux build have parity in that respect? - How do we handle compatible distros, like Oracle Linux and RHEL or Linux Mint and Ubuntu?
- Can we use the same solution for the apphost and the SDK?
Probing Algorithm
The original plan proposed the following probing model for the portable build:
linux-x64
unix
Note: Architecture-specific RIDs are intended for native and ready-to-run code. Architecture-agnostic RIDs are intended for IL code (often related to P/Invokes).
That's probably artificially limited. Source-build users offer a more expansive set of RIDs to probe, which might bias to too much.
We concluded that the following model should satisfy the vast majority of needs.
ubuntu.22.04-x64
ubuntu.22.04
linux-x64
unix
This model would offer symmetry (or symmetry enough) between the portable build and the typical distro-specific source-build build.
Distro compatibility
Expressing distro-compatibility adds a lot of nodes to the centrally-managed RID graph. We also expect that they don't benefit a lot of users. Instead, we propose to move to a model where compatibility relationships are described in app project files for the relatively few apps that need them. We propose roughly similar support as what source-build users have for adding RID information.
We don't want to invent a new format. We have three formats already:
We propose to use the runtime.compatibility.json
format, like the following real example:
"linuxmint.19": [
"linuxmint.19",
"ubuntu.18.04",
"ubuntu",
"debian",
"linux",
"unix",
"any",
"base"
]
This compatibility description is notable since it doesn't include architecture information. Given a desire for apps to run on multiple architectures, we can algorithmically add architecture when apps (or tools) are running (in a given environment).
We'd recommend a more terse description (to describe just the compatability relationship):
"linuxmint.19": [
"linuxmint.19",
"ubuntu.18.04",
]
Multiple compatibility relationships can be described.
{
"linuxmint.19": [
"linuxmint.19",
"ubuntu.18.04",
],
"ol.8": [
"ol.8",
"rhel.8",
],
}
We propose that this compatibility information be included in app project files in some new property, like the following:
<PropertyGroup>
<RuntimeCompatibility>
{
"linuxmint.19": [
"linuxmint.19",
"ubuntu.18.04",
],
"ol.8": [
"ol.8",
"rhel.8",
],
}
</RuntimeCompatibility>
</PropertyGroup>
The following is an alternative format we will consider:
<ItemGroup>
<RuntimeCompatibility Include="linuxmint.19.2" Fallbacks="linuxmint.19.1;linuxmint.19;ubuntu.18.04;ubuntu;debian" />
<RuntimeCompatibility Include="ol.8" Fallbacks="rhel.8" />
</ItemGroup>
This information would be copied into app.deps.json
in a new section.
Embedded JSON is a bit ugly. We could use XML, in the runtimeGroups.props format. That would be workable. We'd need to productize the task that converts that XML to JSON. Given how targeted we think the scenario is, we are proposing the simpler JSON-based approach.
The RID graph includes multiple variations for each distro, like ol.8
and ol.8.0
. For Linux Mint, there is linuxmint.19.0
, linuxmint.19.1
, and linuxmint.19.2
. For Ubuntu, there is ubuntu.18.04
and ubuntu.18.10
. Note the relationships between Linux Mint and Ubuntu releases.
We should reason about distro versions in terms of VERSION_ID
in /etc/os-release
and then rely on stated compatibility relationships for any differences, including between Linux Mint dot versions (for example).
$ docker run --rm linuxmintd/mint21-amd64 cat /etc/os-release | grep VERSION
VERSION="21 (Vanessa)"
VERSION_ID="21"
VERSION_CODENAME=vanessa
$ docker run --rm linuxmintd/mint21.1-amd64 cat /etc/os-release | grep VERSION
VERSION="21.1 (Vera)"
VERSION_ID="21.1"
VERSION_CODENAME=vera
This means that linuxmint.21
and linuxmint.21.1
would be valid RIDs, but linuxmint.21.0
would not. A compatibility relationship would need to be specified (as described above) to enable an app running on Linux Mint 21.1 to use a linuxmint.21
asset.
The following examples demonstrate the differing compatibility relationships offered by Linux Mint and Ubuntu.
Linux Mint 19.2:
"linuxmint.19.2": [
"linuxmint.19.2",
"linuxmint.19.1",
"linuxmint.19",
"ubuntu.18.04",
"ubuntu",
"debian",
"linux",
"unix",
"any",
"base"
],
Ubuntu 18.10:
"ubuntu.18.10": [
"ubuntu.18.10",
"ubuntu",
"debian",
"linux",
"unix",
"any",
"base"
],
They key point being made is that there is no compatibility relationship offered between Ubuntu releases. Ubuntu 18.04 assets are not using when running on Ubuntu 18.10. However, both Linux Mint and Ubuntu RIDs offer compatibility to Debian. We will not offer compatibility for any unversioned distro RIDs, like ubuntu
and debian
.
Note: This doc uses a mix of Linux Mint 19 and 21 since v19.x is the most recent version recorded in the RID graph, while v21.x is the latest stable version. That alone provides evidence of the operational drawbacks of the current model and a benefit that the proposed system would deliver.
Backwards compatibility
The transition to an algorithmic approach to RIDs is a significant change. We will need a backwards compatibility mode.
We propose:
- The apphost uses the algorthmic mode by default.
- There is a property that enables using the existing RID graph approach instead.
- App developers can specify distro compatibility relationships that they want to use.
The expectation is that this combination of modes will enable us to freeze or significantly freeze the RID graph. Assuming we freeze the RID graph, there may be scenarios where users might need to use option #3 in combination with either option 1 or 2.
SDK experience
We need to work through the SDK/tools experience a bit more.
We need an experience that notifies developers that they will not be well-served by the pure algorthmic mode. This will be the case when dotnet restore
detects a non-portable RID-specific asset.
Portable RIDs:
linux
unix
win
osx
wasm
Non-portable RID examples:
linuxmint.19.2
ubuntu.18.04
rhel.8
If a non-portable RID is discovered in the package graph, the SDK should produce a warning, similar to the following:
"Distribution-specific assets were identified in one or more package references. You may want to add distribution-compatibility relationships to your project file. See http://aka.ms/dotnet/rid/compatibility to learn more."
We would offer some pragma disable option to quiet the warning.
For portable apps, this warning would give developers an opportunity to define compatibility relationships so that the correct assets get used on Linux Mint and Ubuntu, for example.
RID-specific apps are in some ways the easier case. The developer will specify a specific RID and there is an opportunity for the developer to inspect the results and for the SDK to present an error (if there was no best-match asset).
We have a couple options on identifiying non-portable RIDs. We can the discrete list above, or we can rely on the existing RID graph to determine what the non-portable RIDs are. Both are non-perfect.
Regression or improvement?
This proposal is simultaneously a regression and an improvement.
How it is a regression:
- Distro compatibility relationships become an app dev concern.
- .NET doesn't come with distro relationship information, and there is no central place for recording that.
How it is an improvement:
- Distro compatibility relationships can be specified at the point of need.
- RID graph updates are not necessary and no longer need to rely on the expensive and lengthy .NET servicing process.
- The RID graph is inconsistent in its updates. It is now as up to date as needed by the app developer.
A related problem is that we've never provided a way to correctly build portable linux
assets. From that point-of-view, it is safer to build distro-specific assets due to that, at least currently. We are considering offering guidance on how to build linux
assets. We're hoping that encourages producing more portable assets where there might be distro-specific ones today.
Impact on Source-Build
Today, source-build users rely on augmenting the RID graph via MSBuild. Assuming we can lock the RID graph, that will no longer make much sense. Instead, we'd need to move source-build users to this plan. That will likely mean making changes in host code instead of to the RID graph.
Deletion of RID graph
We should delete the RID graph with the .NET 9 or 10 releases, ideally the earlier of the two. The .NET 8 release should give us insight into which scenarios have a strong dependency on this data.