source-build on unknown linux distros

[tmds]

For source-build we want to be able to build .NET using source-build on a system where the rid is not known to .NET.

We want this build to succeed, and the rid to be known to .NET.

For the build to succeed, we need to build using another rid which is known and compatible. I think this may be possible already (to some extent) by using the DOTNET_RUNTIME_ID envvar.

For the rid to be known to .NET, it needs to know its place in the graph.

Example pseudo build commands:

Build on Fedora 35 (rid derived from os-release is fedora.35):

./build.sh --build-rid linux --parent-rids fedora,linux

Build on RHEL 8 (rid derived from os-release is rhel.8):

./build.sh --build-rid linux --parent-rids linux

Build on CentOS 8 (rid derived from os-release is centos.8):

./build.sh --build-rid linux --parent-rids rhel.8,linux

Relates to NuGet/Home#5862, dotnet/source-build#297.

cc @ericstj @omajid @crummel

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[omajid]

Thanks for filing this issue, @tmds ! I think this captures our issues with RID well: we just need a way for a random Linux distro to get an RID automatically, and assign some parents RIDs (mostly linux, but maybe musl too).

cc @dagood @MichaelSimons @dleeapho

Tagging subscribers to this area: @ViktorHofer
See info in area-owners.md if you want to be subscribed.

Issue Details

---

For source-build we want to be able to build .NET using source-build on a system where the rid is not known to .NET.

We want this build to succeed, and the rid to be known to .NET.

For the build to succeed, we need to build using another rid which is known and compatible. I think this may be possible already (to some extent) by using the DOTNET_RUNTIME_ID envvar.

For the rid to be known to .NET, it needs to know its place in the graph.

Example pseudo build commands:

Build on Fedora 35 (rid derived from os-release is fedora.35):

./build.sh --build-rid linux --parent-rids fedora,linux

Build on RHEL 8 (rid derived from os-release is rhel.8):

./build.sh --build-rid linux --parent-rids linux

Build on CentOS 8 (rid derived from os-release is centos.8):

./build.sh --build-rid linux --parent-rids rhel.8,linux

Relates to NuGet/Home#5862, dotnet/source-build#297.

cc @ericstj @omajid @crummel

Author:	tmds
Assignees:	-
Labels:	area-Infrastructure, untriaged
Milestone:	-

[ericstj]

For the build to succeed, we need to build using another rid which is known and compatible. I think this may be possible already (to some extent) by using the DOTNET_RUNTIME_ID envvar.

It would be good to understand if this is sufficient or if you would also need to modify the shared-framework's deps file (where the RID fallbacks live) in the tool shared framework. Doing the latter is possible but grows the complexity of this.

runtime/src/libraries/Microsoft.NETCore.Platforms/pkg/runtimeGroups.props

Lines 248 to 258 in 09a1256

	<Target Name="GenerateRuntimeJson" BeforeTargets="CreatePackage">
	<!-- Generates a Runtime graph using RuntimeGroups and diffs it with the graph described by runtime.json and runtime.compatibility.json
	Specifying UpdateRuntimeFiles=true skips the diff and updates those files.
	The graph can be visualized using the generated dmgl -->
	<MakeDir Directories="$(PackageReportDir)" />
	<GenerateRuntimeGraph RuntimeGroups="@(RuntimeGroupWithQualifiers)"
	RuntimeJson="runtime.json"
	CompatibilityMap="runtime.compatibility.json"
	RuntimeDirectedGraph="$(PackageReportDir)$(Id)$(NuspecSuffix)-runtime.json.dgml"
	UpdateRuntimeFiles="$(UpdateRuntimeFiles)" />
	</Target>

can be used to generate, today it writes to the repo location but that could be overridden. We'd need to expose the right set of options to extend the graph. Then when building make sure that the generated runtime.json was used everywhere instead of the checked in one.

[ericstj]

Also, once you produce such a shared framework, are you OK with nuget restore not knowing about? NuGet still gets the runtimegraph from the Micrsoft.NETCore.Platforms package which the user's package graph brings down. This is what is used for resolving and publishing self-contained apps.

[omajid]

Also, once you produce such a shared framework, are you OK with nuget restore not knowing about?

Is there a fallback here? To something like linux-x64? Or is it the case that if nuget doesn't know about the RID, restore wont work at all for RID-specific assets?

This is what is used for resolving and publishing self-contained apps.

Does that mean self-contained apps wont build/work? What about normal Framework-Dependent apps?

[ericstj]

I may be mistaken. It looks like the SDK did the work to pull out the runtime.json, bundle it in the SDK and provide it to NuGet. Thanks @dsplaisted, @nkolev92
NuGet/NuGet.Client@04166a6
dotnet/sdk@2723589

Assuming we flow the built content of Microsoft.NETCore.Platfroms from sourcebuild up through the SDK/installer then it looks like this should work. NuGet will "merge" the contents of the SDK's runtimegraph with whatever is defined in packages. So even if someone added the public shipped platforms package it would still have your source-build defined RIDs.

[tmds]

It would be good to understand if this is sufficient or if you would also need to modify the shared-framework's deps file (where the RID fallbacks live) in the tool shared framework. Doing the latter is possible but grows the complexity of this.

If it doesn't work already, probably we can pick up DOTNET_RUNTIME_ID in a few more places and avoid the complexity.

[ericstj]

@tmds can you better specify the inputs of what you'd like to see happen?

Describe what needs to be specified, vs what is calculated from the system, and how that is used by the build system. Also talk about how things interact with existing properties / parameters to the build

Interesting considerations:

• Is cross-targeting relevant, or perhaps required? Both by arch and distro. IOW will you always be building on the system that exposes the RID you're trying to add or are you sometimes building on a different RID?
• Does this also need to impact the configuration system in dotnet/runtime libraries for selecting which configurations to build?