Add CG2 to source-build bundle

[am11]

so that this last line https://github.com/filipnavara/dotnet-riscv/blob/ee6f66cef9df27c4783561219f6589112533ab43/.github/workflows/build.yml#L52 is not needed in SB scenarios (followed by manually copying it to <SDK install prefix>/library-packs/ after extraction on target machine to enable PublishReadyToRun=true).

cc @jkoritzinsky, @filipnavara

[jkoritzinsky]

cc: @dotnet/distro-maintainers

[jkoritzinsky]

@tmds this might help address the issues you mentioned in dotnet/source-build#5225

[tmds]

@tmds this might help address the issues you mentioned in dotnet/source-build#5225

It's a different problem.

[tmds]

@am11 have you checked how much the size increase is?

I've extracted Private.SourceBuilt.Artifacts and I see Microsoft.NETCore.App.Crossgen2.fedora.41-x64.10.0.0-preview.6.25303.101.nupkg is 129MB.

I think the debug symbols are bloating the package significantly.

[am11]

For x64 and other MSFT platforms, PublishReadyToRun=true pulls it from nuget.org. For community supported platforms, the CG2 runtime package is not available. We have to manually copy the nuget package to the local store. We can limit it to non-MSFT platforms with a long condition (either include/exclude pattern).

[tmds]

When we added source-built NativeAOT we spend some time to reduce the size impact on the SDK. We placed the native files under packs/runtime.<rid>.Microsoft.DotNet.ILCompiler where they get stripped from symbols during distribution packaging. This directory is packed as a separate distro package to avoid the size impact on the overall SDK footprint (dotnet-sdk-aot-[major].[minor]). See https://learn.microsoft.com/en-us/dotnet/core/distribution-packaging.

I don't think we should have this 100+MB file by default under library-packs.

We can limit it to non-MSFT platforms with a long condition (either include/exclude pattern).

Yes, or let the maintainer set /p:BundleCrossgen2=true if they want to include it.

[tmds]

@am11 I think this needs some changes to GenerateBundledVersions.targets to update Crossgen2SupportedRids when BundleCrossgen2 is set?

[am11]

On linux-riscv64, it's a 32M package:

 32M    Microsoft.NETCore.App.Crossgen2.linux-riscv64.10.0.0-preview.5.25256.10.nupkg

Funny thing is Microsoft.NETCore.App.Crossgen2.xx.symbols.nupkg (separate symbols package) is also produced:

 32M  Microsoft.NETCore.App.Crossgen2.linux-riscv64.10.0.0-preview.5.25256.10.symbols.nupkg

They have identical content. Looks like a packaging bug.

@am11 I think this needs some changes to GenerateBundledVersions.targets to update Crossgen2SupportedRids when BundleCrossgen2 is set?

I was testing it with linux-riscv64 which is covered

sdk/src/Layout/redist/targets/GenerateBundledVersions.targets

Line 370 in 427918c

<Crossgen2SupportedRids Include="@(Net90Crossgen2SupportedRids);" />

I can add target RID for non-portable. Good catch!

[tmds]

On linux-riscv64, it's a 32M package:

The debug symbols shouldn't be stripped for source-build: https://github.com/dotnet/runtime/blob/624737eb3796e1a760465912b27ac349965d8ba5/Directory.Build.props#L276.

This is what is in the Fedora 41 x64 package:

35M	crossgen2
46M	libclrjit_universal_arm64_x64.so
37M	libclrjit_universal_arm_x64.so
46M	libclrjit_unix_x64_x64.so
46M	libclrjit_win_x64_x64.so
45M	libclrjit_win_x86_x64.so
268K	libjitinterface_x64.so

What files are in the linux-riscv64 package?

[am11]

It is downloadable from one of the runs https://github.com/filipnavara/dotnet-riscv/actions/runs/15317256008. For community platforms, there is one JIT for the target RID tools/libclrjit_unix_riscv64_riscv64.so. As long as we don't want to publish for different platform, it is enough. Empirically speaking, it is not common from end-users' perspective who use PublishReadyToRun to cross publish stuff. With various CI, cloud and docker --platform <whatever> type of options, people don't really miss cross-platform publishing these days.

However, this - too - doesn't seem like something done intentionally but rather an infra bug.

[tmds]

I don't even consider it common for end-users to publish r2r for the same platform, which is why I've not pursued bundling it with the SDK.

Due to the large increase in size, I don't think we should bundle it by default for platforms where Microsoft provides a package through through nuget.org.

Even for community platforms, you might choose to out the feature rather than take on the extra size.

[filipnavara]

Even for community platforms, you might choose to out the feature rather than take on the extra size.

There're are additional problems for community platforms. Firstly, there's no mechanism to simply obtain these packages since they are not on NuGet. Suddenly you need extra infrastructure in addition to the xcopy deployed SDK. Secondly, I use the SDKs to bootstrap the VMR/runtime builds on target machines for local testing. Obviously I expect that to be niche but it's important at this stage.

[am11]

I don't even consider it common for end-users to publish r2r for the same platform

Quick github search suggests there are more PublishReadyToRun usages than PublishAot. Then supporting PublishAot out of the box and leaving behind PublishReadyToRun does not make sense..

[tmds]

The main suggestion I'm making is to not have BundleCrossgen2 default to true for platforms where Microsoft provides nuget.org packages due to the additional size.

For the community platforms, I don't have a strong opinion. I see both reasons to build BundleCrossgen2=true and reasons to build BundleCrossgen2=false. The maintainer can control the flag if they want something different than what is the default.

[tmds]

@am11 I think it would be an option to default BundleCrossgen2 to false and let users that want it opt-in. It seems you have a strong preference for this to be included by default on the community platforms.

library-packs provides a convenient way to bundle NuGet packages that Microsoft provides through nuget.org.
Unfortunately, when these packages have native libraries, those files contain debug symbols which causes the NuGet packages to be bloated.

[am11]

It seems you have a strong preference for this to be included by default on the community platforms.

MSFT supported RID gets it from nuget.org so it is optional. Unsupported RIDs will get it bundled by default because there is no other way to obtain it. Size optimization is not important when the question is about completeness. This is a choice of .NET team exclusively, as third party packages with native assets do not distinguish between official vs. community platforms: e.g. https://nuget.info/packages/SQLitePCLRaw.lib.e_sqlite3/2.1.11 so it is not a widespread issue.

[tmds]

I don't think an end-user would have much of an issue if crossgen is not available on a community platform. They wouldn't be able to optimize their app.

For ILCompiler the nupkg was also 100MB+ in size, so we did a different design that allowed to strip the symbols.

If we could do something similar for Crossgen2 then it would be bundled by default with all source-built SDKs.
It would require a more complex solution though.

I'm ok with the PR as it is.

[am11]

The size issue is most likely a bug #49241 (comment). Symbols package is created but it has the same content as the original.

I don't think an end-user would have much of an issue if crossgen is not available on a community platform. They wouldn't be able to optimize their app.

PublishReadyToRun does not work, while PublishAot works. You may not see it as an issue, others do.