Fix missing coverage of F# resources in linker RemoveResourcesStep

[T-Gro]

The original solution was missing 3 further dimensions of F# resources:

• Being compressed or not
• Being part of FSharp.Core (STD lib for F#) or user code
• Being original stream or a backwards-compatible stream for newer language features, particulary nullable reference types

The full set of F#-compiler-generated prefixes currently is:
FSharpSignatureData.*
FSharpSignatureDataB.*
FSharpSignatureCompressedData.*
FSharpSignatureCompressedDataB.*

FSharpOptimizationData.*
FSharpOptimizationDataB.*
FSharpOptimizationCompressedData.*
FSharpOptimizationCompressedDataB.*

FSharpOptimizationInfo.*
FSharpSignatureInfo.*

This closes dotnet/linker#329

[Copilot]

Pull Request Overview

This PR extends the resource removal logic in the linker to also cover additional F# resource variations, ensuring that both data and optimization resources—with and without compression, and including legacy stream handling—are correctly processed.

• Broadened prefix checks from "FSharpSignatureData" and "FSharpOptimizationData" to "FSharpSignature" and "FSharpOptimization"
• Now covers all required F#-compiler-generated resource prefixes

Comments suppressed due to low confidence (2)

src/tools/illink/src/linker/Linker.Steps/RemoveResourcesStep.cs:36

• The new condition generalizes the match to cover all resources starting with 'FSharpSignature'. Please verify that this broader check does not unintentionally remove any resources that should be retained.

				=> resource.Name.StartsWith ("FSharpSignature", StringComparison.Ordinal)

src/tools/illink/src/linker/Linker.Steps/RemoveResourcesStep.cs:37

• Similarly, ensure that generalizing the match to 'FSharpOptimization' does not inadvertently target unrelated resources. Consider verifying against the complete list of expected F# resource prefixes.

				|| resource.Name.StartsWith ("FSharpOptimization", StringComparison.Ordinal);

[marek-safar]

I think you will have to keep existing prefixes to not break nugets compiled with older F# compiler.

[agocke]

Since that issue was filed we've tried to stop special-casing individual libraries and patterns in the linker. This is doubly true for anything which is not trim-compatible. Instead we rely on static analysis and trim warnings to determine what's reachable and what's not. It would be better if this entire approach were re-worked to safely determine reachability of resources.

Is there a reason why we need to take this change now?

[T-Gro]

I think you will have to keep existing prefixes to not break nugets compiled with older F# compiler.

The existing prefixes are covered with this PR as well, this really just fixes a bug of the existing prefixes not covering everything.

[T-Gro]

Since that issue was filed we've tried to stop special-casing individual libraries and patterns in the linker. This is doubly true for anything which is not trim-compatible. Instead we rely on static analysis and trim warnings to determine what's reachable and what's not. It would be better if this entire approach were re-worked to safely determine reachability of resources.

Is there a reason why we need to take this change now?

The reason is to make resulting artifacts smaller - the existing mechanism was done incompletely and this change increases its coverage to respect resources emitted by the F# compiler.

This is not special casing any library - those are prefix-patterns which are relevant for all compiled F# projects.

[agocke]

This is not special casing any library - those are prefix-patterns which are relevant for all compiled F# projects.

F# resources themselves are the special case, as evidenced by the fact that they're being specially recognized by name. The general pattern would be to remove all unreachable managed resources.

We're also trying to avoid as many differences between ILLink and Native AOT. Since this is a custom step I assume this feature isn't implemented for Native AOT?

[T-Gro]

This is not special casing any library - those are prefix-patterns which are relevant for all compiled F# projects.

F# resources themselves are the special case, as evidenced by the fact that they're being specially recognized by name. The general pattern would be to remove all unreachable managed resources.

We're also trying to avoid as many differences between ILLink and Native AOT. Since this is a custom step I assume this feature isn't implemented for Native AOT?

Not that I know of. I could only find a mention of F# resources that needed adjusting at dotnet/linker#1040 , I do not see any other one.

[MichalStrehovsky]

We have a general purpose mechanism to remove resources from assemblies:

runtime/docs/tools/illink/data-formats.md

Lines 270 to 278 in 72a5ef7

	### Remove embedded resources
	```xml
	<linker>
	<assembly fullname="Assembly">
	<resource name="Resource" action="remove" />
	</assembly>
	</linker>
	```

If the F# compiler wishes to have these removed, it can embed such manifest as an additional resource. We also support conditioning the removal on a feature name. So the entry can also be conditional and there can be an opt out if needed:

runtime/docs/tools/illink/data-formats.md

Line 280 in 72a5ef7

### Conditional substitutions and descriptors

I don't think it's a good idea for ILLink to remove things it should know nothing about (will anyone re-evaluate things if these resources do become necessary at runtime for F# for some reason in the future?).

The general purpose format didn't exist at the time the dotnet/linker#1040 PR was created (however it did exist at the time the PR got merged a year later, but it probably fell through cracks at that point).

It would be better if this entire approach were re-worked to safely determine reachability of resources.

I briefly looked into this in the past but most existing uses of managed resources are not actually statically analyzable. It would require a new set of annotations and analysis capabilities and even then I'm not sure if they could cover existing uses just in our dotnet repos (names of resources being looked up are often computed, and so is the assembly containing the resource - it's rarely a statically analyzable pattern, but something computed from a Type that is passed around). Might be worth at least filing an issue though so that we can track potential use cases.

[Numpsy]

We have a general purpose mechanism to remove resources from assemblies

The comment at dotnet/fsharp#15605 (comment) says

but it's not meant (at least yet) to be a general feature.

so has that changed?

[T-Gro]

We have a general purpose mechanism to remove resources from assemblies:

runtime/docs/tools/illink/data-formats.md

Lines 270 to 278 in 72a5ef7

	### Remove embedded resources
	```xml
	<linker>
	<assembly fullname="Assembly">
	<resource name="Resource" action="remove" />
	</assembly>
	</linker>
	```

If the F# compiler wishes to have these removed, it can embed such manifest as an additional resource. We also support conditioning the removal on a feature name. So the entry can also be conditional and there can be an opt out if needed:

runtime/docs/tools/illink/data-formats.md

Line 280 in 72a5ef7

### Conditional substitutions and descriptors

I don't think it's a good idea for ILLink to remove things it should know nothing about (will anyone re-evaluate things if these resources do become necessary at runtime for F# for some reason in the future?).

Thanks for the explanation @MichalStrehovsky .

We know how to remove the resources for the compiler itself, but the goal was to remove it for every user compilation going into ILLink.

Is there an extension point (e.g. a set of tasks/properties) by which the SDK could author such manifest automatically for every user created .fsproj which is going into illink ? (the language compiler itself should not know about illink)

[MichalStrehovsky]

but it's not meant (at least yet) to be a general feature.

so has that changed?

Yes, see context at #107341 (comment).

FSharp.Core already has this here: https://github.com/dotnet/fsharp/blob/9995898e73a174fbe094cdedbde11423ec4e33db/src/FSharp.Core/ILLink.Substitutions.xml

Is there an extension point (e.g. a set of tasks/properties) by which the SDK could author such manifest automatically for every user created .fsproj which is going into illink ? (the language compiler itself should not know about illink)

You'd need to add this to all fsproj files, not just those that go into illink. We don't know if something goes into illink if it's a library - library could be consumed anywhere. You'd basically default-include it as an <EmbeddedResource> item, similar to how FSharp.Core currently does it in the FSharp repo. Whether the compiler should do it or the F# targets should do it is up to you. I'd add it to the component that knows about these resources and their purpose. ILLink definitely doesn't know about them and should not be in business of removing them.

[MichalStrehovsky]

Just thinking out loud: alternative option could be not to store the FSharpSignatureData in embedded resources (that are readily accessible with .NET APIs and part of the ECMA-335 file format) but store it e.g. as a custom PE section in the executable (i.e. similar addon data as authenticode digital signatures etc.) that can be read by whoever wants to read it, but has no meaning to .NET.

Tools like ILLink would likely just drop such data automatically since it has no meaning for .NET and the tool doesn't know what to do with it. Then no XMLs would be needed.

[T-Gro]

Just thinking out loud: alternative option could be not to store the FSharpSignatureData in embedded resources (that are readily accessible with .NET APIs and part of the ECMA-335 file format) but store it e.g. as a custom PE section in the executable (i.e. similar addon data as authenticode digital signatures etc.) that can be read by whoever wants to read it, but has no meaning to .NET.

Tools like ILLink would likely just drop such data automatically since it has no meaning for .NET and the tool doesn't know what to do with it. Then no XMLs would be needed.

This really sounds like a good idea @MichalStrehovsky , those resources are only a mechanism to communicate between different instances (and versions!) of the F# compiler.

Due to time-to-live for existing compiler versions, we would need to introduce the "reading" side of this several years upfront and only then migrate, but of course it is a sensible idea.

To cover the period in between, could the team re-evaluate this PR that changes two string constants in existing logic?

[agocke]

Custom steps are considered legacy for the linker. We can't currently delete the ones we have (although that is a goal), but we are not adding any more and do not intend to support any new custom steps.

Since this issue is not a regression, does not break apps (just makes them larger than they may be otherwise), and relies on an undocumented feature (trimming resources) I don't think this meets the bar.

FWIW I think the ILLink subsitution embedding suggested above would be relatively simple to implement. Ideally we would delete this custom step entirely.

[T-Gro]

Thank you for your views.

One thing I am still not getting is how exactly you envision the integration point for automatically adding the .xml manifest to projects going into ILLink if they happen to be .fsproj projects, but not for all F# compilations.

Let's say a custom SDK target would be the right fit - how can it detect which projects and libraries are going to be illinked?
The same way IlLink is not interested in embedding knowledge about specific F# resources, a language compiler should not generate a tool-specific file for every compilation produced.

Is there an SDK hook to detect this correctly and not infect every build with it?

Also, is there a composable mechanism for having more of the ILLink.Substitutions.xml files, since users might be having their existing substitutions already?

[MichalStrehovsky]

Let's say a custom SDK target would be the right fit - how can it detect which projects and libraries are going to be illinked? The same way IlLink is not interested in embedding knowledge about specific F# resources, a language compiler should not generate a tool-specific file for every compilation produced.

Is there an SDK hook to detect this correctly and not infect every build with it?

It's not possible to detect this. Any OutputType=Library project could just be wrapped in a NuGet and then trimmed elsewhere later. This is the same problem as the existing FSharpSignatureData blobs this is trying to address: the F# compiler just embeds it everywhere even though this data would only used by the F# compiler; we don't know if the embedded data is going to be useful or just garbage.

Also, is there a composable mechanism for having more of the ILLink.Substitutions.xml files, since users might be having their existing substitutions already?

You'd need to merge the XMLs in that case.

Ultimately, the ILLink.Substitutions.xml format is a contract. The contract is implement for both PublishTrimmed and PublishAot (PublishAot does respect ILLink.Substitutions.xml but it does not run RemoveResourcesStep because PublishAot doesn't use ILLink to trim; trimming is done differently). Using the XML would be an improvement because both PublishTrimmed and PublishAot would respect that. Any fix to RemoveResourcesStep only benefits ILLinker because it's ILLinker implementation. We have no plans to add RemoveResourcesStep equivalent to ILC (tool that implements trimming for AOT).

[T-Gro]

So the idea would be to have the SDK targets modified for every F# compilation to always add an additional .xml content as a EmbeddedResource. Even though I do not like the size increase for everyone in order to help some, it is still better than creating a file-system file which would confuse unaware users and likely trigger unintended git commits etc.

Thanks for the idea, this seems to be a good compromise indeed.

Meta-question: Does PublishTrimmed and PublishAOT then automatically remove this EmbeddedResource ILLink.substitutions.xml manifest from the .dll , or would this then also need to go into the substitutions list as well?

One last thing I did not find at https://github.com/dotnet/runtime/blob/72a5ef7a9e5721d050623bacde82f8a384513966/docs/tools/illink/data-formats.md#remove-embedded-resources specifically for <resource> : Does ` support wildcards just like other elements do?

[MichalStrehovsky]

Meta-question: Does PublishTrimmed and PublishAOT then automatically remove this EmbeddedResource ILLink.substitutions.xml manifest from the .dll , or would this then also need to go into the substitutions list as well?

Yes, this embedded resource is defined as only having meaning for trimming and trimming will remove it.

One last thing I did not find at https://github.com/dotnet/runtime/blob/72a5ef7a9e5721d050623bacde82f8a384513966/docs/tools/illink/data-formats.md#remove-embedded-resources specifically for ` : Does `` support wildcards just like other elements do?

Only <type fullname="..." supports wildcards