Skip to content

[wasm] Standup EnableAgressiveTrimming for library tests #46367

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 48 commits into from
Jan 4, 2021

Conversation

mdh1418
Copy link
Member

@mdh1418 mdh1418 commented Dec 23, 2020

This PR looks to enable trimming of the Browser wasm library tests to reduce the managed library size as part of #45126.

An important step to reducing the size is to enable PublishTrimmed in the library tests and trim more aggressively by removing unused members from types via <TrimMode>link</TrimMode>.

The linker from the nuget package is imported to override the .NET SDK version of the linker that the mobile library tests use because the nuget package version includes fixes that have not made it to the .NET SDK version.
For example, System.ArgumentOutOfRangeException: Index was out of range. Must be non-negative and less than the size of the collection. (Parameter 'index') was an error that was resolved with the import override.

The ConfigureTrimming target had been added to force <TrimMode>link</TrimMode> of all assemblies except for the test assembly, which should be TrimMode copy. *xunit files were still not being linked out with <TrimMode>link</TrimMode> and resulted in numerous other assemblies being included, so they are linked out by manually adding the *xunit files to ManagedAssemblyToLink. Several assembly methods could not be tracked by the linker itself, so a linker description for xunit code had been added in eng/testing/ILLink.Descriptor.xunit.xml.

As a result of these addition, the library size for a test suite such as System.Buffers.Tests is reduced from 30MB to ~3MB.

The changes lie behind an EnableAggressiveTrimming flag so that CI will not yet run the tests in trimming mode.

Mitchell Hwang and others added 30 commits December 15, 2020 14:33
@mdh1418 mdh1418 added the arch-wasm WebAssembly architecture label Dec 23, 2020
@ghost
Copy link

ghost commented Dec 23, 2020

Tagging subscribers to 'arch-wasm': @lewing
See info in area-owners.md if you want to be subscribed.

Issue Details

This PR looks to enable trimming of the library tests to reduce the managed library size as part of #45126.

Author: mdh1418
Assignees: -
Labels:

arch-wasm

Milestone: -

@Dotnet-GitSync-Bot
Copy link
Collaborator

I couldn't figure out the best area label to add to this PR. If you have write-permissions please help me learn by adding exactly one area label.

…ker" as MainAssembly is back to $(PublishDir)\WasmTestRunner.dll

This reverts commit 79dc240.
Comment on lines +168 to +177
<ManagedAssemblyToLink Include="$(OutDir)\*xunit*">
<IsTrimmable>true</IsTrimmable>
<TrimMode>link</TrimMode>
</ManagedAssemblyToLink>
<ManagedAssemblyToLink Condition="('%(ManagedAssemblyToLink.FileName).dll' != '$(MSBuildProjectName).dll')" >
<TrimMode>link</TrimMode>
</ManagedAssemblyToLink>
<ManagedAssemblyToLink Condition="('%(ManagedAssemblyToLink.FileName).dll' == '$(AssemblyName).dll')" >
<TrimMode>copy</TrimMode>
</ManagedAssemblyToLink>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at https://github.com/dotnet/sdk/blob/master/src/Tasks/Microsoft.NET.Build.Tasks/targets/Microsoft.NET.ILLink.targets#L142-L144 , I think instead of this, we should have a target that adds to @(ResolvedFileToPublish), and sets the TrimMode metadata as needed.

This will get processed by https://github.com/dotnet/sdk/blob/master/src/Tasks/Microsoft.NET.Build.Tasks/targets/Microsoft.NET.ILLink.targets#L235-L237 and subsequently by PrepareForILLink target.

  • IIUC, you just need to add the xunit files, if they aren't there already, and set TrimMode=link for them.
  • And make sure that the test assembly doesn't have IsTrimmable, or TrimMode set. this should handle the rest for you.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using ResolvedFileToPublish makes sense if we can hook it early enough. Though I'd keep copy setting to be explicit and maybe add a comment that we want the tested assembly to be always copied only.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will address in a follow-up PR

Comment on lines +178 to +181
<TrimmerRootAssembly Condition="'$(TargetOS)' == 'Android'" Include="AndroidTestRunner"/>
<TrimmerRootAssembly Condition="'$(TargetOS)' == 'Browser'" Include="WasmTestRunner"/>
<TrimmerRootAssembly Condition="'$(TargetOS)' == 'iOS'" Include="AppleTestRunner"/>
<TrimmerRootAssembly Include="$(AssemblyName)"/>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These shouldn't be needed, I think, since the PrepareForILLink target will set these up for us.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are different settings. I think PrepareForILLink will set only <TrimmerRootAssembly Include="$(AssemblyName)"/>

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking that they might get picked up by

<_runnerFilesToPublish Include="$(AndroidTestRunnerDir)*" Condition="'$(TargetOS)' == 'Android'" />
<_runnerFilesToPublish Include="$(AppleTestRunnerDir)*" Condition="'$(TargetOS)' == 'iOS' or '$(TargetOS)' == 'tvOS'" />
<_runnerFilesToPublish Include="$(WasmTestRunnerDir)*" Condition="'$(TargetOS)' == 'Browser'" />
<ResolvedFileToPublish Include="@(_runnerFilesToPublish)" RelativePath="%(FileName)%(Extension)" CopyToPublishDirectory="PreserveNewest" PostprocessAssembly="true" />
into @(ResolvedFileToPublish)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just tried removing L181 <TrimmerRootAssembly Include="$(AssemblyName)"/> and started seeing

 fail: 10:28:48.0231240 System.AggregateException: AggregateException_ctor_DefaultMessage ()

  fail: 10:28:48.0232360  ---> System.IO.FileNotFoundException:

  fail: 10:28:48.0232420 IO_FileName_Name, System.Buffers.Tests

  fail: 10:28:48.0232440    at System.Reflection.Assembly.Load(AssemblyName assemblyRef, StackCrawlMark& stackMark, AssemblyLoadContext assemblyLoadContext)

  fail: 10:28:48.0232480    at System.Reflection.Assembly.Load(AssemblyName assemblyRef)

  fail: 10:28:48.0232610    at Xunit.Sdk.ReflectionAssemblyInfo..ctor(String assemblyFileName)

  fail: 10:28:48.0232650    at System.Reflection.RuntimeConstructorInfo.InternalInvoke(Object obj, Object[] parameters, Boolean wrapExceptions)

  fail: 10:28:48.0232660 --- End of stack trace from previous location ---

for ./dotnet.sh build /t:Test src/libraries/System.Buffers/tests/ /p:TargetOS=Browser /p:TargetArchitecture=wasm /p:Configuration=Debug /p:EnableAggressiveTrimming=true

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will address in a follow-up PR

@mdh1418 mdh1418 merged commit 1e6826a into dotnet:master Jan 4, 2021
@mdh1418 mdh1418 deleted the mdhwang/trim_library_tests branch January 4, 2021 21:22
@ghost ghost locked as resolved and limited conversation to collaborators Feb 3, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
arch-wasm WebAssembly architecture
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants