Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use live illink for trimming tests #88929

Merged
merged 12 commits into from
Aug 1, 2023
Merged

Use live illink for trimming tests #88929

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
77d0535
Use live illink for trimming tests
sbomer Jul 14, 2023
cfc3557
Fix import path
sbomer Jul 14, 2023
10b5b92
Fix NativeAot tests
sbomer Jul 14, 2023
503139d
Fix illink config for nativeaot tests
sbomer Jul 15, 2023
57bb464
Prevent double file copy
sbomer Jul 15, 2023
af43ff6
PR feedback
sbomer Jul 19, 2023
b406651
Merge remote-tracking branch 'origin/main' into liveILLink
sbomer Jul 19, 2023
8126cdb
Remove TFM condition
sbomer Jul 20, 2023
140a8bd
Import from Directory.Build.props
sbomer Jul 27, 2023
7ab3b07
Don't change outputpath
sbomer Jul 27, 2023
4763bc9
Remove ToolsArtifactsPath
sbomer Jul 28, 2023
8333694
Use NetCoreAppToolCurrent
sbomer Jul 31, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions eng/liveBuilds.targets
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
<MonoArtifactsPath Condition="'$(MonoArtifactsPath)' == ''">$([MSBuild]::NormalizeDirectory('$(RepoRoot)', 'artifacts', 'bin', 'mono', '$(TargetOS).$(TargetArchitecture).$(MonoConfiguration)'))</MonoArtifactsPath>
<LibrariesArtifactsPath Condition="'$(LibrariesArtifactsPath)' == ''">$([MSBuild]::NormalizeDirectory('$(RepoRoot)', 'artifacts'))</LibrariesArtifactsPath>
<LibrariesAllConfigurationsArtifactsPath Condition="'$(LibrariesAllConfigurationsArtifactsPath)' == ''">$([MSBuild]::NormalizeDirectory('$(RepoRoot)', 'artifacts'))</LibrariesAllConfigurationsArtifactsPath>
<ToolsArtifactsPath Condition="'$(ToolsArtifactsPath)' == ''">$([MSBuild]::NormalizeDirectory('$(RepoRoot)', 'artifacts'))</ToolsArtifactsPath>
sbomer marked this conversation as resolved.
Show resolved Hide resolved
</PropertyGroup>

<!-- Set up artifact subpaths. -->
Expand All @@ -29,6 +30,8 @@
<CoreCLRAotSdkDir>$([MSBuild]::NormalizeDirectory('$(CoreCLRArtifactsPath)', 'aotsdk'))</CoreCLRAotSdkDir>
<CoreCLRBuildIntegrationDir>$([MSBuild]::NormalizeDirectory('$(CoreCLRArtifactsPath)', 'build'))</CoreCLRBuildIntegrationDir>

<ToolsILLinkDir>$([MSBuild]::NormalizeDirectory('$(ToolsArtifactsPath)', 'bin', 'ILLink.Tasks', '$(Configuration)'))</ToolsILLinkDir>
sbomer marked this conversation as resolved.
Show resolved Hide resolved

<MonoAotCrossDir>$([MSBuild]::NormalizeDirectory('$(MonoArtifactsPath)', 'cross', $(TargetOS)-$(TargetArchitecture.ToLowerInvariant())))</MonoAotCrossDir>
<GrpcServerDockerImageDir>$([MSBuild]::NormalizeDirectory('$(LibrariesArtifactsPath)', 'obj', 'grpcserver', 'docker'))</GrpcServerDockerImageDir>

Expand Down
4 changes: 2 additions & 2 deletions eng/pipelines/runtime-linker-tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@ extends:
or(
eq(dependencies.evaluate_paths.outputs['SetPathVars_non_mono_and_wasm.containsChange'], true),
eq(variables['isRollingBuild'], true))
buildArgs: -s clr+libs -c $(_BuildConfig)
buildArgs: -s clr+libs+tools.illink -c $(_BuildConfig)
extraStepsTemplate: /eng/pipelines/libraries/execute-trimming-tests-steps.yml

#
Expand All @@ -118,7 +118,7 @@ extends:
testGroup: innerloop
timeoutInMinutes: 120
nameSuffix: Runtime_Release
buildArgs: -s mono+libs -c $(_BuildConfig) -p:WasmBuildNative=false -p:AotHostArchitecture=x64 -p:AotHostOS=$(_hostedOS)
buildArgs: -s mono+libs+tools.illink -c $(_BuildConfig) -p:WasmBuildNative=false -p:AotHostArchitecture=x64 -p:AotHostOS=$(_hostedOS)
condition:
or(
eq(variables['isRollingBuild'], true),
Expand Down
2 changes: 1 addition & 1 deletion eng/pipelines/runtime.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -286,7 +286,7 @@ extends:
testGroup: innerloop
timeoutInMinutes: 120
nameSuffix: NativeAOT
buildArgs: -s clr.aot+host.native+libs -rc $(_BuildConfig) -lc Release -hc Release
buildArgs: -s clr.aot+host.native+libs+tools.illink -c $(_BuildConfig) -rc $(_BuildConfig) -lc Release -hc Release
sbomer marked this conversation as resolved.
Show resolved Hide resolved
extraStepsTemplate: /eng/pipelines/coreclr/nativeaot-post-build-steps.yml
extraStepsParameters:
creator: dotnet-bot
Expand Down
1 change: 1 addition & 0 deletions eng/testing/linker/SupportFiles/Directory.Build.targets
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
<Project>
<Import Project="$(ToolsILLinkDir)build/Microsoft.NET.ILLink.Tasks.props" />
sbomer marked this conversation as resolved.
Show resolved Hide resolved
<Import Project="$(RepositoryEngineeringDir)testing\tests.mobile.targets" Condition="'$(RuntimeIdentifier)' == 'browser-wasm'" />
<Import Project="$(RepositoryEngineeringDir)targetingpacks.targets" />

Expand Down
12 changes: 3 additions & 9 deletions eng/testing/linker/project.csproj.template
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,9 @@
<_ExtraTrimmerArgs>{ExtraTrimmerArgs} $(_ExtraTrimmerArgs)</_ExtraTrimmerArgs>
{AdditionalProperties}

<!-- Needed for PublishTrimmed -->
<ToolsILLinkDir>{ToolsILLinkDir}</ToolsILLinkDir>

<!-- Needed for PublishAot -->
<IlcToolsPath>{IlcToolsPath}</IlcToolsPath>
<IlcBuildTasksPath>{IlcBuildTasksPath}</IlcBuildTasksPath>
Expand All @@ -52,15 +55,6 @@
{AdditionalProjectReferences}
</ItemGroup>

<!-- Logic to override the default IlLink tasks that come from the SDK and use the one
we use in dotnet/runtime repo -->
<ItemGroup>
<PackageReference Include="Microsoft.NET.ILLink.Tasks"
Version="{MicrosoftNETILLinkTasksVersion}"
PrivateAssets="all"
IsImplicitlyDefined="true" />
</ItemGroup>

<Target Name="LocateNativeCompiler"
Condition="'$(PublishAot)' == 'true' and '$(_hostOS)' != 'win'"
BeforeTargets="SetupOSSpecificProps">
Expand Down
2 changes: 1 addition & 1 deletion eng/testing/linker/trimmingTests.targets
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,9 +84,9 @@
.Replace('{UseMonoRuntime}','$(UseMonoRuntime)')
.Replace('{RuntimeIdentifier}','%(TestConsoleApps.TestRuntimeIdentifier)')
.Replace('{PublishAot}','$(IsNativeAotTestProject)')
.Replace('{MicrosoftNETILLinkTasksVersion}', '$(MicrosoftNETILLinkTasksVersion)')
.Replace('{ExtraTrimmerArgs}', '%(TestConsoleApps.ExtraTrimmerArgs)')
.Replace('{AdditionalProperties}', '$(_additionalPropertiesString)')
.Replace('{ToolsILLinkDir}', '$(ToolsILLinkDir)')
.Replace('{IlcToolsPath}', '$(CoreCLRILCompilerDir)')
.Replace('{IlcBuildTasksPath}', '$(CoreCLRILCompilerDir)netstandard/ILCompiler.Build.Tasks.dll')
.Replace('{IlcSdkPath}', '$(CoreCLRAotSdkDir)')
Expand Down
8 changes: 0 additions & 8 deletions src/tools/illink/src/ILLink.RoslynAnalyzer/Directory.Build.props
View file
Open in desktop

This file was deleted.

2 changes: 2 additions & 0 deletions src/tools/illink/src/ILLink.RoslynAnalyzer/ILLink.RoslynAnalyzer.csproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@
<AnalysisLevel>Latest</AnalysisLevel>
<NoWarn Condition="'$(DotNetBuildFromSource)' == 'true'">$(NoWarn);CS8524</NoWarn>
<AnalyzerLanguage>cs</AnalyzerLanguage>
<!-- This analyzer doesn't follow the convention for generator projects so we set the property manually. -->
<IsGeneratorProject>true</IsGeneratorProject>
sbomer marked this conversation as resolved.
Show resolved Hide resolved
</PropertyGroup>

<ItemGroup>
Expand Down
0 ...yzer/Microsoft.NET.ILLink.Analyzers.props → ...uild/Microsoft.NET.ILLink.Analyzers.props
View file
Open in desktop
File renamed without changes.
8 changes: 6 additions & 2 deletions src/tools/illink/src/ILLink.Tasks/ILLink.Tasks.csproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@
NU5129: Targets don't match the package id. This is intentional. -->
<NoWarn>$(NoWarn);NU5128;NU5129</NoWarn>
<IncludeBuildOutput>false</IncludeBuildOutput>
<!-- Place build output into a layout that matches the package layout. -->
<OutputPath>$(OutputPath)/tools</OutputPath>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The build and package output paths are intentionally not aligned. Why do you need the tools subdirectory for the output path?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm trying to set it up so that the tests can just import Microsoft.NET.ILLink.Tasks.props directly from the build output path, and have this logic work without further overrides:

runtime/src/tools/illink/src/ILLink.Tasks/build/Microsoft.NET.ILLink.Tasks.props

Line 21 in fb2ae67

<ILLinkTasksAssembly Condition="'$(MSBuildRuntimeType)' == 'Core'">$(MSBuildThisFileDirectory)..\tools\net8.0\ILLink.Tasks.dll</ILLinkTasksAssembly>

Let me know if you'd prefer another approach.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should keep the output path and package layout separate, otherwise we will hit other issues over time. Please revert the OutputPath change.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reverted this change. It unfortunately means using live illink isn't as simple as a single import statement. I'm curious if you have specific concerns over the approach of matching the package layout in the output?

<IncludeMultiTargetRoslynComponentTargets>false</IncludeMultiTargetRoslynComponentTargets>
<TargetsForTfmSpecificContentInPackage>$(TargetsForTfmSpecificContentInPackage);AddBuildOutputToToolsPackage</TargetsForTfmSpecificContentInPackage>
<!-- Disable analyzer to prevent a circular dependency due to PackageReference to the LKG Microsoft.NET.ILLink.Tasks. -->
Expand All @@ -29,8 +31,10 @@
<ItemGroup>
<!-- Note: 'build/Microsoft.NET.ILLink.targets' should not match the package name, because we don't want the targets
to be imported by nuget. The SDK will import them in the right order. -->
<None Include="**\*.props;**\*.targets" Pack="true" PackagePath="%(RecursiveDir)%(Filename)%(Extension)" />
<None Include="..\ILLink.RoslynAnalyzer\Microsoft.NET.ILLink.Analyzers.props" Pack="true" PackagePath="build/" />
<Content Include="**\*.props;**\*.targets" Pack="true" PackagePath="%(RecursiveDir)%(Filename)%(Extension)"
CopyToOutputDirectory="PreserveNewest" TargetPath="../../%(RecursiveDir)%(Filename)%(Extension)" />
<Content Include="..\ILLink.RoslynAnalyzer\**\*.props" Pack="true" PackagePath="%(RecursiveDir)%(Filename)%(Extension)"
CopyToOutputDirectory="PreserveNewest" TargetPath="../../%(RecursiveDir)%(Filename)%(Extension)" />
</ItemGroup>

<ItemGroup>
Expand Down
Loading