Skip to content

Produce a package with the host-running ILC for repos in the VMR #111443

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jan 17, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion eng/Publishing.props
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,12 @@
<ItemGroup Condition="'$(DotNetBuildOrchestrator)' == 'true'">
<Artifact Condition="'$(RuntimeFlavor)' != 'Mono' and '$(OutputRID)' != '$(NETCoreSdkRuntimeIdentifier)'"
Update="$(ArtifactsPackagesDir)**\Microsoft.NETCore.App.Crossgen2.$(NETCoreSdkRuntimeIdentifier).*.nupkg"
Visibility="Vertical" />
Visibility="Vertical"
IsShipping="false" />
<Artifact Condition="'$(RuntimeFlavor)' != 'Mono' and '$(OutputRID)' != '$(NETCoreSdkRuntimeIdentifier)'"
Update="$(ArtifactsPackagesDir)**\runtime.$(NETCoreSDKRuntimeIdentifier).Microsoft.DotNet.ILCompiler.*.nupkg"
Visibility="Vertical"
IsShipping="false" />
</ItemGroup>

<!--
Expand Down
5 changes: 3 additions & 2 deletions eng/Subsets.props
Original file line number Diff line number Diff line change
Expand Up @@ -391,7 +391,7 @@
<ProjectToBuild Include="$(CoreClrProjectRoot)tools\aot\ILCompiler\ILCompiler.csproj" Category="clr" Condition="'$(NativeAotSupported)' == 'true' and '$(StageOneBuild)' != 'true'" />
<ProjectToBuild Include="$(CoreClrProjectRoot)nativeaot\BuildIntegration\BuildIntegration.proj" Category="clr" Condition="'$(NativeAotSupported)' == 'true'" />

<ProjectToBuild Condition="'$(NativeAotSupported)' == 'true' and ('$(CrossBuild)' == 'true' or '$(BuildArchitecture)' != '$(TargetArchitecture)' or '$(HostOS)' != '$(TargetOS)' or '$(EnableNativeSanitizers)' != '')" Include="$(CoreClrProjectRoot)tools\aot\ILCompiler\ILCompiler_inbuild.csproj" Category="clr" />
<ProjectToBuild Include="$(CoreClrProjectRoot)tools\aot\ILCompiler\ILCompiler_inbuild.csproj" Category="clr" />

<ProjectToBuild Include="$(CoreClrProjectRoot)tools\aot\crossgen2\crossgen2_inbuild.csproj" Category="clr" />

Expand Down Expand Up @@ -576,6 +576,7 @@
<_BuildNativeAOTRuntimePack Condition="'$(BuildNativeAOTRuntimePack)' == 'true'">true</_BuildNativeAOTRuntimePack>
<_BuildHostPack Condition="'$(RuntimeFlavor)' == '$(PrimaryRuntimeFlavor)' and '$(TargetsMobile)' != 'true' and '$(BuildNativeAOTRuntimePack)' != 'true'">true</_BuildHostPack>
<_BuildBundle Condition="'$(BuildNativeAOTRuntimePack)' != 'true' and '$(RuntimeFlavor)' == '$(PrimaryRuntimeFlavor)' and '$(TargetsMobile)' != 'true'">true</_BuildBundle>
<BuildHostTools Condition="'$(DotNetBuildOrchestrator)' == 'true' and '$(OutputRID)' != '$(NETCoreSdkRuntimeIdentifier)'">true</BuildHostTools>
</PropertyGroup>

<PropertyGroup Condition="'$(DotNetBuildAllRuntimePacks)' == 'true' and '$(DotNetBuildSourceOnly)' != 'true'">
Expand Down Expand Up @@ -613,7 +614,7 @@
When we're building in the VMR, we need to provide a crossgen2 that runs on the host machine for downstream repos to use to R2R their code.
In non-VMR builds, downstream repos can use the crossgen2 built for the target host SDK from another build leg, but in the VMR we need to provide one to use.
-->
<ProjectToBuild Condition="'$(RuntimeFlavor)' != 'Mono' and '$(TargetsMobile)' != 'true' and '$(TargetsLinuxBionic)' != 'true' and '$(DotNetBuildOrchestrator)' == 'true' and '$(OutputRID)' != '$(NETCoreSdkRuntimeIdentifier)'" Include="$(InstallerProjectRoot)pkg\sfx\Microsoft.NETCore.App\Microsoft.NETCore.App.Crossgen2.Host.sfxproj" Category="packs" />
<ProjectToBuild Condition="'$(RuntimeFlavor)' != 'Mono' and '$(TargetsMobile)' != 'true' and '$(TargetsLinuxBionic)' != 'true' and '$(BuildHostTools)' == 'true'" Include="$(InstallerProjectRoot)pkg\sfx\Microsoft.NETCore.App\Microsoft.NETCore.App.Crossgen2.Host.sfxproj" Category="packs" />
</ItemGroup>
<ItemGroup>
<SharedFrameworkProjectToBuild Condition="'$(_BuildHostPack)' == 'true'" Include="$(InstallerProjectRoot)pkg\archives\dotnet-nethost.proj" />
Expand Down
2 changes: 1 addition & 1 deletion eng/liveBuilds.targets
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
<PropertyGroup>
<CoreCLRSharedFrameworkDir>$([MSBuild]::NormalizeDirectory('$(CoreCLRArtifactsPath)', 'sharedFramework'))</CoreCLRSharedFrameworkDir>
<CoreCLRILCompilerDir>$([MSBuild]::NormalizeDirectory('$(CoreCLRArtifactsPath)', 'ilc-published'))</CoreCLRILCompilerDir>
<CoreCLRCrossILCompilerDir Condition="'$(CrossBuild)' == 'true' or '$(BuildArchitecture)' != '$(TargetArchitecture)' or '$(HostOS)' != '$(TargetOS)' or '$(EnableNativeSanitizers)' != ''">$([MSBuild]::NormalizeDirectory('$(CoreCLRArtifactsPath)', '$(BuildArchitecture)', 'ilc'))</CoreCLRCrossILCompilerDir>
<CoreCLRCrossILCompilerDir>$([MSBuild]::NormalizeDirectory('$(CoreCLRArtifactsPath)', '$(BuildArchitecture)', 'ilc'))</CoreCLRCrossILCompilerDir>
<CoreCLRAotSdkDir>$([MSBuild]::NormalizeDirectory('$(CoreCLRArtifactsPath)', 'aotsdk'))</CoreCLRAotSdkDir>
<CoreCLRBuildIntegrationDir>$([MSBuild]::NormalizeDirectory('$(CoreCLRArtifactsPath)', 'build'))</CoreCLRBuildIntegrationDir>

Expand Down
3 changes: 1 addition & 2 deletions eng/testing/linker/trimmingTests.targets
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
<Project>
<PropertyGroup>
<IlcToolsPath>$(CoreCLRILCompilerDir)</IlcToolsPath>
<IlcToolsPath Condition="'$(TargetArchitecture)' != '$(BuildArchitecture)' or '$(EnableNativeSanitizers)' != ''">$(CoreCLRCrossILCompilerDir)</IlcToolsPath>
<IlcToolsPath>$(CoreCLRCrossILCompilerDir)</IlcToolsPath>
</PropertyGroup>

<ItemGroup>
Expand Down
3 changes: 1 addition & 2 deletions eng/testing/tests.singlefile.targets
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,7 @@
</PropertyGroup>

<PropertyGroup Condition="'$(TestNativeAot)' == 'true'">
<IlcToolsPath>$(CoreCLRILCompilerDir)</IlcToolsPath>
<IlcToolsPath Condition="'$(CrossBuild)' == 'true' or '$(TargetArchitecture)' != '$(BuildArchitecture)' or '$(EnableNativeSanitizers)' != ''">$(CoreCLRCrossILCompilerDir)</IlcToolsPath>
<IlcToolsPath>$(CoreCLRCrossILCompilerDir)</IlcToolsPath>
<SysRoot Condition="('$(CrossBuild)' == 'true' or '$(BuildArchitecture)' != '$(TargetArchitecture)') and '$(HostOS)' != 'windows'">$(ROOTFS_DIR)</SysRoot>
<IlcBuildTasksPath>$(CoreCLRILCompilerDir)netstandard/ILCompiler.Build.Tasks.dll</IlcBuildTasksPath>
<IlcSdkPath>$(CoreCLRAotSdkDir)</IlcSdkPath>
Expand Down
23 changes: 19 additions & 4 deletions src/coreclr/tools/aot/ILCompiler/ILCompiler_inbuild.csproj
Original file line number Diff line number Diff line change
@@ -1,8 +1,23 @@
<Project Sdk="Microsoft.NET.Sdk">
<Project Sdk="Microsoft.NET.Sdk" DefaultTargets="Publish;PublishItemsOutputGroup">
<PropertyGroup>
<CrossHostArch>$(BuildArchitecture)</CrossHostArch>
<OutputPath>$(RuntimeBinDir)/$(CrossHostArch)/ilc/</OutputPath>
<RuntimeIdentifier>$(ToolsRID)</RuntimeIdentifier>
<CrossHostArch Condition="'$(CrossBuild)' == 'true' or '$(TargetArchitecture)' != '$(BuildArchitecture)' or '$(HostOS)' != '$(TargetOS)' or '$(EnableNativeSanitizers)' != ''">$(BuildArchitecture)</CrossHostArch>
<_IsPublishing>true</_IsPublishing>
<RuntimeIdentifier>$(NETCoreSdkRuntimeIdentifier)</RuntimeIdentifier>
<PublishDir>$(RuntimeBinDir)$(BuildArchitecture)/ilc/</PublishDir>
<SelfContained>true</SelfContained>
<PublishTrimmed>true</PublishTrimmed>
<!--
Publish with AOT on Windows as Single file doesn't work well when an app has to include DiaSymReader.Native.
Publish single-file elsewhere so we don't need to adjust our "AOT in build" logic to handle targeting the SDK RID.
-->
<PublishAot Condition="$(NETCoreSdkPortableRuntimeIdentifier.StartsWith('win-'))">true</PublishAot>
<PublishSingleFile Condition="'$(PublishAot)' != 'true'">true</PublishSingleFile>
<TrimmerSingleWarn>false</TrimmerSingleWarn>
</PropertyGroup>

<PropertyGroup Condition="'$(CrossBuild)' != 'true' or ('$(StageOneBuild)' != 'true' and '$(StageTwoBuild)' != 'true')">
<PublishReadyToRun>true</PublishReadyToRun>
<PublishReadyToRunComposite>true</PublishReadyToRunComposite>
</PropertyGroup>
<Import Project="ILCompiler.props" />
</Project>
Original file line number Diff line number Diff line change
Expand Up @@ -19,11 +19,52 @@
<ExcludeLineupReference Condition="'$(DotNetBuildSourceOnly)' == 'true'">true</ExcludeLineupReference>
</PropertyGroup>

<Target Name="AddBuildIntegrationFiles"
DependsOnTargets="ResolveLibrariesFromLocalBuild"
BeforeTargets="GetPackageFiles"
Condition="'$(PackageTargetRuntime)' == ''">
<ItemGroup>
<File Include="$(CoreCLRBuildIntegrationDir)*" TargetPath="build" />
<File Include="$(CoreCLRILCompilerDir)netstandard\*" TargetPath="tools/netstandard" />
</ItemGroup>
</Target>

<Target Name="GetHostIlcCompilerFiles"
DependsOnTargets="ResolveLibrariesFromLocalBuild"
BeforeTargets="GetPackageFiles"
Condition="'$(PackHostILCompiler)' == 'true' and '$(PackageTargetRuntime)' != ''">

<ItemGroup>
<File Include="$(CoreCLRCrossILCompilerDir)*" TargetPath="tools" />
</ItemGroup>

<ItemGroup Condition="'$(TargetOS)' == 'linux'">
<File Include="$(SharedNativeRoot)libs\System.Globalization.Native\*" TargetPath="native/src/libs/System.Globalization.Native"/>
<File Include="$(SharedNativeRoot)libs\System.Security.Cryptography.Native\*" TargetPath="native/src/libs/System.Security.Cryptography.Native"/>
<File Include="$(SharedNativeRoot)libs\build-local.sh" TargetPath="native/src/libs/build-local.sh"/>
<File Include="$(SharedNativeRoot)minipal\*" TargetPath="native/src/minipal"/>
<File Include="$(SharedNativeRoot)libs\Common\*" TargetPath="native/src/libs/Common"/>
</ItemGroup>

<!-- exclude native symbols from ilc package (they are included in symbols package) -->
<ItemGroup>
<!-- on windows, remove the pdbs only from tools directory (both managed and native) -->
<LibPackageExcludes Include="tools\%2A%2A\%2A.pdb"/>

<LibPackageExcludes Include="%2A%2A\%2A.dbg"/>
<LibPackageExcludes Include="%2A%2A\%2A.debug"/>
<LibPackageExcludes Include="%2A%2A\%2A.dSYM"/>
<LibPackageExcludes Include="%2A%2A\%2A.dwarf"/>
</ItemGroup>

</Target>

<Target Name="GetIlcCompilerFiles"
DependsOnTargets="ResolveLibrariesFromLocalBuild"
BeforeTargets="GetPackageFiles">
BeforeTargets="GetPackageFiles"
Condition="'$(PackHostILCompiler)' != 'true' and '$(PackageTargetRuntime)' != ''">

<ItemGroup Condition="'$(PackageTargetRuntime)' != ''">
<ItemGroup>
<!-- Include all libraries (managed, static, etc.), but exclude native shared libraries (those with LibSuffix) -->
<File Include="@(LibrariesRuntimeFiles)" TargetPath="framework" Condition="'%(LibrariesRuntimeFiles.IsNative)' != 'true' or '%(LibrariesRuntimeFiles.Extension)' != '$(LibSuffix)'" />

Expand All @@ -32,19 +73,14 @@
<File Include="$(MibcOptimizationDataDir)/$(TargetOS)/$(TargetArchitecture)/**/*.mibc" TargetPath="mibc" />
</ItemGroup>

<ItemGroup Condition="'$(PackageTargetRuntime)' != '' and '$(TargetOS)' == 'linux'">
<ItemGroup Condition="'$(TargetOS)' == 'linux'">
<File Include="$(SharedNativeRoot)libs\System.Globalization.Native\*" TargetPath="native/src/libs/System.Globalization.Native"/>
<File Include="$(SharedNativeRoot)libs\System.Security.Cryptography.Native\*" TargetPath="native/src/libs/System.Security.Cryptography.Native"/>
<File Include="$(SharedNativeRoot)libs\build-local.sh" TargetPath="native/src/libs/build-local.sh"/>
<File Include="$(SharedNativeRoot)minipal\*" TargetPath="native/src/minipal"/>
<File Include="$(SharedNativeRoot)libs\Common\*" TargetPath="native/src/libs/Common"/>
</ItemGroup>

<ItemGroup Condition="'$(PackageTargetRuntime)' == ''">
<File Include="$(CoreCLRBuildIntegrationDir)*" TargetPath="build" />
<File Include="$(CoreCLRILCompilerDir)netstandard\*" TargetPath="tools/netstandard" />
</ItemGroup>

<!-- exclude native symbols from ilc package (they are included in symbols package) -->
<ItemGroup>
<!-- on windows, remove the pdbs only from tools directory (both managed and native) -->
Expand Down
1 change: 1 addition & 0 deletions src/installer/pkg/projects/nativeaot-packages.proj
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
<Project Include="Microsoft.DotNet.ILCompiler\Microsoft.DotNet.ILCompiler.pkgproj" />
<ProjectReference Include="@(Project)" />
<ProjectReference Include="@(Project)" AdditionalProperties="PackageTargetRuntime=$(OutputRID)" />
<ProjectReference Condition="'$(BuildHostTools)' == 'true'" Include="@(Project)" AdditionalProperties="PackageTargetRuntime=$(NETCoreSdkRuntimeIdentifier);PackHostILCompiler=true" />
</ItemGroup>

<Import Sdk="Microsoft.Build.Traversal" Project="Sdk.targets" />
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@
<PropertyGroup>
<CrossgenPublishProject>$(RepoRoot)src/coreclr/tools/aot/crossgen2/crossgen2_inbuild.csproj</CrossgenPublishProject>
<RuntimeIdentifier>$(NETCoreSdkRuntimeIdentifier)</RuntimeIdentifier>
<IsShipping>false</IsShipping>
</PropertyGroup>

<Import Project="$(MSBuildThisFileDirectory)Microsoft.NETCore.App.Crossgen2.props" />
Expand Down
3 changes: 1 addition & 2 deletions src/tests/Directory.Build.targets
Original file line number Diff line number Diff line change
Expand Up @@ -565,8 +565,7 @@

<IlcTreatWarningsAsErrors>false</IlcTreatWarningsAsErrors>

<IlcToolsPath>$(CoreCLRILCompilerDir)</IlcToolsPath>
<IlcToolsPath Condition="'$(TargetArchitecture)' != '$(BuildArchitecture)' or '$(TargetOS)' != '$(HostOS)' or '$(EnableNativeSanitizers)' != ''">$(CoreCLRCrossILCompilerDir)</IlcToolsPath>
<IlcToolsPath>$(CoreCLRCrossILCompilerDir)</IlcToolsPath>
<IlcBuildTasksPath>$(CoreCLRILCompilerDir)netstandard/ILCompiler.Build.Tasks.dll</IlcBuildTasksPath>
<IlcSdkPath>$(CoreCLRAotSdkDir)</IlcSdkPath>
<IlcFrameworkPath>$(MicrosoftNetCoreAppRuntimePackRidLibTfmDir)</IlcFrameworkPath>
Expand Down