Skip to content

Commit

Permalink
Merge commit 'cb77171f85a1431e67525922a53621d7475c5ebd'
Browse files Browse the repository at this point in the history
  • Loading branch information
Mirroring committed Sep 10, 2024
2 parents 378b27e + cb77171 commit a8108c9
Show file tree
Hide file tree
Showing 103 changed files with 850 additions and 295 deletions.
5 changes: 5 additions & 0 deletions NuGet.config
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,11 @@
<!--Begin: Package sources managed by Dependency Flow automation. Do not edit the sources below.-->
<!-- Begin: Package sources from dotnet-emsdk -->
<add key="darc-pub-dotnet-emsdk-d667257" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/darc-pub-dotnet-emsdk-d6672570/nuget/v3/index.json" />
<add key="darc-pub-dotnet-emsdk-2674f58" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/darc-pub-dotnet-emsdk-2674f580/nuget/v3/index.json" />
<add key="darc-pub-dotnet-emsdk-2674f58-4" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/darc-pub-dotnet-emsdk-2674f580-4/nuget/v3/index.json" />
<add key="darc-pub-dotnet-emsdk-2674f58-3" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/darc-pub-dotnet-emsdk-2674f580-3/nuget/v3/index.json" />
<add key="darc-pub-dotnet-emsdk-2674f58-2" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/darc-pub-dotnet-emsdk-2674f580-2/nuget/v3/index.json" />
<add key="darc-pub-dotnet-emsdk-2674f58-1" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/darc-pub-dotnet-emsdk-2674f580-1/nuget/v3/index.json" />
<!-- End: Package sources from dotnet-emsdk -->
<!-- Begin: Package sources from dotnet-sdk -->
<!-- End: Package sources from dotnet-sdk -->
Expand Down
8 changes: 8 additions & 0 deletions docs/coding-guidelines/libraries-packaging.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,14 @@ Source generators and analyzers can be included in the shared framework by addin

Removing a library from the shared framework is a breaking change and should be avoided.

### References to libraries in the shared framework that produce packages

It's beneficial to avoid project references to libraries that are in the shared framework because it makes the package graph smaller which reduces the number of packages that require servicing and the number of libraries that end up being copied into the application directory.

If a dependency is part of the shared framework a project/package reference is never required on the latest version (`NetCoreAppCurrent`). A reference is required for previous .NET versions even if the dependency is part of the shared framework if the project you are building targets .NETStandard and references the project there. You may completely avoid a package dependency on .NETStandard and .NET if it's not needed for .NETStandard (for example - if it is an implementation only dependency and you're building a PNSE assembly for .NETStandard).

Warning NETPKG0001 is emitted when you have an unnecessary reference to a library that is part of the shared framework. To avoid this warning, make sure your ProjectReference is conditioned so that it doesn't apply on `NetCoreAppCurrent`.

## Transport package

Transport packages are non-shipping packages that dotnet/runtime produces in order to share binaries with other repositories.
Expand Down
4 changes: 4 additions & 0 deletions docs/project/library-servicing.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,10 @@ If a library is packable (check for the `<IsPackable>true</IsPackable>` property

When you make a change to a library & ship it during the servicing release, the `ServicingVersion` must be bumped. This property is found in the library's source project. It's also possible that the property is not in that file, in which case you'll need to add it to the library's source project and set it to 1. If the property is already present in your library's source project, just increment the servicing version by 1.

## Optionally ensure all up-stack packages are also produced

If you wish to ensure that every package that references a serviced package is also serviced itself, you can enable validation by setting `ServiceTransitiveDependencies` to true. This can be done in an individual project, or globally. When doing this then building the repo, eg: `build libs -allConfigurations` you'll see errors from any project that didn't enable servicing. Reasons for forcing packages which depend on your package to service are security servicing or removing dependencies.

## Test your changes

All that's left is to ensure that your changes have worked as expected. To do so, execute the following steps:
Expand Down
13 changes: 7 additions & 6 deletions docs/workflow/building/coreclr/linux-instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,12 +60,13 @@ All official builds are cross-builds with a rootfs for the target OS, and will u

| Host OS | Target OS | Target Arch | Image location | crossrootfs location |
| --------------------- | ------------ | --------------- | -------------------------------------------------------------------------------- | -------------------- |
| CBL-mariner 2.0 (x64) | Alpine 3.13 | x64 | `mcr.microsoft.com/dotnet-buildtools/prereqs:cbl-mariner-2.0-cross-amd64-alpine` | `/crossrootfs/x64` |
| CBL-mariner 2.0 (x64) | Ubuntu 16.04 | x64 | `mcr.microsoft.com/dotnet-buildtools/prereqs:cbl-mariner-2.0-cross-amd64` | `/crossrootfs/x64` |
| CBL-mariner 2.0 (x64) | Alpine | arm32 (armhf) | `mcr.microsoft.com/dotnet-buildtools/prereqs:cbl-mariner-2.0-cross-arm-alpine` | `/crossrootfs/arm` |
| CBL-mariner 2.0 (x64) | Ubuntu 16.04 | arm32 (armhf) | `mcr.microsoft.com/dotnet-buildtools/prereqs:cbl-mariner-2.0-cross-arm` | `/crossrootfs/arm` |
| CBL-mariner 2.0 (x64) | Alpine | arm64 (arm64v8) | `mcr.microsoft.com/dotnet-buildtools/prereqs:cbl-mariner-2.0-cross-arm64-alpine` | `/crossrootfs/arm64` |
| CBL-mariner 2.0 (x64) | Ubuntu 16.04 | arm64 (arm64v8) | `mcr.microsoft.com/dotnet-buildtools/prereqs:cbl-mariner-2.0-cross-arm64` | `/crossrootfs/arm64` |
| Azure Linux (x64) | Alpine 3.13 | x64 | `mcr.microsoft.com/dotnet-buildtools/prereqs:azurelinux-3.0-net8.0-cross-amd64-alpine` | `/crossrootfs/x64` |
| Azure Linux (x64) | Ubuntu 16.04 | x64 | `mcr.microsoft.com/dotnet-buildtools/prereqs:azurelinux-3.0-net8.0-cross-amd64` | `/crossrootfs/x64` |
| Azure Linux (x64) | Alpine | arm32 (armhf) | `mcr.microsoft.com/dotnet-buildtools/prereqs:azurelinux-3.0-net8.0-cross-arm-alpine` | `/crossrootfs/arm` |
| Azure Linux (x64) | Ubuntu 16.04 | arm32 (armhf) | `mcr.microsoft.com/dotnet-buildtools/prereqs:azurelinux-3.0-net8.0-cross-arm` | `/crossrootfs/arm` |
| Azure Linux (x64) | Alpine | arm64 (arm64v8) | `mcr.microsoft.com/dotnet-buildtools/prereqs:azurelinux-3.0-net8.0-cross-arm64-alpine` | `/crossrootfs/arm64` |
| Azure Linux (x64) | Ubuntu 16.04 | arm64 (arm64v8) | `mcr.microsoft.com/dotnet-buildtools/prereqs:azurelinux-3.0-net8.0-cross-arm64` | `/crossrootfs/arm64` |
| Azure Linux (x64) | Ubuntu 16.04 | x86 | `mcr.microsoft.com/dotnet-buildtools/prereqs:azurelinux-3.0-net8.0-cross-x86` | `/crossrootfs/x86` |
| Ubuntu 18.04 (x64) | FreeBSD | x64 | `mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-cross-freebsd-12` | `/crossrootfs/x64` |

These Docker images are built using the Dockerfiles maintained in the [dotnet-buildtools-prereqs-docker repo](https://github.com/dotnet/dotnet-buildtools-prereqs-docker).
Expand Down
8 changes: 4 additions & 4 deletions eng/Version.Details.xml
Original file line number Diff line number Diff line change
Expand Up @@ -99,9 +99,9 @@
<Sha>d6672570f0fde2a4a3c7e65d60cec9f8406b039c</Sha>
<SourceBuild RepoName="emsdk" ManagedOnly="true" />
</Dependency>
<Dependency Name="Microsoft.SourceBuild.Intermediate.source-build-reference-packages" Version="8.0.0-alpha.1.24372.3">
<Dependency Name="Microsoft.SourceBuild.Intermediate.source-build-reference-packages" Version="8.0.0-alpha.1.24415.1">
<Uri>https://github.com/dotnet/source-build-reference-packages</Uri>
<Sha>30ed464acd37779c64e9dc652d4460543ebf9966</Sha>
<Sha>fe3794a68bd668d36d4d5014a9e6c9d22c0e6d86</Sha>
<SourceBuild RepoName="source-build-reference-packages" ManagedOnly="true" />
</Dependency>
<Dependency Name="Microsoft.SourceBuild.Intermediate.source-build-externals" Version="8.0.0-alpha.1.24379.1">
Expand Down Expand Up @@ -354,9 +354,9 @@
<Uri>https://dev.azure.com/dnceng/internal/_git/dotnet-optimization</Uri>
<Sha>67613417f5e1af250e6ddfba79f8f2885d8e90fb</Sha>
</Dependency>
<Dependency Name="Microsoft.DotNet.HotReload.Utils.Generator.BuildTool" Version="8.0.0-alpha.0.24402.1">
<Dependency Name="Microsoft.DotNet.HotReload.Utils.Generator.BuildTool" Version="8.0.0-alpha.0.24453.2">
<Uri>https://github.com/dotnet/hotreload-utils</Uri>
<Sha>907017dae648b642c122f9a34573bd88ea0d9730</Sha>
<Sha>5339e12def2a3605d069c429840089ae27838728</Sha>
</Dependency>
<Dependency Name="System.Runtime.Numerics.TestData" Version="8.0.0-beta.24362.2">
<Uri>https://github.com/dotnet/runtime-assets</Uri>
Expand Down
2 changes: 1 addition & 1 deletion eng/Versions.props
Original file line number Diff line number Diff line change
Expand Up @@ -188,7 +188,7 @@
<MicrosoftDotNetXHarnessTestRunnersCommonVersion>8.0.0-prerelease.24229.2</MicrosoftDotNetXHarnessTestRunnersCommonVersion>
<MicrosoftDotNetXHarnessTestRunnersXunitVersion>8.0.0-prerelease.24229.2</MicrosoftDotNetXHarnessTestRunnersXunitVersion>
<MicrosoftDotNetXHarnessCLIVersion>8.0.0-prerelease.24229.2</MicrosoftDotNetXHarnessCLIVersion>
<MicrosoftDotNetHotReloadUtilsGeneratorBuildToolVersion>8.0.0-alpha.0.24402.1</MicrosoftDotNetHotReloadUtilsGeneratorBuildToolVersion>
<MicrosoftDotNetHotReloadUtilsGeneratorBuildToolVersion>8.0.0-alpha.0.24453.2</MicrosoftDotNetHotReloadUtilsGeneratorBuildToolVersion>
<XUnitVersion>2.4.2</XUnitVersion>
<XUnitAnalyzersVersion>1.0.0</XUnitAnalyzersVersion>
<XUnitRunnerVisualStudioVersion>2.4.5</XUnitRunnerVisualStudioVersion>
Expand Down
58 changes: 54 additions & 4 deletions eng/packaging.targets
Original file line number Diff line number Diff line change
Expand Up @@ -196,6 +196,21 @@
</ItemGroup>
</Target>

<Target Name="WarnOnProjectReferenceToFrameworkAssemblies"
BeforeTargets="IncludeTransitiveProjectReferences"
Condition="'$(TargetFramework)' == '$(NetCoreAppCurrent)' and
'@(ProjectReference)' != ''">
<!-- Find project references that overlap with NetCoreApp, are direct (NuGetPackageId is not set), actually referenced (ReferenceOutputAssembly), and not hidden with PrivateAssets
ProjectReferences can opt out of this checking by setting AllowFrameworkPackageReference, though they should not. -->
<Warning Text="Project reference '%(ProjectReference.Identity)' is a reference to a framework assembly and is not required in $(NetCoreAppCurrent) (NetCoreAppCurrent)."
Code="NETPKG0001"
Condition="$(NetCoreAppLibrary.Contains('%(ProjectReference.Filename);')) and
'%(ProjectReference.ReferenceOutputAssembly)' != 'false' and
'%(ProjectReference.NuGetPackageId)' == '' and
'%(ProjectReference.PrivateAssets)' != 'all' and
'%(ProjectReference.AllowFrameworkPackageReference)' != 'true'" />
</Target>

<Target Name="GenerateMultiTargetRoslynComponentTargetsFile"
Inputs="$(MSBuildProjectFullPath);$(_MultiTargetRoslynComponentTargetsTemplate)"
Outputs="$(MultiTargetRoslynComponentTargetsFileIntermediatePath)">
Expand Down Expand Up @@ -303,12 +318,47 @@
</ItemGroup>
</Target>

<Target Name="ValidateServicingVersionIsProperlySet"
<ItemDefinitionGroup>
<TargetPathWithTargetPlatformMoniker>
<!-- When ServiceTransitiveDependencies is set, flow the packaging state -->
<GeneratePackageOnBuild Condition="'$(ServiceTransitiveDependencies)' == 'true'">$(GeneratePackageOnBuild)</GeneratePackageOnBuild>
</TargetPathWithTargetPlatformMoniker>
</ItemDefinitionGroup>

<!-- Flows the list of ProjectReferences that are enabled for packaging when building a multi-targeting project -->
<Target Name="_AddTransitiveServicedPackagesToOutput"
AfterTargets="GetTargetPathWithTargetPlatformMoniker"
Condition="'$(IsInnerBuild)' == 'true'">
<PropertyGroup>
<_TransitiveServicedPackages
Condition="'%(ReferencePath.ReferenceSourceTarget)' == 'ProjectReference' and
'%(ReferencePath.GeneratePackageOnBuild)' == 'true'"
>@(ReferencePath->'%(OriginalProjectReferenceItemSpec)')</_TransitiveServicedPackages>
</PropertyGroup>
<ItemGroup>
<TargetPathWithTargetPlatformMoniker TransitiveServicedPackages="$(_TransitiveServicedPackages)" />
</ItemGroup>
</Target>

<!-- Validate that ServicingVersion is set and packing is enabled. Runs once in the outer build (or only build if no outer build exists). -->
<Target Name="ValidateServicingProperties"
Condition="'$(PreReleaseVersionLabel)' == 'servicing' and
'$(PackageUseIncrementalServicingVersion)' == 'true' and
'$(DotNetBuildFromSource)' != 'true'"
AfterTargets="GenerateNuspec">
<Error Condition="'$(ServicingVersion)' == '0'" Text="ServicingVersion is set to 0 and it should be an increment of the patch version from the last released package." />
'$(DotNetBuildFromSource)' != 'true' and
'$(IsInnerBuild)' != 'true'"
AfterTargets="Build">
<ItemGroup>
<TransitiveServicedPackages Include="%(InnerOutput.TransitiveServicedPackages)" Condition="'$(IsCrossTargetingBuild)' == 'true'" />
<TransitiveServicedPackages Include="'%(ReferencePath.OriginalProjectReferenceItemSpec)"
Condition="'%(ReferencePath.ReferenceSourceTarget)' == 'ProjectReference' and
'%(ReferencePath.GeneratePackageOnBuild)' == 'true'" />
</ItemGroup>
<Error Condition="'$(ServicingVersion)' == '0' and '$(GeneratePackageOnBuild)' == 'true'"
Text="ServicingVersion is set to 0 and it should be an increment of the patch version from the last released package." />

<Error Condition="'$(GeneratePackageOnBuild)' != 'true' and
'@(TransitiveServicedPackages)' != ''"
Text="This project did not set GeneratePackageOnBuild, but dependencies '@(TransitiveServicedPackages)' did. Please ship this project by setting GeneratePackageOnBuild to true and incrementing ServicingVersion." />
</Target>

</Project>
14 changes: 7 additions & 7 deletions eng/pipelines/common/templates/pipeline-with-resources.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ extends:

containers:
linux_arm:
image: mcr.microsoft.com/dotnet-buildtools/prereqs:cbl-mariner-2.0-cross-arm
image: mcr.microsoft.com/dotnet-buildtools/prereqs:azurelinux-3.0-net8.0-cross-arm
env:
ROOTFS_DIR: /crossrootfs/arm

Expand All @@ -27,23 +27,23 @@ extends:
ROOTFS_DIR: /crossrootfs/armv6

linux_arm64:
image: mcr.microsoft.com/dotnet-buildtools/prereqs:cbl-mariner-2.0-cross-biarch-amd64-arm64
image: mcr.microsoft.com/dotnet-buildtools/prereqs:azurelinux-3.0-net8.0-cross-arm64
env:
ROOTFS_HOST_DIR: /crossrootfs/x64
ROOTFS_DIR: /crossrootfs/arm64

linux_musl_x64:
image: mcr.microsoft.com/dotnet-buildtools/prereqs:cbl-mariner-2.0-cross-amd64-alpine
image: mcr.microsoft.com/dotnet-buildtools/prereqs:azurelinux-3.0-net8.0-cross-amd64-alpine
env:
ROOTFS_DIR: /crossrootfs/x64

linux_musl_arm:
image: mcr.microsoft.com/dotnet-buildtools/prereqs:cbl-mariner-2.0-cross-arm-alpine
image: mcr.microsoft.com/dotnet-buildtools/prereqs:azurelinux-3.0-net8.0-cross-arm-alpine
env:
ROOTFS_DIR: /crossrootfs/arm

linux_musl_arm64:
image: mcr.microsoft.com/dotnet-buildtools/prereqs:cbl-mariner-2.0-cross-arm64-alpine
image: mcr.microsoft.com/dotnet-buildtools/prereqs:azurelinux-3.0-net8.0-cross-arm64-alpine
env:
ROOTFS_DIR: /crossrootfs/arm64

Expand All @@ -56,12 +56,12 @@ extends:
image: mcr.microsoft.com/dotnet-buildtools/prereqs:cbl-mariner-2.0-android-docker

linux_x64:
image: mcr.microsoft.com/dotnet-buildtools/prereqs:cbl-mariner-2.0-cross-amd64
image: mcr.microsoft.com/dotnet-buildtools/prereqs:azurelinux-3.0-net8.0-cross-amd64
env:
ROOTFS_DIR: /crossrootfs/x64

linux_x86:
image: mcr.microsoft.com/dotnet-buildtools/prereqs:cbl-mariner-2.0-cross-x86
image: mcr.microsoft.com/dotnet-buildtools/prereqs:azurelinux-3.0-net8.0-cross-x86
env:
ROOTFS_DIR: /crossrootfs/x86

Expand Down
2 changes: 1 addition & 1 deletion eng/pipelines/coreclr/templates/helix-queues-setup.yml
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ jobs:

# Android x64
- ${{ if in(parameters.platform, 'android_x64') }}:
- Ubuntu.2004.Amd64.Android.29.Open
- Ubuntu.2204.Amd64.Android.29.Open

# Browser wasm
- ${{ if eq(parameters.platform, 'browser_wasm') }}:
Expand Down
3 changes: 2 additions & 1 deletion eng/pipelines/libraries/helix-queues-setup.yml
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@ jobs:
- Ubuntu.2204.Amd64.Open
- (Debian.12.Amd64.Open)Ubuntu.2204.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:debian-12-helix-amd64
- (Mariner.2.0.Amd64.Open)Ubuntu.2204.amd64.open@mcr.microsoft.com/dotnet-buildtools/prereqs:cbl-mariner-2.0-helix-amd64
- (AzureLinux.3.0.Amd64.Open)Ubuntu.2204.Amd64.open@mcr.microsoft.com/dotnet-buildtools/prereqs:azurelinux-3.0-helix-amd64
- (openSUSE.15.2.Amd64.Open)Ubuntu.2204.amd64.open@mcr.microsoft.com/dotnet-buildtools/prereqs:opensuse-15.2-helix-amd64
- ${{ if or(ne(parameters.jobParameters.isExtraPlatforms, true), eq(parameters.jobParameters.includeAllPlatforms, true)) }}:
- (Centos.9.Amd64.Open)Ubuntu.2204.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:centos-stream9-helix
Expand Down Expand Up @@ -99,7 +100,7 @@ jobs:

# Android
- ${{ if in(parameters.platform, 'android_x86', 'android_x64', 'linux_bionic_x64') }}:
- Ubuntu.2004.Amd64.Android.29.Open
- Ubuntu.2204.Amd64.Android.29.Open
- ${{ if in(parameters.platform, 'android_arm', 'android_arm64', 'linux_bionic_arm64') }}:
- Windows.11.Amd64.Android.Open

Expand Down
3 changes: 1 addition & 2 deletions src/coreclr/jit/codegencommon.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -1289,9 +1289,8 @@ bool CodeGen::genCreateAddrMode(
{
cns += addConst->IconValue();
op2 = op2->AsOp()->gtOp1;
goto AGAIN;
}

goto AGAIN;
}
break;

Expand Down
9 changes: 4 additions & 5 deletions src/coreclr/jit/lowerxarch.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -4865,8 +4865,9 @@ GenTree* Lowering::LowerHWIntrinsicDot(GenTreeHWIntrinsic* node)

horizontalAdd = NI_SSE3_HorizontalAdd;

if (!comp->compOpportunisticallyDependsOn(InstructionSet_SSE3))
if ((simdSize == 8) || !comp->compOpportunisticallyDependsOn(InstructionSet_SSE3))
{
// We also do this for simdSize == 8 to ensure we broadcast the result as expected
shuffle = NI_SSE_Shuffle;
}
break;
Expand Down Expand Up @@ -4917,10 +4918,8 @@ GenTree* Lowering::LowerHWIntrinsicDot(GenTreeHWIntrinsic* node)

horizontalAdd = NI_SSE3_HorizontalAdd;

if (!comp->compOpportunisticallyDependsOn(InstructionSet_SSE3))
{
shuffle = NI_SSE2_Shuffle;
}
// We need to ensure we broadcast the result as expected
shuffle = NI_SSE2_Shuffle;
break;
}

Expand Down
10 changes: 0 additions & 10 deletions src/coreclr/jit/morph.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -10724,16 +10724,6 @@ GenTree* Compiler::fgOptimizeHWIntrinsic(GenTreeHWIntrinsic* node)
break;
}

#if defined(TARGET_XARCH)
if ((node->GetSimdSize() == 8) && !compOpportunisticallyDependsOn(InstructionSet_SSE41))
{
// When SSE4.1 isn't supported then Vector2 only needs a single horizontal add
// which means the result isn't broadcast across the entire vector and we can't
// optimize
break;
}
#endif // TARGET_XARCH

GenTree* op1 = node->Op(1);
GenTree* sqrt = nullptr;
GenTree* toScalar = nullptr;
Expand Down
6 changes: 6 additions & 0 deletions src/coreclr/pal/src/thread/context.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -1496,6 +1496,12 @@ CONTEXT_GetThreadContextFromThreadState(

// AMD64's FLOATING_POINT includes the xmm registers.
memcpy(&lpContext->Xmm0, &pState->__fpu_xmm0, 16 * 16);

if (threadStateFlavor == x86_FLOAT_STATE64)
{
// There was just a floating point state, so make sure the CONTEXT_XSTATE is not set
lpContext->ContextFlags &= ~(CONTEXT_XSTATE & CONTEXT_AREA_MASK);
}
}
break;
}
Expand Down
Loading

0 comments on commit a8108c9

Please sign in to comment.