Improve Arm64 for .NET11

[a74nh]

We will continue to improve the Arm64 port for .NET11. This will include improving code quality, enhancing SVE support and other general performance improvements.

Improvements to SVE APIs

• SVE2 APIs - Finishing the remaining SVE2 APIs that were started in .NET10 and fix up any inconsistencies
  • Sve2 scatterstores (7 APIs) Implement SVE2 non-temporal scatter stores #123892
  • Sve2 gatherloads (11 APIs) Implement SVE2 non-temporal gather loads #123890
  • Sve2 counting (2 APIs) Implement SVE2 Counting APIs #122602
  • Sve2 mask (10 APIs) SVE: Match, NoMatch #125163 Arm64 SVE2: CreateWhileGreaterThan* and CreateWhileReadAfterWrite* #127538 [SVE2] CreateWhileWriteAfterRead* and SaturatingExtract* #128233
• Fix SVE1/SVE2 API inconsistencies
  • [API Proposal]: ARM64 SVE: Add additional types for the mask APIs #108233
    • Sve: Expand CreateWhile APIs #124081
    • SVE: Add float/double args for mask APIs #123947
  • [API Proposal]: SVE2 API for UQSHRNB and UQSHRNT #122022 Sve2: Add ShiftRightLogicalNarrowingSaturate(Even|Odd) #123888
  • Remove duplicated SVE GatherVector APIs #124033
  • SVE2: Fixup SVE2 Math/bitmanipulate/FP APIs #121961
    • Rename SVE2 AddSaturate signed/unsigned addend API #122283
    • Fix SVE2 API for Odd/Even methods #122173
    • Fix SVE2 API for RotateComplex methods #122172
• Testing cleanups. This will simplify testing, expand the testing scope and possibly find bugs
  • Improve SVE binary test template #118730
  • Make the HardwareIntrinsic tests rebuild when templates are updated #118486
  • arm64: Refactor GenerateHWIntrinsicTests_Arm.cs #121584
  • arm64: Small fixes to GenerateHWIntrinsicTests_Arm.cs #125954
• Add benchmarks to dotnet/performance
  • Add OddEvenSort to SVE microbenchmark performance#5046
  • Add ComplexDotProduct to SVE microbenchmark performance#5045
  • Add floating-point routines to SVE microbenchmark performance#5043
  • Add VectorMax and Clamp to SVE microbenchmark performance#5042
  • Add ScatterStore and GatherLoad to SVE microbenchmark performance#5041
  • Add SobelFilter to SVE microbenchmark performance#5040
  • Add TCPChecksum to SVE microbenchmark performance#5039
  • Add UpscaleFilter to SVE microbenchmark performance#5038
  • Add PairwiseAdd and ComplexMultiply to SVE microbenchmark performance#5037
  • Add SquareRoot and Logarithm to SVE microbenchmark performance#5021
  • Add MultiplyAdd and AddReduction to SVE microbenchmark performance#5020
  • Fix Sve benchmark warning and config performance#5012
  • Improve Performance of SVE microbenchmarks performance#4973
  • Add Partition to SVE micro benchmark performance#4943

More SVE/AdvSimd Extensions

• SVE Extensions: AES, SHA3, SM4
  • Arm64: Add SM4 and SHA3 instructions #124504
  • Add instruction set detection for SVE_AES, SVE_SHA3, SVE_SM4, SHA3, SM4 #124637
  • Add SVE AES
  • Add SVE SHA3: Add SHA3 and SVE SHA3 APIs #126941
  • Add SVE SM4: Add Sm4 and SVE Sm4 APIs #130039
• AdvSimd versions of the above SVE Extensions: SHA3, SM4
  • Add SHA3: Add SHA3 and SVE SHA3 APIs #126941
  • Add SM4: Add Sm4 and SVE Sm4 APIs #130039

Agnostic SVE

• As much of the agnostic SVE work as we can complete from Accelerating Vector<T> with SVE on ARM64 #120599
  • Add scalable vector type to JIT and HFA type for Vector<T> #121114 (initial research)
  • Map SIMD types directly from class names #121489
  • Introduce TYP_SIMD to the JIT for Vector<T> #121548
  • Introduce local variables with unknown size #122638
  • Implement UnknownSizeFrame for locals with unknown size #125491
  • Support for variable sized locals in morph and lclmorph #124516
  • Support spill temps with unknown size on ARM64 #127917
  • Arm64: Initialization of TYP_SIMD/TYP_MASK locals #128148
  • Pass Vector<T> by reference when InstructionSet_VectorT is available #125729
  • Add InstructionSet_VectorT for ARM64 #123992
  • Support InstructionSet_VectorT in HWIntrinsic importer #125226
  • Arm64: Improve support for HW_Flag_ReturnsPerElementMask #128326
  • Arm64 SVE: Support scalable constant vectors and masks #127520

SVE Performance

• General performance improvements
  • Arm64 SVE: Allow SVE nodes to be treated as generic ops #119352
• Fix movpfrx usage
  • arm64: Refactor mov/movprfx for unmasked operations #123717
  • arm64: Refactor mov/movprfx for embedded masked operations #126398
  • arm64: Clean up SVE embedded masked codegen #127164

Codegen Improvements

• Continue to add more compact encodings
  • arm64: Optimise GT/GE/LT/LE comparisons to constant zero #119758
  • arm64: Optimise x < 0 and x >= 0 #120667
  • arm64: Fixed using ands/bics to compare against static unsigned constant 0 #124601
  • arm64: Replace RSH/RSZ -> CAST nodes with clearing register #121007
  • arm64: Remove widening casts before truncating #123546
  • arm64: Fix and/bic issue when bailing early #125428
  • arm64: Fold negative variable eq/ne to 0 #124332
  • arm64: Add support for BFI and BFX instruction #123138

Improvements in GC

• Add an Arm64 version of the vxsort algorithm in the GC.
  • vxsort: Add Arm64 Neon implementation and tests #110692

PAC-RET

• Enable PAC-RET protection for the stack
  • Arm64:[PAC-RET] Instruction encodings #127949
  • Arm64: [PAC-RET] Jit Support #127838
  • Arm64: [PAC-RET] VM and coreclr support #128147
  • Arm64:[PAC-RET] Use hint-space instruction for stripping return address #129130
  • Arm64: [PAC-RET] NativeAOT changes #128950

[a74nh]

@dotnet/arm64-contrib @dotnet/jit-contrib @JulieLeeMSFT

[tannergooding]

Add benchmartks to dotnet/performance

I left a comment related to this here: dotnet/performance#5046, I will repost for simplicity

I have a general concern over all the SVE specific microbenchmarks being added.

Benchmarks are fairly expensive in terms of runtime and even a small number of them can have significant cost to CI and our tracking. We don't really have any platform specific intrinsic benchmarks correspondingly (i.e. you don't see explicit benchmarks covering AdvSimd or Avx512).

Rather instead we typically have our normal benchmarks like for Span, Linq, Tensors and other areas which internally accelerate using SIMD and we measure those across a range of hardware. We have some machines that support newer ISAs and can then correlate and see the more real world improvements based on that.

I would expect here that we aren't directly testing SVE either. But rather would be testing with SVE enabled and comparing that against a run with it disabled. This will require a bit more work in the JIT to enable first, but significantly reduces cost and gives better metrics as to the benefit customers will see.

CC. @DrewScoggins

[a74nh]

Add benchmartks to dotnet/performance

I left a comment related to this here: dotnet/performance#5046, I will repost for simplicity

I have a general concern over all the SVE specific microbenchmarks being added.
Benchmarks are fairly expensive in terms of runtime and even a small number of them can have significant cost to CI and our tracking. We don't really have any platform specific intrinsic benchmarks correspondingly (i.e. you don't see explicit benchmarks covering AdvSimd or Avx512).
Rather instead we typically have our normal benchmarks like for Span, Linq, Tensors and other areas which internally accelerate using SIMD and we measure those across a range of hardware. We have some machines that support newer ISAs and can then correlate and see the more real world improvements based on that.
I would expect here that we aren't directly testing SVE either. But rather would be testing with SVE enabled and comparing that against a run with it disabled. This will require a bit more work in the JIT to enable first, but significantly reduces cost and gives better metrics as to the benefit customers will see.
CC. @DrewScoggins

Let's take a step back then. What are my motivations for the benchmarks?

• We know that codegen for SVE is harder to get right compared to AdvSimd. Mostly due to mask handling, but probably other reasons too.
• Right now we don't know how much impact the non-optimal codegen is having on performance
• Even if codegen is fixed, it's fairly easy for the user to accidentally write non-optimal SVE code.
• Having some well coded C# routines for SVE, alongside their scalar and AdvSimd equivalents will allow us to keep track of the performance limitations.
• In addition, they will serve as examples for users writing their own code - either directly or via AI scraping.
• We don't require these benchmarks to be run constantly. Once a week, or even less, feels reasonable given we don't expect figures to change unless explicit intrinsic changes are made.
• We do want something a developer can easily run to check their own changes. We need to know if the changes we are doing are worthwhile.
• It doesn't have to be in dotnet/performance, but that felt like the most public place
• Do we have different tiers for the performance tests?

[tannergooding]

Writing intrinsics is an advanced scenario that is primarily getting done by low level framework authors, it isn't something we want apps and other common library code to deal with or think about. Rather we want most apps/libraries to just use things like Span<T>, LINQ, TensorPrimitives, and other key APIs we provide that are already accelerated for them, allowing them to work at a higher level.

For the low level framework authors, we then don't want them to end up writing platform specific code paths for most scenarios. We rather want and encourage them to use the xplat API surface because it trivializes the support considerations and gives the JIT the flexibility to optimized based on the underlying hardware.

For example, consider that we have the following potential SIMD platforms and ISA targeting considerations:

• Arm64
  • AdvSimd
  • Sve
• x86/x64
  • Sse42
  • Avx2
  • Avx512
  • Avx10v2
• WASM
  • PackedSimd
• RISC-V
• Loongarch
• etc

Writing a platform-specific path for each platform is untenable and largely unnecessary for most domains; so, we would much prefer they write "one path" using Vector<T> which then allows it to implicitly work and be optimal across all of these. There will of course be exceptions, particularly where there is some opportunistic lightup and the platform-specific path can be mixed into the xplat logic to enable a key significant optimization. But that should still end up being with most logic using the xplat APIs (i.e. vector + vector, not Sve.Add(vector, vector)) and will effectively look like:

// xplat logic

if (Sve.IsSupported)
{
    // sve specific logic
}
else
{
    // xplat fallback
}

// xplat logic

We ultimately want to provide an API surface that is easy, natural, and largely intuitive for users to get correct. This is critical to their success and the success of the APIs we expose as well. So, the lower level the API, the more "responsibility" is on the user to write correct code and understand the potential implications of what they write.

---

Given all of that, I feel many of the points raised on why SVE perf tests are being added end up being related to internal details to the runtime, which are not relevant to the user. The things that are important to measure, particularly as they apply to the user, are the higher level APIs which should have Vector<T> paths or have those paths added, and which should be measured with SVE enabled vs disabled so that we can get more accurate "real world" impact measurements of the SVE changes. Those same users wanting to test their code are ideally just running their existing end to end apps, getting their existing profiling data, etc

For AI, this feels rather something we should create a base prompt or explicit documentation for instead. We shouldn't be relying on AI taking the "average scrapings of the internet" for the user-experience. I expect doing that will end up suboptimal, particularly given this is an advanced and niche scenario that is hand rolling your own vectorization logic, so the ratio of "correct information" vs "subtly incorrect or partial information" is likely not high enough to get reliably accurate results. Providing explicit context/prompts for developers working in this space will help ensure they get the best suggestions without relying solely on the "average scrapings"

Do we have different tiers for the performance tests?

I don't believe so, but @DrewScoggins would likely be the one to answer