Fill WASM SIMD gaps: Base64 vectorization and byte/ushort LeadingZeroCount - #131239
Conversation
The Vector128 encode loop was gated on Ssse3 or AdvSimd.Arm64 with no cross-platform fallback, so WASM fell back to the scalar EncodeThreeAndWrite path. Widen the gate to PackedSimd and add the corresponding branches: SimdShuffle uses PackedSimd.Swizzle (matching Ssse3.Shuffle's high-bit-clears-lane semantics via mask8F), SubtractSaturate uses PackedSimd.SubtractSaturate, and the Sse2.MultiplyHigh reshuffle is emulated with per-lane logical shifts (multiplying the u16 lanes by 2^6/2^10 is a right shift by 10/6). The remaining ops are already cross-platform Vector128 APIs the JIT lowers on WASM. New branches constant-fold away off-WASM. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The Vector128 decode loop was gated on Ssse3 or AdvSimd.Arm64 with no cross-platform fallback, so WASM fell back to the scalar path. Widen the gate to PackedSimd and emulate the two MultiplyAddAdjacent merges: pmaddubsw by {64,1,...} becomes (even_byte << 6) + odd_byte and pmaddwd by {4096,1,...} becomes (even_i16 << 12) + odd_i16, both exact for the 6-bit/12-bit intermediate values. SimdShuffle already handles PackedSimd, and TryLoadVector128/TryDecode128Core use cross-platform Vector128 APIs. Emulations validated bit-identical against the Sse2/Ssse3 intrinsics over random inputs; branches constant-fold away off-WASM.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Smear the most-significant set bit down and compute bitWidth - PopCount, reusing PopCountOperator. Only byte/ushort are vectorized this way; the serial smear chain over the 2-4 lanes of uint/ulong doesn't beat a single-instruction scalar clz (x86 lzcnt, wasm i32/i64.clz), so those keep their existing hardware-vector-clz gates. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
Azure Pipelines: Successfully started running 3 pipeline(s). 13 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
|
Tagging subscribers to this area: @dotnet/area-system-numerics |
|
CC. @dotnet/wasm-contrib. This should be the only vector paths that didn't have WASM lightup. The search was oriented around RyuJIT WASM, but the necessary intrinsics should all be accelerated for Mono WASM as well (CC. @lewing) |
There was a problem hiding this comment.
Pull request overview
This PR expands WebAssembly SIMD (PackedSimd) coverage for existing vectorized library implementations by adding PackedSimd-gated branches where code previously only accelerated on x86/Arm-specific intrinsics, and by introducing a software-vector fallback for byte/ushort LeadingZeroCount in TensorPrimitives.
Changes:
- Enable the
Vector128Base64 encode/decode fast paths whenPackedSimd.IsSupported, and provide PackedSimd equivalents / emulations for the small set of ISA-specific primitives used in those loops. - Add a table-free software-vector fallback for
TensorPrimitives.LeadingZeroCount<byte/ushort>using bit-smear + vectorPopCount, while keeping hardware vector-CLZ paths for wider element sizes.
Show a summary per file
| File | Description |
|---|---|
| src/libraries/System.Private.CoreLib/src/System/Buffers/Text/Base64Helper/Base64EncoderHelper.cs | Adds PackedSimd to the Vector128 encode gate and implements PackedSimd-specific equivalents for operations previously only available via x86/Arm intrinsics. |
| src/libraries/System.Private.CoreLib/src/System/Buffers/Text/Base64Helper/Base64DecoderHelper.cs | Adds PackedSimd to the Vector128 decode gate and implements a PackedSimd Swizzle-based shuffle to match SSSE3/AdvSimd semantics. |
| src/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/netcore/TensorPrimitives.LeadingZeroCount.cs | Adds a software-vector fallback for byte/ushort leading-zero count using smear + PopCount, preserving hardware-vector-CLZ-only behavior for uint/ulong. |
Copilot's findings
- Files reviewed: 3/3 changed files
- Comments generated: 0
|
There's notably some opportunities to reuse existing xplat vector APIs, but I'm not sure Mono handles them all today and so I went with the platform specific paths instead. We can clean up in .NET 12 |
The main arch we care about for mono on 11 is wasm and I'm willing to fix any issues there if you let me know what apis you're looking at. |
I'll take a deeper look and get back to you. It's mostly just about simplification of the managed code at this point, not accelerating it more, so I viewed it as a lower priority than this PR (which explicitly accelerates). |
This is the first, low-risk slice of an audit of the vectorized libraries code (
Vector<T>,Vector128/256/512<T>,System.Runtime.Intrinsics.*) for WebAssembly SIMD (PackedSimd) coverage. It fixes the genuine WASM gaps found so far and leaves the broader audit tracked for follow-ups.The audit is grounded on the RyuJIT WASM backend, not Mono+LLVM. RyuJIT auto-lowers cross-platform
Vector128/Vector<T>intoPackedSimdduring import/lowering, so xplat code is already WASM-accelerated. A genuine WASM gap therefore exists only where code uses rawSse*/Avx*/AdvSimdintrinsics with no xplat fallback (so WASM drops to scalar), and the operation has a WASM equivalent.Base64 encode/decode (
Base64EncoderHelper.cs,Base64DecoderHelper.cs)The
Vector128encode/decode loops were gated onSsse3 || AdvSimd.Arm64with no cross-platform fallback, so WASM fell all the way back to the scalar path. These are the only true "vector cliffs" the audit found. Both helpers are shared byBase64andBase64Url, so this covers all four public surfaces.The gate is widened to include
PackedSimd, and the handful of x86/Arm-specific primitives are givenPackedSimdbranches:SimdShuffle→PackedSimd.Swizzle(matchesSsse3.Shuffle's high-bit-clears-lane semantics viamask8F).SubtractSaturate→PackedSimd.SubtractSaturate.pmaddubsw/pmaddwd/MultiplyHighreshuffles are emulated with per-lane logical shifts, exact for the 6-/10-/12-bit intermediates (e.g.pmaddubswby{64,1,…}is(even_byte << 6) + odd_byte).The emulations were validated bit-identical against the
Sse2/Ssse3intrinsics over random inputs. All new branches constant-fold away off-WASM (PackedSimd.IsSupportedis a JIT constant), so there's no change to x86/Arm codegen.byte/ushort
LeadingZeroCountsoftware fallback (TensorPrimitives.LeadingZeroCount.cs)LeadingZeroCountpreviously vectorized only via a hardware vector clz —Avx512CD/Avx512Vbmion x86,AdvSimdon Arm. WASM has no vector clz opcode, so it ran fully scalar; x86-without-AVX512 also ran scalar forbyte/ushort.This adds a table-free software fallback for
byte/ushortonly: smear the most-significant set bit down so every lower bit is set, thenLeadingZeroCount == bitWidth - PopCount(smeared), reusing the existingPopCountOperator(the same idiomTrailingZeroCountalready uses). Hardware paths keep priority; the fallback only fills the gap on platforms lacking a vector clz for these widths (WASM, x86-SSE).Why byte/ushort only
The smear is a serial dependency chain of
O(log bitWidth)instructions independent of lane count, so it only pays off with enough lanes to amortize the chain. Measured on an AVX2 box withAvx512CD.VLdisabled to force the software path, comparing the actualTensorPrimitives.LeadingZeroCount<T>path before/after (best-of-N, warmed):byteushortuintulongSo
uint/ulongkeep their existing hardware-vector-clz gates and stay scalar where no vector clz exists — a single-instruction scalar clz (x86 lzcnt,wasm i32/i64.clz) beats a serial smear over only 2–4 lanes.I also checked whether widening to
Vector256would flipuint/ulongto a win (more lanes amortizing the same chain, plus wider loads for memory-bound inputs). It doesn't hold up —uintonly wins at the L1-resident (1.21x) and truly DRAM-bound (1.18x) extremes but loses ~2x across the L2/L3 middle where most inputs land;ulongnever breaks even (0.54–0.75x). The scalar loop sustains higher memory-level parallelism than the long dependency chain, so the wide-vector version is left out.Audit coverage (for reviewer context)
~68 vectorized files were reviewed. Breakdown:
PackedSimdbranch alongside x86/Arm (e.g.HexConverter,Guid,Matrix4x4, the ASCII/UTF8/Teddy/ProbabilisticSearchValues).Vector128/Vector<T>fallback is what RyuJIT lowers on WASM, so no cliff (e.g.SpanHelpers.Byte/Char,Ascii.*,Math/MathF,TrailingZeroCount, Latin1 narrowing).SearchValuesfamily (SpanHelpers.Packed.cs+Any*CharPacked*). Its throughput win rests on a cheap movemask + saturating narrow. Those are cheap when a WASM engine runs on an x86 host but lower to the same expensive emulation on an Arm host — which is exactly why native Arm (AdvSimd) is deliberately excluded there. Since we can't condition on the host ISA from within WASM, enabling it risks the Arm worst-case regression, soPackedIndexOfIsSupportedstays x86-only. WASM already uses the non-packed (still vector-accelerated) searcher.Note
This PR description and the code changes were authored with GitHub Copilot.