Skip to content

Commit 4e28c96

Browse files
lewingCopilot
lewing
and
Copilot
authored
[wasm] Enable Vector128 fast paths on Wasm via PackedSimd: hex/Guid, UTF-8, SearchValues, Teddy, Adler32/XXH3 (#129838)
Enable several `Vector128` fast paths in CoreLib and `System.IO.Hashing` on browser-wasm by adding a `PackedSimd.IsSupported` branch alongside the existing `Sse2`/`Ssse3`/`AdvSimd.Arm64` gates. Before these changes, the SIMD code paths were unreachable on Wasm even though the wasm runtime supports PackedSimd (the test pipeline default sets `WasmEnableSIMD=true`). ## Scope and commits | # | Commit | Area | |---|--------|------| | 1 | `c69998277eb` `0e99f4a194c` | Add Wasm `PackedSimd` path to `Vector128.UnpackLow/UnpackHigh` (composed from two `PackedSimd.Swizzle` calls + OR; the two-vector `PackedSimd.Shuffle` requires constant lane indices and is impractical from generic code). | | 2 | `3dc956b6465` | Enable Vector128 hex/Guid format fast path on Wasm — widen the gates on `HexConverter.AsciiToHexVector128`, `HexConverter.EncodeToUtf8/Utf16`, `HexConverter.EncodeTo_Vector128`, and `Guid.FormatGuidVector128Utf8`. Bodies were already portable (`Vector128.ShuffleNative`, `Vector128.UnpackLow/High`, constant-index `Vector128.Shuffle`). | | 3 | `061b22bffcf` `2543702633` | Vectorize `Utf8Utility.Validation` ASCII fast path on Wasm — add a `PackedSimd` branch alongside `AdvSimd.Arm64`/`Sse2` in the inner ASCII-scan loop of `GetPointerToFirstInvalidChar`, using portable `Vector128.LoadUnsafe` + `ExtractMostSignificantBits`. | | 4 | `da08d905c6e` | Vectorize `Utf8Utility.Transcoding` ASCII fast path on Wasm — the 8-char narrow-store loop and the two 4-char tail stores, using `PackedSimd.ConvertNarrowingSaturateUnsigned`. | | 5 | `2caa5e25997` `0e99f4a194c` | Enable Teddy multi-string search on Wasm — `TeddyHelper.LoadAndPack16AsciiChars`, `GetNibbles`, two-table `Shuffle`, `RightShift1/RightShift2` (composed from two `PackedSimd.Swizzle` calls + OR), plus the `StringSearchValues.CreateFromNormalizedValues` entry gate and `AsciiStringSearchValuesTeddyBase.IndexOfAnyN2/N3` `[CompExactlyDependsOn]`. | | 6 | `819c9fe3960` | Enable `ProbabilisticMap` vectorized `SearchValues<char>` on Wasm. Subtle because the bitmap **layout itself** branches on the gate (`SetCharBit`/`IsCharBitSet`, marked `[BypassReadyToRun]`). Widens the layout choice, `ContainsMask16Chars` (new PackedSimd narrowing branch), the `IndexOfAny`/`LastIndexOfAny` entry dispatcher, and the worker `[CompExactlyDependsOn]` attributes consistently. | | 7 | `80522ca629b` | `Adler32.UpdateVector128` and `XxHashShared.MultiplyWideningLower`: replace the portable `Vector128.Widen + multiply + add` sequences with `PackedSimd.AddPairwiseWidening` + `MultiplyWideningLower/Upper` and `PackedSimd.MultiplyWideningLower` (`i64x2.extmul_low_i32x4_u`). Already vectorized on Wasm via the generic else branch; this turns 3–5 ops/iter into 1. | ## Test results Validated end-to-end with `WasmEnableSIMD=true` (the browser-wasm test pipeline default). | Suite | Host arm64 | browser-wasm (V8 v15) | Coverage | |------|-----------|------------------------|----------| | `System.Runtime.Tests` | 69,714 / 69,714 ✅ | 67,835 / 67,835 ✅ | Guid format/parse | | `System.Runtime.Extensions.Tests` | 8,350 / 8,350 ✅ | 8,224 / 8,224 ✅ | `Convert.ToHexString`/`FromHexString` | | `System.Memory.Tests` | 52,906 / 52,906 ✅ | 52,249 / 52,249 ✅ | UTF-8 validation/transcoding, Ascii, `SearchValues<string>` (Teddy), `SearchValues<char>` (ProbabilisticMap), `SpanHelpers` | | `System.IO.Hashing.Tests` | 4,196 / 4,196 ✅ | 4,196 / 4,196 ✅ | Adler32, XxHash (lane-order is checked end-to-end via the hash output bytes — a swap would corrupt every hash) | The original push exposed two AOT-only failure modes in browser-wasm Helix legs: 1. `PackedSimd.Shuffle` (two-vector `i8x16.shuffle`) requires compile-time-constant lane indices; Mono AOT can't fold a `Vector128.Create(...)` operand and throws `PlatformNotSupportedException`. Fixed in `c69998277eb` → `0e99f4a194c` by composing with single-vector `PackedSimd.Swizzle` + OR. 2. The same pattern was originally applied in `TeddyHelper.RightShift1/RightShift2` via `Vector128.ShuffleNative` (a dispatcher with an `Ssse3 → AdvSimd.Arm64 → PackedSimd` if/else chain). The Mono SIMD intrinsic recognizer doesn't always lower that chain cleanly for less-traveled paths — the safer pattern is to call `PackedSimd.Swizzle` directly under the `PackedSimd.IsSupported` branch. Applied consistently in `0e99f4a194c`. ## Files changed - `src/libraries/System.Private.CoreLib/src/System/Runtime/Intrinsics/Vector128.cs` - `src/libraries/Common/src/System/HexConverter.cs` - `src/libraries/System.Private.CoreLib/src/System/Guid.cs` - `src/libraries/System.Private.CoreLib/src/System/Text/Unicode/Utf8Utility.Validation.cs` - `src/libraries/System.Private.CoreLib/src/System/Text/Unicode/Utf8Utility.Transcoding.cs` - `src/libraries/System.Private.CoreLib/src/System/SearchValues/Strings/Helpers/TeddyHelper.cs` - `src/libraries/System.Private.CoreLib/src/System/SearchValues/Strings/StringSearchValues.cs` - `src/libraries/System.Private.CoreLib/src/System/SearchValues/Strings/AsciiStringSearchValuesTeddyBase.cs` - `src/libraries/System.Private.CoreLib/src/System/SearchValues/ProbabilisticMap.cs` - `src/libraries/System.IO.Hashing/src/System/IO/Hashing/Adler32.cs` - `src/libraries/System.IO.Hashing/src/System/IO/Hashing/XxHashShared.cs` ## Not in this PR - `Base64DecoderHelper` / `Base64EncoderHelper` — need `pmaddubsw` and `pmulhuw` analogs composed from `MultiplyWideningLower/Upper` + `AddPairwiseWidening` or `Dot`, with careful 8-short lane preservation. Worth a dedicated follow-up with benchmarks. > [!NOTE] > This PR description and the commits in this branch were drafted with AI/Copilot assistance. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent 6fe369a commit 4e28c96

11 files changed

Lines changed: 173 additions & 30 deletions

File tree

‎src/libraries/Common/src/System/HexConverter.cs‎

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -96,9 +96,10 @@ public static void ToCharsBuffer(byte value, Span<char> buffer, int startingInde
9696
[MethodImpl(MethodImplOptions.AggressiveInlining)]
9797
[CompExactlyDependsOn(typeof(Ssse3))]
9898
[CompExactlyDependsOn(typeof(AdvSimd.Arm64))]
99+
[CompExactlyDependsOn(typeof(PackedSimd))]
99100
internal static (Vector128<byte>, Vector128<byte>) AsciiToHexVector128(Vector128<byte> src, Vector128<byte> hexMap)
100101
{
101-
Debug.Assert(Ssse3.IsSupported || AdvSimd.Arm64.IsSupported);
102+
Debug.Assert(Ssse3.IsSupported || AdvSimd.Arm64.IsSupported || PackedSimd.IsSupported);
102103

103104
// The algorithm is simple: a single srcVec (contains the whole 16b Guid) is converted
104105
// into nibbles and then, via hexMap, converted into a HEX representation via
@@ -115,6 +116,7 @@ internal static (Vector128<byte>, Vector128<byte>) AsciiToHexVector128(Vector128
115116

116117
[CompExactlyDependsOn(typeof(Ssse3))]
117118
[CompExactlyDependsOn(typeof(AdvSimd.Arm64))]
119+
[CompExactlyDependsOn(typeof(PackedSimd))]
118120
private static void EncodeTo_Vector128<TChar>(ReadOnlySpan<byte> source, Span<TChar> destination, Casing casing)
119121
{
120122
Debug.Assert(source.Length >= (Vector128<TChar>.Count / 2));
@@ -187,7 +189,7 @@ public static void EncodeToUtf8(ReadOnlySpan<byte> source, Span<byte> utf8Destin
187189
Debug.Assert(utf8Destination.Length >= (source.Length * 2));
188190

189191
#if SYSTEM_PRIVATE_CORELIB
190-
if ((AdvSimd.Arm64.IsSupported || Ssse3.IsSupported) && (source.Length >= (Vector128<byte>.Count / 2)))
192+
if ((AdvSimd.Arm64.IsSupported || Ssse3.IsSupported || PackedSimd.IsSupported) && (source.Length >= (Vector128<byte>.Count / 2)))
191193
{
192194
EncodeTo_Vector128(source, utf8Destination, casing);
193195
return;
@@ -204,7 +206,7 @@ public static void EncodeToUtf16(ReadOnlySpan<byte> source, Span<char> destinati
204206
Debug.Assert(destination.Length >= (source.Length * 2));
205207

206208
#if SYSTEM_PRIVATE_CORELIB
207-
if ((AdvSimd.Arm64.IsSupported || Ssse3.IsSupported) && (source.Length >= (Vector128<ushort>.Count / 2)))
209+
if ((AdvSimd.Arm64.IsSupported || Ssse3.IsSupported || PackedSimd.IsSupported) && (source.Length >= (Vector128<ushort>.Count / 2)))
208210
{
209211
EncodeTo_Vector128(source, Unsafe.BitCast<Span<char>, Span<ushort>>(destination), casing);
210212
return;

‎src/libraries/System.IO.Hashing/src/System/IO/Hashing/Adler32.cs‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@
88
#if NET
99
using System.Runtime.Intrinsics;
1010
using System.Runtime.Intrinsics.Arm;
11+
using System.Runtime.Intrinsics.Wasm;
1112
using System.Runtime.Intrinsics.X86;
1213
#endif
1314

@@ -300,6 +301,23 @@ private static uint UpdateVector128(uint adler, ReadOnlySpan<byte> source)
300301
wprod2 = AdvSimd.MultiplyWideningUpperAndAdd(wprod2, bytes2, tap2.AsByte());
301302
vs2 = AdvSimd.AddPairwiseWideningAndAdd(vs2, wprod2);
302303
}
304+
else if (PackedSimd.IsSupported)
305+
{
306+
// Widening byte sum: each byte -> ushort pair sum -> uint pair sum, then accumulate into vs1.
307+
// Because weights are all positive (1-32), unsigned byte * unsigned byte multiply is valid for vs2.
308+
Vector128<ushort> sumPairs1 = PackedSimd.AddPairwiseWidening(bytes1);
309+
Vector128<ushort> sumPairs2 = PackedSimd.AddPairwiseWidening(bytes2);
310+
vs1 += PackedSimd.AddPairwiseWidening(sumPairs1) + PackedSimd.AddPairwiseWidening(sumPairs2);
311+
312+
// bytes * weights -> 8 ushorts low + 8 ushorts high, sum pairwise to 4 uints + 4 uints.
313+
Vector128<ushort> wprod1Lo = PackedSimd.MultiplyWideningLower(bytes1, tap1.AsByte());
314+
Vector128<ushort> wprod1Hi = PackedSimd.MultiplyWideningUpper(bytes1, tap1.AsByte());
315+
vs2 += PackedSimd.AddPairwiseWidening(wprod1Lo) + PackedSimd.AddPairwiseWidening(wprod1Hi);
316+
317+
Vector128<ushort> wprod2Lo = PackedSimd.MultiplyWideningLower(bytes2, tap2.AsByte());
318+
Vector128<ushort> wprod2Hi = PackedSimd.MultiplyWideningUpper(bytes2, tap2.AsByte());
319+
vs2 += PackedSimd.AddPairwiseWidening(wprod2Lo) + PackedSimd.AddPairwiseWidening(wprod2Hi);
320+
}
303321
else
304322
{
305323
(Vector128<ushort> lo1, Vector128<ushort> hi1) = Vector128.Widen(bytes1);

‎src/libraries/System.IO.Hashing/src/System/IO/Hashing/XxHashShared.cs‎

Lines changed: 17 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@
99
#if NET
1010
using System.Runtime.Intrinsics;
1111
using System.Runtime.Intrinsics.Arm;
12+
using System.Runtime.Intrinsics.Wasm;
1213
using System.Runtime.Intrinsics.X86;
1314
#endif
1415

@@ -702,12 +703,25 @@ private static Vector128<ulong> MultiplyWideningLower(Vector128<uint> source)
702703
Vector64<uint> sourceHigh = Vector128.Shuffle(source, Vector128.Create(1u, 3, 0, 0)).GetLower();
703704
return AdvSimd.MultiplyWideningLower(sourceLow, sourceHigh);
704705
}
706+
else if (Sse2.IsSupported)
707+
{
708+
Vector128<uint> sourceLow = Vector128.Shuffle(source, Vector128.Create(1u, 0, 3, 0));
709+
return Sse2.Multiply(source, sourceLow);
710+
}
711+
else if (PackedSimd.IsSupported)
712+
{
713+
// PackedSimd.MultiplyWideningLower (i64x2.extmul_low_i32x4_u) does
714+
// result[i] = (ulong)a[i] * (ulong)b[i] for i in {0, 1}.
715+
// We need { source[0]*source[1], source[2]*source[3] } to match the Sse2/AdvSimd paths,
716+
// so first move the even lanes into one operand and the odd lanes into the other.
717+
Vector128<uint> evens = Vector128.Shuffle(source, Vector128.Create(0u, 2, 0, 0));
718+
Vector128<uint> odds = Vector128.Shuffle(source, Vector128.Create(1u, 3, 0, 0));
719+
return PackedSimd.MultiplyWideningLower(evens, odds);
720+
}
705721
else
706722
{
707723
Vector128<uint> sourceLow = Vector128.Shuffle(source, Vector128.Create(1u, 0, 3, 0));
708-
return Sse2.IsSupported ?
709-
Sse2.Multiply(source, sourceLow) :
710-
(source & Vector128.Create(~0u, 0u, ~0u, 0u)).AsUInt64() * (sourceLow & Vector128.Create(~0u, 0u, ~0u, 0u)).AsUInt64();
724+
return (source & Vector128.Create(~0u, 0u, ~0u, 0u)).AsUInt64() * (sourceLow & Vector128.Create(~0u, 0u, ~0u, 0u)).AsUInt64();
711725
}
712726
}
713727
#endif

‎src/libraries/System.Private.CoreLib/src/System/Guid.cs‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@
1010
using System.Runtime.InteropServices;
1111
using System.Runtime.Intrinsics;
1212
using System.Runtime.Intrinsics.Arm;
13+
using System.Runtime.Intrinsics.Wasm;
1314
using System.Runtime.Intrinsics.X86;
1415
using System.Runtime.Versioning;
1516
using System.Text;
@@ -1345,7 +1346,7 @@ internal unsafe bool TryFormatCore<TChar>(Span<TChar> destination, out int chars
13451346
}
13461347
flags >>= 8;
13471348

1348-
if ((Ssse3.IsSupported || AdvSimd.Arm64.IsSupported) && BitConverter.IsLittleEndian)
1349+
if ((Ssse3.IsSupported || AdvSimd.Arm64.IsSupported || PackedSimd.IsSupported) && BitConverter.IsLittleEndian)
13491350
{
13501351
// Vectorized implementation for D, N, P and B formats:
13511352
// [{|(]dddddddd[-]dddd[-]dddd[-]dddd[-]dddddddddddd[}|)]
@@ -1513,9 +1514,10 @@ static void WriteHex(Span<TChar> dest, int offset, int val, bool appendComma = t
15131514
[MethodImpl(MethodImplOptions.AggressiveInlining)]
15141515
[CompExactlyDependsOn(typeof(Ssse3))]
15151516
[CompExactlyDependsOn(typeof(AdvSimd.Arm64))]
1517+
[CompExactlyDependsOn(typeof(PackedSimd))]
15161518
private static (Vector128<byte>, Vector128<byte>, Vector128<byte>) FormatGuidVector128Utf8(Guid value, bool useDashes)
15171519
{
1518-
Debug.Assert((Ssse3.IsSupported || AdvSimd.Arm64.IsSupported) && BitConverter.IsLittleEndian);
1520+
Debug.Assert((Ssse3.IsSupported || AdvSimd.Arm64.IsSupported || PackedSimd.IsSupported) && BitConverter.IsLittleEndian);
15191521
// Vectorized implementation for D, N, P and B formats:
15201522
// [{|(]dddddddd[-]dddd[-]dddd[-]dddd[-]dddddddddddd[}|)]
15211523

‎src/libraries/System.Private.CoreLib/src/System/Runtime/Intrinsics/Vector128.cs‎

Lines changed: 32 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -4388,33 +4388,59 @@ internal static void SetElementUnsafe<T>(in this Vector128<T> vector, int index,
43884388
[MethodImpl(MethodImplOptions.AggressiveInlining)]
43894389
[CompExactlyDependsOn(typeof(AdvSimd.Arm64))]
43904390
[CompExactlyDependsOn(typeof(Sse2))]
4391+
[CompExactlyDependsOn(typeof(PackedSimd))]
43914392
internal static Vector128<byte> UnpackLow(Vector128<byte> left, Vector128<byte> right)
43924393
{
43934394
if (Sse2.IsSupported)
43944395
{
43954396
return Sse2.UnpackLow(left, right);
43964397
}
4397-
else if (!AdvSimd.Arm64.IsSupported)
4398+
else if (AdvSimd.Arm64.IsSupported)
43984399
{
4399-
ThrowHelper.ThrowNotSupportedException();
4400+
return AdvSimd.Arm64.ZipLow(left, right);
4401+
}
4402+
else if (PackedSimd.IsSupported)
4403+
{
4404+
// Compose with two PackedSimd.Swizzle calls (clamp out-of-range to 0) plus OR.
4405+
// We call PackedSimd.Swizzle directly rather than Vector128.ShuffleNative because
4406+
// the latter goes through a Ssse3 -> AdvSimd.Arm64 -> PackedSimd dispatcher chain
4407+
// that the Mono SIMD intrinsic recognizer doesn't always lower cleanly.
4408+
// PackedSimd.Shuffle (two-vector i8x16.shuffle) requires constant lane indices
4409+
// and is impractical to call portably from generic code paths.
4410+
Vector128<byte> leftPart = PackedSimd.Swizzle(left,
4411+
Vector128.Create((byte)0, 0xFF, 1, 0xFF, 2, 0xFF, 3, 0xFF, 4, 0xFF, 5, 0xFF, 6, 0xFF, 7, 0xFF));
4412+
Vector128<byte> rightPart = PackedSimd.Swizzle(right,
4413+
Vector128.Create((byte)0xFF, 0, 0xFF, 1, 0xFF, 2, 0xFF, 3, 0xFF, 4, 0xFF, 5, 0xFF, 6, 0xFF, 7));
4414+
return leftPart | rightPart;
44004415
}
4401-
return AdvSimd.Arm64.ZipLow(left, right);
4416+
ThrowHelper.ThrowNotSupportedException();
4417+
return default;
44024418
}
44034419

44044420
[MethodImpl(MethodImplOptions.AggressiveInlining)]
44054421
[CompExactlyDependsOn(typeof(AdvSimd.Arm64))]
44064422
[CompExactlyDependsOn(typeof(Sse2))]
4423+
[CompExactlyDependsOn(typeof(PackedSimd))]
44074424
internal static Vector128<byte> UnpackHigh(Vector128<byte> left, Vector128<byte> right)
44084425
{
44094426
if (Sse2.IsSupported)
44104427
{
44114428
return Sse2.UnpackHigh(left, right);
44124429
}
4413-
else if (!AdvSimd.Arm64.IsSupported)
4430+
else if (AdvSimd.Arm64.IsSupported)
44144431
{
4415-
ThrowHelper.ThrowNotSupportedException();
4432+
return AdvSimd.Arm64.ZipHigh(left, right);
4433+
}
4434+
else if (PackedSimd.IsSupported)
4435+
{
4436+
Vector128<byte> leftPart = PackedSimd.Swizzle(left,
4437+
Vector128.Create((byte)8, 0xFF, 9, 0xFF, 10, 0xFF, 11, 0xFF, 12, 0xFF, 13, 0xFF, 14, 0xFF, 15, 0xFF));
4438+
Vector128<byte> rightPart = PackedSimd.Swizzle(right,
4439+
Vector128.Create((byte)0xFF, 8, 0xFF, 9, 0xFF, 10, 0xFF, 11, 0xFF, 12, 0xFF, 13, 0xFF, 14, 0xFF, 15));
4440+
return leftPart | rightPart;
44164441
}
4417-
return AdvSimd.Arm64.ZipHigh(left, right);
4442+
ThrowHelper.ThrowNotSupportedException();
4443+
return default;
44184444
}
44194445
}
44204446
}

‎src/libraries/System.Private.CoreLib/src/System/SearchValues/ProbabilisticMap.cs‎

Lines changed: 14 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -80,7 +80,7 @@ public ProbabilisticMap(ReadOnlySpan<char> values)
8080
[BypassReadyToRun]
8181
private static void SetCharBit(ref uint charMap, byte value)
8282
{
83-
if (Sse41.IsSupported || AdvSimd.Arm64.IsSupported)
83+
if (Sse41.IsSupported || AdvSimd.Arm64.IsSupported || PackedSimd.IsSupported)
8484
{
8585
Unsafe.Add(ref Unsafe.As<uint, byte>(ref charMap), value & VectorizedIndexMask) |= (byte)(1u << (value >> VectorizedIndexShift));
8686
}
@@ -92,7 +92,7 @@ private static void SetCharBit(ref uint charMap, byte value)
9292

9393
[MethodImpl(MethodImplOptions.AggressiveInlining)]
9494
[BypassReadyToRun]
95-
private static bool IsCharBitSet(ref uint charMap, byte value) => Sse41.IsSupported || AdvSimd.Arm64.IsSupported
95+
private static bool IsCharBitSet(ref uint charMap, byte value) => Sse41.IsSupported || AdvSimd.Arm64.IsSupported || PackedSimd.IsSupported
9696
? (Unsafe.Add(ref Unsafe.As<uint, byte>(ref charMap), value & VectorizedIndexMask) & (1u << (value >> VectorizedIndexShift))) != 0
9797
: (Unsafe.Add(ref charMap, value & PortableIndexMask) & (1u << (value >> PortableIndexShift))) != 0;
9898

@@ -220,6 +220,7 @@ private static Vector256<byte> IsCharBitNotSetAvx2(Vector256<byte> charMapLower,
220220
[MethodImpl(MethodImplOptions.AggressiveInlining)]
221221
[CompExactlyDependsOn(typeof(AdvSimd.Arm64))]
222222
[CompExactlyDependsOn(typeof(Sse2))]
223+
[CompExactlyDependsOn(typeof(PackedSimd))]
223224
private static Vector128<byte> ContainsMask16Chars(Vector128<byte> charMapLower, Vector128<byte> charMapUpper, ref char searchSpace)
224225
{
225226
Vector128<ushort> source0 = Vector128.LoadUnsafe(ref searchSpace);
@@ -238,6 +239,11 @@ private static Vector128<byte> ContainsMask16Chars(Vector128<byte> charMapLower,
238239
sourceLower = AdvSimd.Arm64.UnzipEven(source0.AsByte(), source1.AsByte());
239240
sourceUpper = AdvSimd.Arm64.UnzipOdd(source0.AsByte(), source1.AsByte());
240241
}
242+
else if (PackedSimd.IsSupported)
243+
{
244+
sourceLower = PackedSimd.ConvertNarrowingSaturateUnsigned((source0 & Vector128.Create((ushort)255)).AsInt16(), (source1 & Vector128.Create((ushort)255)).AsInt16());
245+
sourceUpper = PackedSimd.ConvertNarrowingSaturateUnsigned((source0 >>> 8).AsInt16(), (source1 >>> 8).AsInt16());
246+
}
241247
else
242248
{
243249
// We explicitly recheck each IsSupported query to ensure that the trimmer can see which paths are live/dead
@@ -392,7 +398,7 @@ private static unsafe int ProbabilisticLastIndexOfAny(ref char searchSpace, int
392398
internal static int IndexOfAny<TUseFastContains>(ref char searchSpace, int searchSpaceLength, ref ProbabilisticMapState state)
393399
where TUseFastContains : struct, SearchValues.IRuntimeConst
394400
{
395-
if ((Sse41.IsSupported || AdvSimd.Arm64.IsSupported) && searchSpaceLength >= 16)
401+
if ((Sse41.IsSupported || AdvSimd.Arm64.IsSupported || PackedSimd.IsSupported) && searchSpaceLength >= 16)
396402
{
397403
return Vector512.IsHardwareAccelerated && Avx512Vbmi.VL.IsSupported
398404
? IndexOfAnyVectorizedAvx512<TUseFastContains>(ref searchSpace, searchSpaceLength, ref state)
@@ -406,7 +412,7 @@ internal static int IndexOfAny<TUseFastContains>(ref char searchSpace, int searc
406412
internal static int LastIndexOfAny<TUseFastContains>(ref char searchSpace, int searchSpaceLength, ref ProbabilisticMapState state)
407413
where TUseFastContains : struct, SearchValues.IRuntimeConst
408414
{
409-
if ((Sse41.IsSupported || AdvSimd.Arm64.IsSupported) && searchSpaceLength >= 16)
415+
if ((Sse41.IsSupported || AdvSimd.Arm64.IsSupported || PackedSimd.IsSupported) && searchSpaceLength >= 16)
410416
{
411417
return Vector512.IsHardwareAccelerated && Avx512Vbmi.VL.IsSupported
412418
? LastIndexOfAnyVectorizedAvx512<TUseFastContains>(ref searchSpace, searchSpaceLength, ref state)
@@ -501,10 +507,11 @@ private static int IndexOfAnyVectorizedAvx512<TUseFastContains>(ref char searchS
501507

502508
[CompExactlyDependsOn(typeof(AdvSimd.Arm64))]
503509
[CompExactlyDependsOn(typeof(Sse41))]
510+
[CompExactlyDependsOn(typeof(PackedSimd))]
504511
private static int IndexOfAnyVectorized<TUseFastContains>(ref char searchSpace, int searchSpaceLength, ref ProbabilisticMapState state)
505512
where TUseFastContains : struct, SearchValues.IRuntimeConst
506513
{
507-
Debug.Assert(Sse41.IsSupported || AdvSimd.Arm64.IsSupported);
514+
Debug.Assert(Sse41.IsSupported || AdvSimd.Arm64.IsSupported || PackedSimd.IsSupported);
508515
Debug.Assert(searchSpaceLength >= 16);
509516

510517
ref char searchSpaceEnd = ref Unsafe.Add(ref searchSpace, searchSpaceLength);
@@ -679,10 +686,11 @@ private static int LastIndexOfAnyVectorizedAvx512<TUseFastContains>(ref char sea
679686

680687
[CompExactlyDependsOn(typeof(AdvSimd.Arm64))]
681688
[CompExactlyDependsOn(typeof(Sse41))]
689+
[CompExactlyDependsOn(typeof(PackedSimd))]
682690
private static int LastIndexOfAnyVectorized<TUseFastContains>(ref char searchSpace, int searchSpaceLength, ref ProbabilisticMapState state)
683691
where TUseFastContains : struct, SearchValues.IRuntimeConst
684692
{
685-
Debug.Assert(Sse41.IsSupported || AdvSimd.Arm64.IsSupported);
693+
Debug.Assert(Sse41.IsSupported || AdvSimd.Arm64.IsSupported || PackedSimd.IsSupported);
686694
Debug.Assert(searchSpaceLength >= 16);
687695

688696
ref char cur = ref Unsafe.Add(ref searchSpace, searchSpaceLength);

‎src/libraries/System.Private.CoreLib/src/System/SearchValues/Strings/AsciiStringSearchValuesTeddyBase.cs‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@
88
using System.Runtime.InteropServices;
99
using System.Runtime.Intrinsics;
1010
using System.Runtime.Intrinsics.Arm;
11+
using System.Runtime.Intrinsics.Wasm;
1112
using System.Runtime.Intrinsics.X86;
1213
using static System.Buffers.StringSearchValuesHelper;
1314
using static System.Buffers.TeddyHelper;
@@ -150,6 +151,7 @@ protected AsciiStringSearchValuesTeddyBase(string[][] buckets, ReadOnlySpan<stri
150151

151152
[CompExactlyDependsOn(typeof(Ssse3))]
152153
[CompExactlyDependsOn(typeof(AdvSimd.Arm64))]
154+
[CompExactlyDependsOn(typeof(PackedSimd))]
153155
protected int IndexOfAnyN2(ReadOnlySpan<char> span)
154156
{
155157
// The behavior of the rest of the function remains the same if Avx2 or Avx512BW aren't supported
@@ -170,6 +172,7 @@ protected int IndexOfAnyN2(ReadOnlySpan<char> span)
170172

171173
[CompExactlyDependsOn(typeof(Ssse3))]
172174
[CompExactlyDependsOn(typeof(AdvSimd.Arm64))]
175+
[CompExactlyDependsOn(typeof(PackedSimd))]
173176
protected int IndexOfAnyN3(ReadOnlySpan<char> span)
174177
{
175178
// The behavior of the rest of the function remains the same if Avx2 or Avx512BW aren't supported
@@ -190,6 +193,7 @@ protected int IndexOfAnyN3(ReadOnlySpan<char> span)
190193

191194
[CompExactlyDependsOn(typeof(Ssse3))]
192195
[CompExactlyDependsOn(typeof(AdvSimd.Arm64))]
196+
[CompExactlyDependsOn(typeof(PackedSimd))]
193197
private int IndexOfAnyN2Vector128(ReadOnlySpan<char> span)
194198
{
195199
// See comments in 'IndexOfAnyN3Vector128' below.
@@ -350,6 +354,7 @@ private int IndexOfAnyN2Avx512(ReadOnlySpan<char> span)
350354

351355
[CompExactlyDependsOn(typeof(Ssse3))]
352356
[CompExactlyDependsOn(typeof(AdvSimd.Arm64))]
357+
[CompExactlyDependsOn(typeof(PackedSimd))]
353358
private int IndexOfAnyN3Vector128(ReadOnlySpan<char> span)
354359
{
355360
// We can't process inputs shorter than 18 characters in a vectorized manner here.

0 commit comments

Comments
 (0)