Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

wasmi_core: add support for the Wasm simd proposal #1395

Merged
merged 87 commits into from
Mar 20, 2025
Merged
Changes from 1 commit
Commits
Show all changes
87 commits
Select commit Hold shift + click to select a range
86a9890
add simd crate feature to wasmi_core crate
Robbepop Mar 16, 2025
5c7a578
implement V128 skeleton
Robbepop Mar 16, 2025
53e526f
rename IntoLane and Lane (appending 's')
Robbepop Mar 16, 2025
769727a
add docs to most (or all) simd.rs definitions
Robbepop Mar 16, 2025
a8c0ae9
add note comments to unsafe transmutes
Robbepop Mar 16, 2025
bc87e4c
implement SIMD splat instructions
Robbepop Mar 16, 2025
d437879
add extract_lane SIMD APIs
Robbepop Mar 17, 2025
64d3c18
expect dead_code warnings
Robbepop Mar 17, 2025
1deaba6
remove empty line
Robbepop Mar 17, 2025
d8b74ad
remove non-existing extract_lane APIs
Robbepop Mar 17, 2025
5952ab1
add replace_lane SIMD APIs
Robbepop Mar 17, 2025
9336f6b
add V128 From impls for bytes and i128
Robbepop Mar 17, 2025
9c15875
add SIMD binary integer math ops
Robbepop Mar 17, 2025
f532fed
add unary integer math SIMD ops
Robbepop Mar 17, 2025
5e55754
add SIMD integer abs ops
Robbepop Mar 17, 2025
2181ab5
add unsigned SIMD lanes types
Robbepop Mar 17, 2025
43635a3
use unsigned SIMD types where applicable for now
Robbepop Mar 17, 2025
0b134b3
improve docs for IntoLanes and IntoLaneIdx
Robbepop Mar 17, 2025
42cc71c
add helper IntoLanewiseWidening trait
Robbepop Mar 17, 2025
ca36c58
clean-up API design
Robbepop Mar 17, 2025
62dea1c
add SIMD ext_mul_{low,high} ops
Robbepop Mar 17, 2025
01cf099
generate docs for new SIMD extmul ops
Robbepop Mar 17, 2025
431ee78
add extadd_pairwise SIMD ops
Robbepop Mar 17, 2025
7dece0a
implement add_sat and sub_sat SIMD ops
Robbepop Mar 17, 2025
2f3c753
add i16x8_q15mulr_sat_s SIMD operation
Robbepop Mar 17, 2025
edd0ea9
add SIMD integer min and max operations
Robbepop Mar 17, 2025
9edb2aa
add SIMD avgr_u ops
Robbepop Mar 17, 2025
2f9f0c0
implement Wasm shift SIMD ops
Robbepop Mar 17, 2025
c4268a8
add bitwise SIMD ops
Robbepop Mar 17, 2025
448cb31
add v128.bitselect SIMD operation
Robbepop Mar 17, 2025
d4bbd34
add SIMD i8x16.popcnt instruction
Robbepop Mar 17, 2025
40b4219
add i32x4.dot_i16x8_s SIMD operation
Robbepop Mar 17, 2025
500d904
implement SIMD reduce ops
Robbepop Mar 17, 2025
707150d
add SIMD comparison instructions
Robbepop Mar 17, 2025
b7e9afc
add unary float SIMD instructions
Robbepop Mar 18, 2025
de36135
move op! macro to top of file
Robbepop Mar 18, 2025
c2bd961
add float binary SIMD instructions
Robbepop Mar 18, 2025
78bf871
Merge branch 'main' into rf-implement-simd-proposal
Robbepop Mar 19, 2025
f992a54
add lanewise unary cast SIMD ops
Robbepop Mar 19, 2025
cf4b5bb
rewrite i32x4.dot_i16x8_s in terms of other fns
Robbepop Mar 19, 2025
c4956ad
add FromNarrow trait and impls
Robbepop Mar 19, 2025
983c271
add V128 convenience methods for FromNarrow
Robbepop Mar 19, 2025
45d3969
rewrite extmul ops in terms of FromNarrow APIs
Robbepop Mar 19, 2025
d275444
rewrite extadd ops in terms of FromNarrow APIs
Robbepop Mar 19, 2025
9d80b48
add new extend SIMD ops
Robbepop Mar 19, 2025
7390796
remove old widening and narrowing APIs
Robbepop Mar 19, 2025
055b579
no longer silence dead_code warnings
Robbepop Mar 19, 2025
9317bbd
add missing docs for convenience method
Robbepop Mar 19, 2025
92e7193
refactor unary widen SIMD ops generation
Robbepop Mar 19, 2025
e953c9f
add more widen-low SIMD ops
Robbepop Mar 19, 2025
df55aac
fix signedness of some SIMD extend-{low,high} ops
Robbepop Mar 19, 2025
8ab2f53
rename macro
Robbepop Mar 19, 2025
e89fa00
add narrowing-low-high SIMD ops
Robbepop Mar 19, 2025
1b47827
rename FromNarrow method
Robbepop Mar 19, 2025
37a3b83
fix i32x4_dot_i16x8_s SIMD impl
Robbepop Mar 19, 2025
9344e29
re-format code
Robbepop Mar 19, 2025
3c5b067
re-write from_low_high impl
Robbepop Mar 19, 2025
9184272
add narrowing low-or SIMD ops
Robbepop Mar 19, 2025
127d340
convert all V128 methods to functions in the simd module
Robbepop Mar 19, 2025
4ec17dc
align docs with others
Robbepop Mar 19, 2025
32e5ced
align more docs
Robbepop Mar 19, 2025
718655e
replace register128 feature with simd feature
Robbepop Mar 19, 2025
3c9ecf7
move load/store stuff into memory.rs module
Robbepop Mar 19, 2025
83230b6
redesign WrapInto and ExtendInto macros
Robbepop Mar 19, 2025
cd11fe2
Merge branch 'main' into rf-implement-simd-proposal
Robbepop Mar 19, 2025
ba56cb9
remove ImmByte since it is unused
Robbepop Mar 19, 2025
aa72956
rename OutOfBoundsLaneId
Robbepop Mar 19, 2025
48c4d4a
merge macro invocations
Robbepop Mar 19, 2025
8055b3d
re-design extmul macros
Robbepop Mar 19, 2025
68da596
refactor macros
Robbepop Mar 19, 2025
52e266e
refactor memory submodule
Robbepop Mar 19, 2025
630481a
Merge branch 'main' into rf-implement-simd-proposal
Robbepop Mar 19, 2025
22cd724
add v128 store instructions
Robbepop Mar 19, 2025
b6b355b
add v128.load evaluators
Robbepop Mar 20, 2025
63601ae
add LaneIndex trait
Robbepop Mar 20, 2025
29ea33b
add more IntoLaneIdx trait impls
Robbepop Mar 20, 2025
f23072c
add impls for v128.load{32,64}_zero
Robbepop Mar 20, 2025
8f01d8a
update docs
Robbepop Mar 20, 2025
f375a29
add v128.loadN_splat instruction impls
Robbepop Mar 20, 2025
62f2895
add v128.loadN_lane SIMD instruction impls
Robbepop Mar 20, 2025
1ac376a
update docs error sections
Robbepop Mar 20, 2025
c5895d0
implement V128 load_mxn_{s,u} ops
Robbepop Mar 20, 2025
fd2d37a
configure Cargo to generate simd docs
Robbepop Mar 20, 2025
c80fcb0
remove value128 crate feature
Robbepop Mar 20, 2025
d25333b
fix avgr SIMD instructions and clippy warnings
Robbepop Mar 20, 2025
d4d95eb
rename some FromNarrow methods to please clippy
Robbepop Mar 20, 2025
b9c5446
add SIMD shuffle and swizzle ops
Robbepop Mar 20, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
add narrowing low-or SIMD ops
Robbepop committed Mar 19, 2025
commit 91842726e9e9f0033d947ab64ee451a364bc48e5
61 changes: 61 additions & 0 deletions crates/core/src/simd.rs
Original file line number Diff line number Diff line change
@@ -6,7 +6,7 @@

macro_rules! op {
($ty:ty, $op:tt) => {{
|lhs: $ty, rhs: $ty| lhs $op rhs

Check warning on line 9 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L9

Added line #L9 was not covered by tests
}};
}

@@ -16,51 +16,51 @@
pub struct V128([u8; 16]);

impl From<[u8; 16]> for V128 {
fn from(bytes: [u8; 16]) -> Self {
Self(bytes)

Check warning on line 20 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L19-L20

Added lines #L19 - L20 were not covered by tests
}
}

impl From<i128> for V128 {
fn from(value: i128) -> Self {
Self(value.to_le_bytes())

Check warning on line 26 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L25-L26

Added lines #L25 - L26 were not covered by tests
}
}

impl V128 {
/// Creates an `i128` value from the bytes of `self`.
pub(crate) fn to_i128(self) -> i128 {
i128::from_le_bytes(self.0)

Check warning on line 33 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L32-L33

Added lines #L32 - L33 were not covered by tests
}
}

impl From<UntypedVal> for V128 {
fn from(value: UntypedVal) -> Self {
let u128 = (u128::from(value.hi64) << 64) | (u128::from(value.lo64));
Self(u128.to_le_bytes())

Check warning on line 40 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L38-L40

Added lines #L38 - L40 were not covered by tests
}
}

impl From<V128> for UntypedVal {
fn from(value: V128) -> Self {
let u128 = u128::from_le_bytes(value.0);
let lo64 = u128 as u64;
let hi64 = (u128 >> 64) as u64;

Check warning on line 48 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L45-L48

Added lines #L45 - L48 were not covered by tests
Self { lo64, hi64 }
}
}

impl ReadAs<V128> for UntypedVal {
fn read_as(&self) -> V128 {

Check warning on line 54 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L54

Added line #L54 was not covered by tests
// Note: we can re-use the `From` impl since both types are of equal size.
V128::from(*self)

Check warning on line 56 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L56

Added line #L56 was not covered by tests
}
}

impl WriteAs<V128> for UntypedVal {
fn write_as(&mut self, value: V128) {

Check warning on line 61 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L61

Added line #L61 was not covered by tests
// Note: we can re-use the `From` impl since both types are of equal size.
*self = UntypedVal::from(value);

Check warning on line 63 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L63

Added line #L63 was not covered by tests
}
}

@@ -87,9 +87,9 @@
) => {
$(
$( #[$attr] )*
#[derive(Debug, Copy, Clone, PartialEq, Eq)]

Check warning on line 90 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L90

Added line #L90 was not covered by tests
#[repr(transparent)]
pub struct $name(u8);

Check warning on line 92 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L92

Added line #L92 was not covered by tests

impl IntoLaneIdx for [(); $n] {
type LaneIdx = $name;
@@ -102,19 +102,19 @@
/// Returns the lane id as `u8`.
///
/// This will never return a `u8` value that is out of bounds for `self`.
pub fn get(self) -> u8 {
self.0 & Self::MASK

Check warning on line 106 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L105-L106

Added lines #L105 - L106 were not covered by tests
}
}

impl TryFrom<u8> for $name {
type Error = OutOfBoundsLaneId;

fn try_from(lane: u8) -> Result<Self, Self::Error> {
if lane > Self::MASK {
return Err(OutOfBoundsLaneId)

Check warning on line 115 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L113-L115

Added lines #L113 - L115 were not covered by tests
}
Ok(Self(lane))

Check warning on line 117 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L117

Added line #L117 was not covered by tests
}
}
)*
@@ -211,7 +211,7 @@
) => {
$(
$( #[$attr] )*
#[derive(Copy, Clone)]

Check warning on line 214 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L214

Added line #L214 was not covered by tests
#[repr(transparent)]
struct $name([$ty; $n]);

@@ -252,47 +252,47 @@
Self([value; $n])
}

fn extract_lane(self, lane: Self::LaneIdx) -> Self::Item {
self.0[lane.get() as usize]

Check warning on line 256 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L255-L256

Added lines #L255 - L256 were not covered by tests
}

fn replace_lane(self, lane: Self::LaneIdx, item: Self::Item) -> Self {
let mut this = self;
this.0[lane.get() as usize] = item;
this

Check warning on line 262 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L259-L262

Added lines #L259 - L262 were not covered by tests
}

fn lanewise_unary(self, f: impl Fn(Self::Item) -> Self::Item) -> Self {
let mut this = self.0;
for i in 0..Self::LANES {
this[i] = f(this[i]);

Check warning on line 268 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L265-L268

Added lines #L265 - L268 were not covered by tests
}
Self(this)

Check warning on line 270 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L270

Added line #L270 was not covered by tests
}

fn lanewise_binary(self, other: Self, f: impl Fn(Self::Item, Self::Item) -> Self::Item) -> Self {
let mut lhs = self.0;
let rhs = other.0;
for i in 0..Self::LANES {
lhs[i] = f(lhs[i], rhs[i]);

Check warning on line 277 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L273-L277

Added lines #L273 - L277 were not covered by tests
}
Self(lhs)

Check warning on line 279 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L279

Added line #L279 was not covered by tests
}

fn lanewise_comparison(self, other: Self, f: impl Fn(Self::Item, Self::Item) -> bool) -> Self {
self.lanewise_binary(other, |lhs, rhs| match f(lhs, rhs) {
true => Self::ALL_ONES,
false => Self::ALL_ZEROS,

Check warning on line 285 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L282-L285

Added lines #L282 - L285 were not covered by tests
})
}

fn lanewise_reduce<T>(self, acc: T, f: impl Fn(u8, Self::Item, T) -> T) -> T {
let this = self.0;
let mut acc = acc;
for i in 0..Self::LANES {
acc = f(i as u8, this[i], acc);

Check warning on line 293 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L289-L293

Added lines #L289 - L293 were not covered by tests
}
acc

Check warning on line 295 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L295

Added line #L295 was not covered by tests
}
}
)*
@@ -363,12 +363,12 @@
( $( impl FromNarrow<$narrow_ty:ty> for $self_ty:ty; )* ) => {
$(
impl FromNarrow<$narrow_ty> for $self_ty {
fn from_pairwise_unary(

Check warning on line 366 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L366

Added line #L366 was not covered by tests
narrow: $narrow_ty,
f: impl Fn(<$narrow_ty as Lanes>::Item, <$narrow_ty as Lanes>::Item) -> Self::Item,
) -> Self {
let narrow = narrow.0;
Self(array::from_fn(|i| f(narrow[2 * i], narrow[2 * i + 1])))

Check warning on line 371 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L370-L371

Added lines #L370 - L371 were not covered by tests
}

fn from_pairwise_binary(
@@ -386,33 +386,33 @@
}))
}

fn from_low_unary(narrow: $narrow_ty, f: impl Fn(<$narrow_ty as Lanes>::Item) -> Self::Item) -> Self {
Self(array::from_fn(|i| f(narrow.0[i])))

Check warning on line 390 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L389-L390

Added lines #L389 - L390 were not covered by tests
}

fn from_high_unary(narrow: $narrow_ty, f: impl Fn(<$narrow_ty as Lanes>::Item) -> Self::Item) -> Self {
Self(array::from_fn(|i| f(narrow.0[i + Self::LANES])))

Check warning on line 394 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L393-L394

Added lines #L393 - L394 were not covered by tests
}

fn from_low_binary(

Check warning on line 397 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L397

Added line #L397 was not covered by tests
narrow_lhs: $narrow_ty,
narrow_rhs: $narrow_ty,
f: impl Fn(<$narrow_ty as Lanes>::Item, <$narrow_ty as Lanes>::Item) -> Self::Item,
) -> Self {
let narrow_lhs = narrow_lhs.0;
let narrow_rhs = narrow_rhs.0;
Self(array::from_fn(|i| f(narrow_lhs[i], narrow_rhs[i])))

Check warning on line 404 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L402-L404

Added lines #L402 - L404 were not covered by tests
}

fn from_high_binary(

Check warning on line 407 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L407

Added line #L407 was not covered by tests
narrow_lhs: $narrow_ty,
narrow_rhs: $narrow_ty,
f: impl Fn(<$narrow_ty as Lanes>::Item, <$narrow_ty as Lanes>::Item) -> Self::Item,
) -> Self {
let narrow_lhs = narrow_lhs.0;
let narrow_rhs = narrow_rhs.0;
Self(array::from_fn(|i| {
f(narrow_lhs[i + Self::LANES], narrow_rhs[i + Self::LANES])

Check warning on line 415 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L412-L415

Added lines #L412 - L415 were not covered by tests
}))
}
}
@@ -443,6 +443,13 @@
high: WideLanes,
f: impl Fn(WideLanes::Item) -> Self::Item,
) -> Self;

/// Construct `Self` from the application of `f` to the wide `low` or evaluate `high`.
fn from_low_or(
low: WideLanes,
high: impl Fn() -> Self::Item,
f: impl Fn(WideLanes::Item) -> Self::Item,
) -> Self;
}

macro_rules! impl_from_wide_for {
@@ -451,25 +458,42 @@
) => {
$(
impl FromWide<$wide_ty> for $narrow_ty {
fn from_low_high(

Check warning on line 461 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L461

Added line #L461 was not covered by tests
low: $wide_ty,
high: $wide_ty,
f: impl Fn(<$wide_ty as Lanes>::Item) -> Self::Item,
) -> Self {
let low = low.0;
let high = high.0;
Self(array::from_fn(|i| {
match i < <$wide_ty as Lanes>::LANES {
true => f(low[i]),
false => f(high[i]),

Check warning on line 471 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L466-L471

Added lines #L466 - L471 were not covered by tests
}
}))
}

fn from_low_or(

Check warning on line 476 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L476

Added line #L476 was not covered by tests
low: $wide_ty,
high: impl Fn() -> Self::Item,
f: impl Fn(<$wide_ty as Lanes>::Item) -> Self::Item,
) -> Self {
let low = low.0;
Self(array::from_fn(|i| {
match i < <$wide_ty as Lanes>::LANES {
true => f(low[i]),
false => high(),

Check warning on line 485 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L481-L485

Added lines #L481 - L485 were not covered by tests
}
}))
}
}
)*
};
}
impl_from_wide_for! {
impl FromWide<F64x2> for I32x4;
impl FromWide<F64x2> for U32x4;
impl FromWide<F64x2> for F32x4;
impl FromWide<I32x4> for I16x8;
impl FromWide<U32x4> for U16x8;
impl FromWide<I16x8> for I8x16;
@@ -483,14 +507,14 @@
macro_rules! impl_reinterpret_as_for {
( $ty0:ty, $ty1:ty ) => {
impl ReinterpretAs<$ty0> for $ty1 {
fn reinterpret_as(self) -> $ty0 {
<$ty0>::from_ne_bytes(self.to_ne_bytes())

Check warning on line 511 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L510-L511

Added lines #L510 - L511 were not covered by tests
}
}

impl ReinterpretAs<$ty1> for $ty0 {
fn reinterpret_as(self) -> $ty1 {
<$ty1>::from_ne_bytes(self.to_ne_bytes())

Check warning on line 517 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L516-L517

Added lines #L516 - L517 were not covered by tests
}
}
};
@@ -507,60 +531,60 @@
}

/// Convenience method to help implement lane extraction methods.
fn extract_lane<T: IntoLanes>(self, lane: <T as IntoLanes>::LaneIdx) -> T {
<<T as IntoLanes>::Lanes>::from_v128(self).extract_lane(lane)

Check warning on line 535 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L534-L535

Added lines #L534 - L535 were not covered by tests
}

/// Convenience method to help implement lane replacement methods.
fn replace_lane<T: IntoLanes>(self, lane: <T as IntoLanes>::LaneIdx, item: T) -> Self {
<<T as IntoLanes>::Lanes>::from_v128(self)
.replace_lane(lane, item)

Check warning on line 541 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L539-L541

Added lines #L539 - L541 were not covered by tests
.into_v128()
}

/// Convenience method to help implement lanewise unary methods.
fn lanewise_unary<T: IntoLanes>(self, f: impl Fn(T) -> T) -> Self {
<<T as IntoLanes>::Lanes>::from_v128(self)
.lanewise_unary(f)

Check warning on line 548 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L546-L548

Added lines #L546 - L548 were not covered by tests
.into_v128()
}

/// Convenience method to help implement lanewise unary cast methods.
fn lanewise_unary_cast<T: IntoLanes, U>(self, f: impl Fn(T) -> U) -> Self

Check warning on line 553 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L553

Added line #L553 was not covered by tests
where
U: ReinterpretAs<T>,
{
<<T as IntoLanes>::Lanes>::from_v128(self)
.lanewise_unary(|v| f(v).reinterpret_as())

Check warning on line 558 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L557-L558

Added lines #L557 - L558 were not covered by tests
.into_v128()
}

/// Convenience method to help implement lanewise binary methods.
fn lanewise_binary<T: IntoLanes>(lhs: Self, rhs: Self, f: impl Fn(T, T) -> T) -> Self {
let lhs = <<T as IntoLanes>::Lanes>::from_v128(lhs);
let rhs = <<T as IntoLanes>::Lanes>::from_v128(rhs);
lhs.lanewise_binary(rhs, f).into_v128()

Check warning on line 566 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L563-L566

Added lines #L563 - L566 were not covered by tests
}

/// Convenience method to help implement lanewise comparison methods.
fn lanewise_comparison<T: IntoLanes>(lhs: Self, rhs: Self, f: impl Fn(T, T) -> bool) -> Self {
let lhs = <<T as IntoLanes>::Lanes>::from_v128(lhs);
let rhs = <<T as IntoLanes>::Lanes>::from_v128(rhs);
lhs.lanewise_comparison(rhs, f).into_v128()

Check warning on line 573 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L570-L573

Added lines #L570 - L573 were not covered by tests
}

/// Convenience method to help implement lanewise reduce methods.
fn lanewise_reduce<T: IntoLanes, V>(self, acc: V, f: impl Fn(T, V) -> V) -> V {
self.lanewise_reduce_enumerate::<T, V>(acc, |_, v: T, acc: V| f(v, acc))

Check warning on line 578 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L577-L578

Added lines #L577 - L578 were not covered by tests
}

/// Convenience method to help implement lanewise reduce methods with a loop-index.
fn lanewise_reduce_enumerate<T: IntoLanes, V>(self, acc: V, f: impl Fn(u8, T, V) -> V) -> V {
<<T as IntoLanes>::Lanes>::from_v128(self).lanewise_reduce(acc, f)

Check warning on line 583 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L582-L583

Added lines #L582 - L583 were not covered by tests
}

/// Convenience method to help implement pairwise unary methods.
fn pairwise_unary<Narrow: IntoLanes, Wide: IntoLanes>(

Check warning on line 587 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L587

Added line #L587 was not covered by tests
self,
f: impl Fn(Narrow, Narrow) -> Wide,
) -> Self
@@ -568,8 +592,8 @@
<Wide as IntoLanes>::Lanes: FromNarrow<<Narrow as IntoLanes>::Lanes>,
{
<<Wide as IntoLanes>::Lanes as FromNarrow<<Narrow as IntoLanes>::Lanes>>::from_pairwise_unary(
<<Narrow as IntoLanes>::Lanes>::from_v128(self),
f,

Check warning on line 596 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L595-L596

Added lines #L595 - L596 were not covered by tests
)
.into_v128()
}
@@ -592,31 +616,31 @@
}

/// Convenience method to help implement extend-low unary methods.
fn from_low_unary<Narrow: IntoLanes, Wide: IntoLanes>(self, f: impl Fn(Narrow) -> Wide) -> Self

Check warning on line 619 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L619

Added line #L619 was not covered by tests
where
<Wide as IntoLanes>::Lanes: FromNarrow<<Narrow as IntoLanes>::Lanes>,
{
<<Wide as IntoLanes>::Lanes as FromNarrow<<Narrow as IntoLanes>::Lanes>>::from_low_unary(
<<Narrow as IntoLanes>::Lanes>::from_v128(self),
f,

Check warning on line 625 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L624-L625

Added lines #L624 - L625 were not covered by tests
)
.into_v128()
}

/// Convenience method to help implement extend-high unary methods.
fn from_high_unary<Narrow: IntoLanes, Wide: IntoLanes>(self, f: impl Fn(Narrow) -> Wide) -> Self

Check warning on line 631 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L631

Added line #L631 was not covered by tests
where
<Wide as IntoLanes>::Lanes: FromNarrow<<Narrow as IntoLanes>::Lanes>,
{
<<Wide as IntoLanes>::Lanes as FromNarrow<<Narrow as IntoLanes>::Lanes>>::from_high_unary(
<<Narrow as IntoLanes>::Lanes>::from_v128(self),
f,

Check warning on line 637 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L636-L637

Added lines #L636 - L637 were not covered by tests
)
.into_v128()
}

/// Convenience method to help implement extend-low binary methods.
fn from_low_binary<Narrow: IntoLanes, Wide: IntoLanes>(

Check warning on line 643 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L643

Added line #L643 was not covered by tests
lhs: Self,
rhs: Self,
f: impl Fn(Narrow, Narrow) -> Wide,
@@ -625,15 +649,15 @@
<Wide as IntoLanes>::Lanes: FromNarrow<<Narrow as IntoLanes>::Lanes>,
{
<<Wide as IntoLanes>::Lanes as FromNarrow<<Narrow as IntoLanes>::Lanes>>::from_low_binary(
<<Narrow as IntoLanes>::Lanes>::from_v128(lhs),
<<Narrow as IntoLanes>::Lanes>::from_v128(rhs),
f,

Check warning on line 654 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L652-L654

Added lines #L652 - L654 were not covered by tests
)
.into_v128()
}

/// Convenience method to help implement extend-high binary methods.
fn from_high_binary<Narrow: IntoLanes, Wide: IntoLanes>(

Check warning on line 660 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L660

Added line #L660 was not covered by tests
lhs: Self,
rhs: Self,
f: impl Fn(Narrow, Narrow) -> Wide,
@@ -642,15 +666,15 @@
<Wide as IntoLanes>::Lanes: FromNarrow<<Narrow as IntoLanes>::Lanes>,
{
<<Wide as IntoLanes>::Lanes as FromNarrow<<Narrow as IntoLanes>::Lanes>>::from_high_binary(
<<Narrow as IntoLanes>::Lanes>::from_v128(lhs),
<<Narrow as IntoLanes>::Lanes>::from_v128(rhs),
f,

Check warning on line 671 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L669-L671

Added lines #L669 - L671 were not covered by tests
)
.into_v128()
}

/// Convenience method to help implement narrowing low-high methods.
fn from_low_high<Narrow: IntoLanes, Wide: IntoLanes>(

Check warning on line 677 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L677

Added line #L677 was not covered by tests
lhs: Self,
rhs: Self,
f: impl Fn(Wide) -> Narrow,
@@ -659,25 +683,42 @@
<Narrow as IntoLanes>::Lanes: FromWide<<Wide as IntoLanes>::Lanes>,
{
<<Narrow as IntoLanes>::Lanes as FromWide<<Wide as IntoLanes>::Lanes>>::from_low_high(
<<Wide as IntoLanes>::Lanes>::from_v128(lhs),
<<Wide as IntoLanes>::Lanes>::from_v128(rhs),
f,

Check warning on line 688 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L686-L688

Added lines #L686 - L688 were not covered by tests
)
.into_v128()
}

/// Convenience method to help implement narrowing low-or methods.
fn from_low_or<Narrow: IntoLanes, Wide: IntoLanes>(

Check warning on line 694 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L694

Added line #L694 was not covered by tests
self,
high: impl Fn() -> Narrow,
f: impl Fn(Wide) -> Narrow,
) -> Self
where
<Narrow as IntoLanes>::Lanes: FromWide<<Wide as IntoLanes>::Lanes>,
{
<<Narrow as IntoLanes>::Lanes as FromWide<<Wide as IntoLanes>::Lanes>>::from_low_or(
<<Wide as IntoLanes>::Lanes>::from_v128(self),
high,
f,

Check warning on line 705 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L703-L705

Added lines #L703 - L705 were not covered by tests
)
.into_v128()
}
}

/// Concenience identity helper function.
fn identity<T>(x: T) -> T {
x

Check warning on line 713 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L712-L713

Added lines #L712 - L713 were not covered by tests
}

macro_rules! impl_splat_for {
( $( fn $name:ident(value: $ty:ty) -> Self; )* ) => {
$(
#[doc = concat!("Executes a Wasm `", stringify!($name), "` instruction.")]
pub fn $name(value: $ty) -> Self {
Self::splat(value)

Check warning on line 721 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L720-L721

Added lines #L720 - L721 were not covered by tests
}
)*
};
@@ -697,8 +738,8 @@
( $( fn $name:ident(self, lane: $lane_ty:ty) -> $ret_ty:ty = $convert:expr; )* ) => {
$(
#[doc = concat!("Executes a Wasm `", stringify!($name), "` instruction.")]
pub fn $name(self, lane: $lane_ty) -> $ret_ty {
($convert)(self.extract_lane(lane))

Check warning on line 742 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L741-L742

Added lines #L741 - L742 were not covered by tests
}
)*
};
@@ -720,8 +761,8 @@
( $( fn $name:ident(self, lane: $lane_ty:ty, item: $item_ty:ty) -> Self; )* ) => {
$(
#[doc = concat!("Executes a Wasm `", stringify!($name), "` instruction.")]
pub fn $name(self, lane: $lane_ty, item: $item_ty) -> Self {
self.replace_lane(lane, item)

Check warning on line 765 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L764-L765

Added lines #L764 - L765 were not covered by tests
}
)*
};
@@ -741,8 +782,8 @@
( $( fn $name:ident(self) -> Self = $lanewise_expr:expr; )* ) => {
$(
#[doc = concat!("Executes a Wasm `", stringify!($name), "` instruction.")]
pub fn $name(self) -> Self {
Self::lanewise_unary(self, $lanewise_expr)

Check warning on line 786 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L785-L786

Added lines #L785 - L786 were not covered by tests
}
)*
};
@@ -752,24 +793,24 @@
( $( fn $name:ident(self) -> Self = $lanewise_expr:expr; )* ) => {
$(
#[doc = concat!("Executes a Wasm `", stringify!($name), "` instruction.")]
pub fn $name(self) -> Self {
Self::lanewise_unary_cast(self, $lanewise_expr)

Check warning on line 797 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L796-L797

Added lines #L796 - L797 were not covered by tests
}
)*
};
}

/// Lanewise operation for the Wasm `q15mulr_sat` SIMD operation.
fn i16x8_q15mulr_sat(x: i16, y: i16) -> i16 {
(x * y + 0x4000) >> 15

Check warning on line 805 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L804-L805

Added lines #L804 - L805 were not covered by tests
}

macro_rules! impl_binary_for {
( $( fn $name:ident(lhs: Self, rhs: Self) -> Self = $lanewise_expr:expr; )* ) => {
$(
#[doc = concat!("Executes a Wasm `", stringify!($name), "` instruction.")]
pub fn $name(lhs: Self, rhs: Self) -> Self {
Self::lanewise_binary(lhs, rhs, $lanewise_expr)

Check warning on line 813 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L812-L813

Added lines #L812 - L813 were not covered by tests
}
)*
};
@@ -815,13 +856,13 @@
fn i32x4_max_s(lhs: Self, rhs: Self) -> Self = i32::max;
fn i32x4_max_u(lhs: Self, rhs: Self) -> Self = u32::max;

fn i8x16_avgr_u(lhs: Self, rhs: Self) -> Self = |a: u8, b: u8| (a + b + 1) / 2;
fn i16x8_avgr_u(lhs: Self, rhs: Self) -> Self = |a: u16, b: u16| (a + b + 1) / 2;

Check warning on line 860 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L859-L860

Added lines #L859 - L860 were not covered by tests

fn v128_and(lhs: Self, rhs: Self) -> Self = <i64 as BitAnd>::bitand;
fn v128_or(lhs: Self, rhs: Self) -> Self = <i64 as BitOr>::bitor;
fn v128_xor(lhs: Self, rhs: Self) -> Self = <i64 as BitXor>::bitxor;
fn v128_andnot(lhs: Self, rhs: Self) -> Self = |a: i64, b: i64| a & !b;

Check warning on line 865 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L865

Added line #L865 was not covered by tests

fn f32x4_min(lhs: Self, rhs: Self) -> Self = wasm::f32_min;
fn f64x2_min(lhs: Self, rhs: Self) -> Self = wasm::f64_min;
@@ -854,7 +895,7 @@

fn v128_not(self) -> Self = <i64 as Not>::not;

fn i8x16_popcnt(self) -> Self = |v: u8| v.count_ones() as u8;

Check warning on line 898 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L898

Added line #L898 was not covered by tests

fn f32x4_neg(self) -> Self = <f32 as Neg>::neg;
fn f64x2_neg(self) -> Self = <f64 as Neg>::neg;
@@ -884,8 +925,8 @@
( $( fn $name:ident(lhs: Self, rhs: Self) -> Self = $lanewise_expr:expr; )* ) => {
$(
#[doc = concat!("Executes a Wasm `", stringify!($name), "` instruction.")]
pub fn $name(lhs: Self, rhs: Self) -> Self {
Self::lanewise_comparison(lhs, rhs, $lanewise_expr)

Check warning on line 929 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L928-L929

Added lines #L928 - L929 were not covered by tests
}
)*
};
@@ -954,8 +995,8 @@
) => {
$(
#[doc = concat!("Executes a Wasm `", stringify!($name), "` instruction.")]
pub fn $name(self) -> Self {
self.from_low_unary($convert)

Check warning on line 999 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L998-L999

Added lines #L998 - L999 were not covered by tests
}
)*
};
@@ -981,8 +1022,8 @@
) => {
$(
#[doc = concat!("Executes a Wasm `", stringify!($name), "` instruction.")]
pub fn $name(self) -> Self {
self.from_high_unary($convert)

Check warning on line 1026 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L1025-L1026

Added lines #L1025 - L1026 were not covered by tests
}
)*
};
@@ -1010,23 +1051,23 @@
) => {
$(
#[doc = concat!("Executes a Wasm `", stringify!($extmul_low), "` instruction.")]
pub fn $extmul_low(lhs: Self, rhs: Self) -> Self {
fn extmul_low(a: $narrow, b: $narrow) -> $wide {
let a = <$wide>::from(a);
let b = <$wide>::from(b);
a.wrapping_mul(b)

Check warning on line 1058 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L1054-L1058

Added lines #L1054 - L1058 were not covered by tests
}
Self::from_low_binary(lhs, rhs, extmul_low)

Check warning on line 1060 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L1060

Added line #L1060 was not covered by tests
}

#[doc = concat!("Executes a Wasm `", stringify!($extmul_high), "` instruction.")]
pub fn $extmul_high(lhs: Self, rhs: Self) -> Self {
fn extmul_high(a: $narrow, b: $narrow) -> $wide {
let a = <$wide>::from(a);
let b = <$wide>::from(b);
a.wrapping_mul(b)

Check warning on line 1068 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L1064-L1068

Added lines #L1064 - L1068 were not covered by tests
}
Self::from_high_binary(lhs, rhs, extmul_high)

Check warning on line 1070 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L1070

Added line #L1070 was not covered by tests
}
)*
};
@@ -1072,13 +1113,13 @@
) => {
$(
#[doc = concat!("Executes a Wasm `", stringify!($name), "` instruction.")]
pub fn $name(self) -> Self {
fn extadd_pairwise(a: $narrow, b: $narrow) -> $wide {
let a = <$wide>::from(a);
let b = <$wide>::from(b);
a.wrapping_add(b)

Check warning on line 1120 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L1116-L1120

Added lines #L1116 - L1120 were not covered by tests
}
self.pairwise_unary(extadd_pairwise)

Check warning on line 1122 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L1122

Added line #L1122 was not covered by tests
}
)*
};
@@ -1098,8 +1139,8 @@
) => {
$(
#[doc = concat!("Executes a Wasm `", stringify!($name), "` instruction.")]
pub fn $name(self, rhs: u32) -> Self {
self.lanewise_unary(|v| $lanewise_expr(v, rhs))

Check warning on line 1143 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L1142-L1143

Added lines #L1142 - L1143 were not covered by tests
}
)*
};
@@ -1127,29 +1168,49 @@
) => {
$(
#[doc = concat!("Executes a Wasm `", stringify!($name), "` instruction.")]
pub fn $name(low: Self, high: Self) -> Self {
Self::from_low_high(low, high, $f)

Check warning on line 1172 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L1171-L1172

Added lines #L1171 - L1172 were not covered by tests
}
)*
};
}
impl V128 {
impl_narrowing_low_high_ops! {
fn i8x16_narrow_i16x8_s(low: Self, high: Self) -> Self = |v: i16| v as i8;
fn i8x16_narrow_i16x8_u(low: Self, high: Self) -> Self = |v: u16| v as u8;
fn i16x8_narrow_i32x4_s(low: Self, high: Self) -> Self = |v: i32| v as i16;
fn i16x8_narrow_i32x4_u(low: Self, high: Self) -> Self = |v: u32| v as u16;

Check warning on line 1182 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L1179-L1182

Added lines #L1179 - L1182 were not covered by tests
}
}

macro_rules! impl_narrowing_low_high_ops {
(
$( fn $name:ident(self) -> Self = (high: $high:expr, f: $f:expr); )*
) => {
$(
#[doc = concat!("Executes a Wasm `", stringify!($name), "` instruction.")]
pub fn $name(low: Self) -> Self {
Self::from_low_or(low, $high, $f)

Check warning on line 1193 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L1192-L1193

Added lines #L1192 - L1193 were not covered by tests
}
)*
};
}
impl V128 {
impl_narrowing_low_high_ops! {
fn i32x4_trunc_sat_f64x2_s_zero(self) -> Self = (high: || 0, f: wasm::i32_trunc_sat_f64_s);
fn i32x4_trunc_sat_f64x2_u_zero(self) -> Self = (high: || 0, f: wasm::i32_trunc_sat_f64_u);
fn f32x4_demote_f64x2_zero(self) -> Self = (high: || 0.0, f: wasm::f32_demote_f64);

Check warning on line 1202 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L1200-L1202

Added lines #L1200 - L1202 were not covered by tests
}
}

macro_rules! impl_reduce_ops {
(
$( fn $name:ident(self) -> bool = all_true($item_ty:ty); )*
) => {
$(
#[doc = concat!("Executes a Wasm `", stringify!($name), "` instruction.")]
pub fn $name(self) -> bool {
self.lanewise_reduce(true, |v: $item_ty, acc| acc & (v != 0))

Check warning on line 1213 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L1212-L1213

Added lines #L1212 - L1213 were not covered by tests
}
)*
};
@@ -1158,18 +1219,18 @@
) => {
$(
#[doc = concat!("Executes a Wasm `", stringify!($name), "` instruction.")]
pub fn $name(self) -> u32 {
self.lanewise_reduce_enumerate(0_i32, |n, v: $item_ty, acc| {
acc | (i32::from(v < 0).wrapping_shl(u32::from(n)))
}) as _

Check warning on line 1225 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L1222-L1225

Added lines #L1222 - L1225 were not covered by tests
}
)*
};
}
impl V128 {
/// Executes a Wasm `v128.any_true` instruction.
pub fn v128_any_true(self) -> bool {
self.to_i128() != 0

Check warning on line 1233 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L1232-L1233

Added lines #L1232 - L1233 were not covered by tests
}

impl_reduce_ops! {
@@ -1200,8 +1261,8 @@
}

/// Execute a Wasm `v128.bitselect` instruction.
pub fn v128_bitselect(v1: Self, v2: Self, c: Self) -> Self {
Self::v128_or(Self::v128_and(v1, c), Self::v128_andnot(v2, c))

Check warning on line 1265 in crates/core/src/simd.rs

Codecov / codecov/patch

crates/core/src/simd.rs#L1264-L1265

Added lines #L1264 - L1265 were not covered by tests
}
}