Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Vectorizes IndexOfMin/Max/Magnitude #93469

Merged
merged 20 commits into from
Oct 20, 2023
Merged

Vectorizes IndexOfMin/Max/Magnitude #93469

merged 20 commits into from
Oct 20, 2023

Conversation

michaelgsharp
Copy link
Member

Adds a vectorized implementation of IndexOfMin, IndexOfMax, IndexOfMaxMagnitude, IndexOfMinMagnitude.

@ghost ghost assigned michaelgsharp Oct 13, 2023
@ghost
Copy link

ghost commented Oct 13, 2023

Tagging subscribers to this area: @dotnet/area-system-numerics
See info in area-owners.md if you want to be subscribed.

Issue Details

Adds a vectorized implementation of IndexOfMin, IndexOfMax, IndexOfMaxMagnitude, IndexOfMinMagnitude.

Author: michaelgsharp
Assignees: michaelgsharp
Labels:

area-System.Numerics
Milestone: -

@michaelgsharp
Copy link
Member Author

I'm still finishing up vectorizing the NetStandard version so its not included here but I wanted to get it up for review. Should have the NetStandard done here shortly. When its done the ifdefs in TensorPrimitives.cs will go away, so don't worry about those.

Also, not sure if the .sln change should be there or not? Anyone know?

stephentoub
stephentoub reviewed Oct 13, 2023
View reviewed changes
src/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs Outdated Show resolved Hide resolved
src/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.netcore.cs Outdated Show resolved Hide resolved
src/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.netcore.cs Outdated Show resolved Hide resolved
src/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.netcore.cs Outdated Show resolved Hide resolved
src/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.netcore.cs Outdated Show resolved Hide resolved
src/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.netcore.cs Outdated Show resolved Hide resolved
src/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.netcore.cs Outdated Show resolved Hide resolved
src/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.netcore.cs Outdated Show resolved Hide resolved
src/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.netcore.cs Outdated Show resolved Hide resolved
@build-analysis build-analysis bot mentioned this pull request Oct 13, 2023
Open
5 tasks
@michaelgsharp michaelgsharp marked this pull request as ready for review October 16, 2023 18:05
@ericstj
Copy link
Member

ericstj commented Oct 16, 2023 •
edited
Loading

Failures here are test failures, the MathFMaxMagnitude uses IndexOfMaxMagnitude which is changed in this PR.
https://github.com/dotnet/runtime/blob/45ed0ab41c138ad1683b333b5b937b8240f3e4d8/src/libraries/System.Numerics.Tensors/tests/TensorPrimitivesTests.cs#L1476C70-L1476C70
I suspect this PR broke those tests. Probably you don't have those tests locally as Stephen recently added them.

@build-analysis build-analysis bot mentioned this pull request Oct 19, 2023
Open
stephentoub
stephentoub requested changes Oct 19, 2023
View reviewed changes
src/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs Show resolved Hide resolved
src/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs Show resolved Hide resolved
src/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs Show resolved Hide resolved
src/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.cs Show resolved Hide resolved
src/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.netcore.cs Outdated Show resolved Hide resolved
src/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.netcore.cs Outdated Show resolved Hide resolved
...ibraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.netstandard.cs Outdated Show resolved Hide resolved
...ibraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.netstandard.cs Outdated Show resolved Hide resolved
...ibraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.netstandard.cs Outdated Show resolved Hide resolved
...ibraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.netstandard.cs Show resolved Hide resolved
@ghost ghost added needs-author-action An issue or pull request that requires more info or actions from the author. and removed needs-author-action An issue or pull request that requires more info or actions from the author. labels Oct 19, 2023
stephentoub
stephentoub reviewed Oct 20, 2023
View reviewed changes
Copy link
Member

@stephentoub stephentoub left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly LGTM, but two issues to be addressed.

@build-analysis build-analysis bot mentioned this pull request Oct 20, 2023
Closed
@ericstj ericstj self-requested a review October 20, 2023 15:07
@michaelgsharp michaelgsharp merged commit f3f0af8 into dotnet:main Oct 20, 2023
109 checks passed
@michaelgsharp michaelgsharp deleted the indexof branch October 20, 2023 17:19
michaelgsharp added a commit to michaelgsharp/runtime that referenced this pull request Oct 20, 2023
@michaelgsharp @ericstj
Vectorizes IndexOfMin/Max/Magnitude (dotnet#93469)
cb48e75 
* resolved merge conflicts

* net core full done

* minor code cleanup

* NetStandard and PR fixes.

* minor pr changes

* Fix IndexOfMaxMagnitudeOperator

* Fix IndexOfMaxMagnitudeOperator on netcore

* updates from PR comments

* netcore fixed

* net standard updated

* add reference assembly exclusions

* made naive approach better

* resolved PR comments

* minor comment changes

* minor formatting fixes

* added inlining

* fixes from PR comments

* comments from pr

* fixed spacing

---------

Co-authored-by: Eric StJohn <ericstj@microsoft.com>
carlossanlop pushed a commit that referenced this pull request Oct 20, 2023
@michaelgsharp @stephentoub
@tannergooding @radical @radekdoulik @ericstj
[release/8.0] Vectorize TensorPrimitives APIs (#93746)
8262af2 
* Use FMA in TensorPrimitives (#92205)

* Simplify TensorPrimitive's AbsoluteOperator (#92577)

Vector{128/256/512} all provide Abs; no need to do this manually.

* Reduce some boilerplate in TensorPrimitive's IBinaryOperator (#92576)

Change a few of the static abstract interface methods to be virtual, as most implementations throw from these methods; we can consolidate that throwing to the base.

* Minor code cleanup in TensorPrimitives tests (#92575)

* Normalize some test naming

* Alphabetize tests

* Improve mistmatched length tests with all positions of the shorter tensor

* Alphabetize methods in TensorPrimitives.cs

* Vectorize TensorPrimitives.Min/Max{Magnitude} (#92618)

* Vectorize TensorPrimitives.Min/Max{Magnitude}

* Use AdvSimd.Max/Min

* Rename some parameters/locals for consistency

* Improve HorizontalAggregate

* Move a few helpers

* Avoid scalar path for returning found NaN

* Update TensorPrimitives aggregations to vectorize handling of remaining elements (#92672)

* Update TensorPrimitives.CosineSimilarity to vectorize handling of remaining elements

* Vectorize remainder handling for Aggregate helpers

* Flesh out TensorPrimitives XML docs (#92749)

* Flesh out TensorPrimitives XML docs

* Address PR feedback

- Remove use of FusedMultiplyAdd from all but CosineSimilarity
- Remove comments about platform/OS-specific behavior from Add/AddMultiply/Subtract/Multiply/MultiplyAdd/Divide/Negate
- Loosen comments about NaN and which exact one is returned

* Address PR feedback

* Vectorize TensorPrimitives.ConvertToHalf (#92715)

* Enable TensorPrimitives to perform in-place operations (#92820)

Some operations would produce incorrect results if the same span was passed as both an input and an output.  When vectorization was employed but the span's length wasn't a perfect multiple of a vector, we'd do the standard trick of performing one last operation on the last vector's worth of data; however, that relies on the operation being idempotent, and if a previous operation has overwritten input with a new value due to the same memory being used for input and output, some operations won't be idempotent.  This fixes that by masking off the already processed elements.  It adds tests to validate in-place use works, and it updates the docs to carve out this valid overlapping.

* Vectorize TensorPrimitives.ConvertToSingle (#92779)

* Vectorize TensorPrimitives.ConvertToSingle

* Address PR feedback

* Throw exception in TensorPrimitives for unsupported span overlaps (#92838)

* This vectorizes TensorPrimitives.Log2 (#92897)

* Add a way to support operations that can't be vectorized on netstandard

* Updating TensorPrimitives.Log2 to be vectorized on .NET Core

* Update src/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.netstandard.cs

Co-authored-by: Stephen Toub <stoub@microsoft.com>

* Ensure we do an arithmetic right shift in the Log2 vectorization

* Ensure the code can compile on .NET 7

* Ensure that edge cases are properly handled and don't resolve to `x`

* Ensure that Log2 special results are explicitly handled.

---------

Co-authored-by: Stephen Toub <stoub@microsoft.com>

* Adding Log2 tests covering some special values (#92946)

* [wasm] Disable `TensorPrimitivesTests.ConvertToHalf_SpecialValues` (#92953)

Failing test: `System.Numerics.Tensors.Tests.TensorPrimitivesTests.ConvertToHalf_SpecialValues`

Issue: #92885

* Adding a vectorized implementation of TensorPrimitives.Log (#92960)

* Adding a vectorized implementation of TensorPrimitives.Log

* Make sure to hit Ctrl+S

* Consolidate some TensorPrimitivesTests logic around special values (#92982)

* Vectorize TensorPrimitives.Exp (#93018)

* Vectorize TensorPrimitives.Exp

* Update src/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/TensorPrimitives.netstandard.cs

* Vectorize TensorPrimitives.Sigmoid and TensorPrimitives.SoftMax (#93029)

* Vectorize TensorPrimitives.Sigmoid and TensorPrimitives.SoftMax

- Adds a SigmoidOperator that just wraps the ExpOperator
- Vectorizes both passes of SoftMax, on top of ExpOperator. Simplest way to do this was to augment the existing InvokeSpanScalarIntoSpan to take a transform operator.
- In doing so, found some naming inconsistencies I'd previously introduced, so I did some automatic renaming to make things more consistent.
- Added XML comments to all the internal/private surface area.
- Fleshes out some tests (and test values).

* Disable tests on mono

* Address PR feedback

* Vectorize TensorPrimitives.Tanh/Cosh/Sinh (#93093)

* Vectorize TensorPrimitives.Tanh/Cosh/Sinh

Tanh and Cosh are based on AOCL-LibM.

AOCL-LibM doesn't appear to have a sinh implementation, so this Sinh is just based on the sinh formula based on exp(x).

I also augmented the tests further, including:
- Added more tests for sinh/cosh/tanh
- Add an equality routine that supports comparing larger values with a tolerance
- Tightened the tolerance for most functions
- Changed some tests to be theories to be consistent with style elsewhere in the tests
- Fixed some use of Math to be MathF

* Remove unnecessary special-handling path from cosh

* Remove unnecessary special-handling path from tanh

* Redo sinh based on cosh

* Address PR feedback

* Replace confusing new T[] { ... }

* Remove a few unnecessary `unsafe` keyword uses in TensorPrimitives (#93219)

* Consolidate a few exception throws in TensorPrimitives (#93168)

* Fix TensorPrimitives.IndexOfXx corner-case when first element is seed value (#93169)

* Fix TensorPrimitives.IndexOfXx corner-case when first element is seed value

Found as part of adding more tests for Min/Max{Magnitude} to validate they match their IndexOfXx variants.

* Address PR feedback

* Improve a vector implementation to support alignment and non-temporal tores (#93296)

* Improve a vector implementation to support alignment and non-temporal stores

* Fix a build error and mark a couple methods as AggressiveInlining

* Fix the remaining block count computation

* Ensure overlapping for small data on the V256/512 is handled

* Ensure we only go down the vectorized path when supported for netstandard

* Mark TensorPrimitives as unsafe (#93412)

* Use the improved vectorization algorithm for binary and ternary TensorPrimitives operations (#93409)

* Update InvokeSpanSpanIntoSpan<TBinaryOperator> for TensorPrimitives to use the better SIMD algorithm

* Update InvokeSpanScalarIntoSpan<TTransformOperator, TBinaryOperator> for TensorPrimitives to use the better SIMD algorithm

* Update InvokeSpanSpanSpanIntoSpan<TTernaryOperator> for TensorPrimitives to use the better SIMD algorithm

* Update InvokeSpanSpanScalarIntoSpan<TTernaryOperator> for TensorPrimitives to use the better SIMD algorithm

* Update InvokeSpanScalarSpanIntoSpan<TTernaryOperator> for TensorPrimitives to use the better SIMD algorithm

* Improve codegen slightly by using case 0, rather than default

* Adjust the canAlign check to be latter, to reduce branch count for data under the threshold

* Add a comment explaining the NonTemporalByteThreshold

* Make sure xTransformOp.CanVectorize is checked on .NET Standard

* Use the improved vectorization algorithm for aggregate TensorPrimitives operations (#93695)

* Improve the handling of the IAggregationOperator implementations

* Update Aggregate<TTransformOperator, TAggregationOperator> for TensorPrimitives to use the better SIMD algorithm

* Update Aggregate<TBinaryOperator, TAggregationOperator> for TensorPrimitives to use the better SIMD algorithm

* Respond to PR feedback

* [wasm] Remove more active issues for #92885 (#93596)

* adding patch from pr 93556

* Vectorizes IndexOfMin/Max/Magnitude (#93469)

* resolved merge conflicts

* net core full done

* minor code cleanup

* NetStandard and PR fixes.

* minor pr changes

* Fix IndexOfMaxMagnitudeOperator

* Fix IndexOfMaxMagnitudeOperator on netcore

* updates from PR comments

* netcore fixed

* net standard updated

* add reference assembly exclusions

* made naive approach better

* resolved PR comments

* minor comment changes

* minor formatting fixes

* added inlining

* fixes from PR comments

* comments from pr

* fixed spacing

---------

Co-authored-by: Eric StJohn <ericstj@microsoft.com>

---------

Co-authored-by: Stephen Toub <stoub@microsoft.com>
Co-authored-by: Tanner Gooding <tagoo@outlook.com>
Co-authored-by: Ankit Jain <radical@gmail.com>
Co-authored-by: Radek Doulik <radek.doulik@gmail.com>
Co-authored-by: Eric StJohn <ericstj@microsoft.com>
@ghost ghost locked as resolved and limited conversation to collaborators Nov 19, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@stephentoub stephentoub stephentoub approved these changes

@jeffhandley jeffhandley Awaiting requested review from jeffhandley

@tannergooding tannergooding Awaiting requested review from tannergooding

@ericstj ericstj Awaiting requested review from ericstj

Assignees

@michaelgsharp michaelgsharp

Labels
area-System.Numerics
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@michaelgsharp @ericstj @stephentoub @tannergooding