[SLPVectorizer] Refactor HorizontalReduction::createOp (NFC) #121549
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Mel-Chen
requested review from
preames,
RKSimon,
alexey-bataev and
HanKuanChen
|
@llvm/pr-subscribers-vectorizers @llvm/pr-subscribers-llvm-transforms Author: Mel Chen (Mel-Chen) ChangesThis patch refactors the
This patch is mainly to achieve the first item. The main reason is to unify IAnyOf and FAnyOf reduction. #118393 Full diff: https://github.com/llvm/llvm-project/pull/121549.diff 1 Files Affected:
diff --git a/llvm/lib/Transforms/Vectorize/SLPVectorizer.cpp b/llvm/lib/Transforms/Vectorize/SLPVectorizer.cpp
index f52ddfda5e64c7..5a6f952cc91db7 100644
--- a/llvm/lib/Transforms/Vectorize/SLPVectorizer.cpp
+++ b/llvm/lib/Transforms/Vectorize/SLPVectorizer.cpp
@@ -19357,59 +19357,43 @@ class HorizontalReduction {
/// Creates reduction operation with the current opcode.
static Value *createOp(IRBuilderBase &Builder, RecurKind Kind, Value *LHS,
Value *RHS, const Twine &Name, bool UseSelect) {
- unsigned RdxOpcode = RecurrenceDescriptor::getOpcode(Kind);
+ if (UseSelect) {
+ if (RecurrenceDescriptor::isIntMinMaxRecurrenceKind(Kind)) {
+ CmpInst::Predicate Pred = llvm::getMinMaxReductionPredicate(Kind);
+ Value *Cmp = Builder.CreateCmp(Pred, LHS, RHS, Name);
+ return Builder.CreateSelect(Cmp, LHS, RHS, Name);
+ }
+ if ((Kind == RecurKind::Or || Kind == RecurKind::And) &&
+ LHS->getType() == CmpInst::makeCmpResultType(LHS->getType())) {
+ Value *TrueVal = Kind == RecurKind::Or ? Builder.getTrue() : RHS;
+ Value *FalseVal = Kind == RecurKind::Or ? RHS : Builder.getFalse();
+ return Builder.CreateSelect(LHS, TrueVal, FalseVal, Name);
+ }
+ }
+
switch (Kind) {
case RecurKind::Or:
- if (UseSelect &&
- LHS->getType() == CmpInst::makeCmpResultType(LHS->getType()))
- return Builder.CreateSelect(LHS, Builder.getTrue(), RHS, Name);
- return Builder.CreateBinOp((Instruction::BinaryOps)RdxOpcode, LHS, RHS,
- Name);
case RecurKind::And:
- if (UseSelect &&
- LHS->getType() == CmpInst::makeCmpResultType(LHS->getType()))
- return Builder.CreateSelect(LHS, RHS, Builder.getFalse(), Name);
- return Builder.CreateBinOp((Instruction::BinaryOps)RdxOpcode, LHS, RHS,
- Name);
case RecurKind::Add:
case RecurKind::Mul:
case RecurKind::Xor:
case RecurKind::FAdd:
- case RecurKind::FMul:
+ case RecurKind::FMul: {
+ unsigned RdxOpcode = RecurrenceDescriptor::getOpcode(Kind);
return Builder.CreateBinOp((Instruction::BinaryOps)RdxOpcode, LHS, RHS,
Name);
+ }
case RecurKind::FMax:
- return Builder.CreateBinaryIntrinsic(Intrinsic::maxnum, LHS, RHS);
case RecurKind::FMin:
- return Builder.CreateBinaryIntrinsic(Intrinsic::minnum, LHS, RHS);
case RecurKind::FMaximum:
- return Builder.CreateBinaryIntrinsic(Intrinsic::maximum, LHS, RHS);
case RecurKind::FMinimum:
- return Builder.CreateBinaryIntrinsic(Intrinsic::minimum, LHS, RHS);
case RecurKind::SMax:
- if (UseSelect) {
- Value *Cmp = Builder.CreateICmpSGT(LHS, RHS, Name);
- return Builder.CreateSelect(Cmp, LHS, RHS, Name);
- }
- return Builder.CreateBinaryIntrinsic(Intrinsic::smax, LHS, RHS);
case RecurKind::SMin:
- if (UseSelect) {
- Value *Cmp = Builder.CreateICmpSLT(LHS, RHS, Name);
- return Builder.CreateSelect(Cmp, LHS, RHS, Name);
- }
- return Builder.CreateBinaryIntrinsic(Intrinsic::smin, LHS, RHS);
case RecurKind::UMax:
- if (UseSelect) {
- Value *Cmp = Builder.CreateICmpUGT(LHS, RHS, Name);
- return Builder.CreateSelect(Cmp, LHS, RHS, Name);
- }
- return Builder.CreateBinaryIntrinsic(Intrinsic::umax, LHS, RHS);
- case RecurKind::UMin:
- if (UseSelect) {
- Value *Cmp = Builder.CreateICmpULT(LHS, RHS, Name);
- return Builder.CreateSelect(Cmp, LHS, RHS, Name);
- }
- return Builder.CreateBinaryIntrinsic(Intrinsic::umin, LHS, RHS);
+ case RecurKind::UMin: {
+ Intrinsic::ID Id = llvm::getMinMaxReductionIntrinsicOp(Kind);
+ return Builder.CreateBinaryIntrinsic(Id, LHS, RHS);
+ }
default:
llvm_unreachable("Unknown reduction operation.");
}
|
HanKuanChen
reviewed
Jan 3, 2025
Mel-Chen
force-pushed
the
remove-call-getopcode-slp
branch
from
1e5ed86 to
d1a7fd9
Compare
Mel-Chen
force-pushed
the
remove-call-getopcode-slp
branch
from
d1a7fd9 to
1d7e6a7
Compare
|
LLVM Buildbot has detected a new failure on builder Full details are available at: https://lab.llvm.org/buildbot/#/builders/73/builds/11587 Here is the relevant piece of the build log for the reference |
|
LLVM Buildbot has detected a new failure on builder Full details are available at: https://lab.llvm.org/buildbot/#/builders/16/builds/11865 Here is the relevant piece of the build log for the reference |
Mel-Chen
added a commit
to Mel-Chen/llvm-project
that referenced
this pull request
Jan 13, 2025
…1549) This patch simplifies select-based integer min/max reductions by utilizing `llvm::getMinMaxReductionPredicate`, and generates intrinsic-based min/max reductions by utilizing `llvm::getMinMaxReductionIntrinsicOp`.
kazutakahirata
pushed a commit
to kazutakahirata/llvm-project
that referenced
this pull request
Jan 13, 2025
…1549) This patch simplifies select-based integer min/max reductions by utilizing `llvm::getMinMaxReductionPredicate`, and generates intrinsic-based min/max reductions by utilizing `llvm::getMinMaxReductionIntrinsicOp`.
DKLoehr
pushed a commit
to DKLoehr/llvm-project
that referenced
this pull request
Jan 17, 2025
…1549) This patch simplifies select-based integer min/max reductions by utilizing `llvm::getMinMaxReductionPredicate`, and generates intrinsic-based min/max reductions by utilizing `llvm::getMinMaxReductionIntrinsicOp`.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This patch simplifies select-based integer min/max reductions by utilizing
llvm::getMinMaxReductionPredicate, and generating intrinsic-based min/max reductions by utilizingllvm::getMinMaxReductionIntrinsicOp.