[InstCombine] Optimize sub(sext(add(x,y)),sext(add(x,z))). #144174
+413
−10
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This pattern can be often met in Flang generated LLVM IR, for example, for the counts of the loops generated for array expressions like: `a(x:x+y)` or `a(x+z:x+z)` or their variations. In order to compute the loop count, Flang needs to subtract the lower bound of the array slice from the upper bound of the array slice. To avoid the sign wraps, it sign extends the original values (that may be of any user data type) to `i64`. This peephole is really helpful in CPU2017/548.exchange2, where we have multiple following statements like this: ``` block(row+1:row+2, 7:9, i7) = block(row+1:row+2, 7:9, i7) - 10 ``` While this is just a 2x3 iterations loop nest, LLVM cannot figure it out, ending up vectorizing the inner loop really hard (with a vector epilog and scalar remainder). This, in turn, causes problems for LSR that ends up creating too many loop-carried values in the loop containing the above statement, which are then causing too many spills/reloads. Alive2: https://alive2.llvm.org/ce/z/gLgfYX Related to llvm#143219.
|
@llvm/pr-subscribers-llvm-ir @llvm/pr-subscribers-llvm-transforms Author: Slava Zakharin (vzakhari) ChangesThis pattern can be often met in Flang generated LLVM IR, In order to compute the loop count, Flang needs to subtract This peephole is really helpful in CPU2017/548.exchange2, While this is just a 2x3 iterations loop nest, LLVM cannot Alive2: https://alive2.llvm.org/ce/z/gLgfYX Related to #143219. Full diff: https://github.com/llvm/llvm-project/pull/144174.diff 2 Files Affected:
diff --git a/llvm/lib/Transforms/InstCombine/InstCombineAddSub.cpp b/llvm/lib/Transforms/InstCombine/InstCombineAddSub.cpp
index c1ce364eb1794..35de76d50672d 100644
--- a/llvm/lib/Transforms/InstCombine/InstCombineAddSub.cpp
+++ b/llvm/lib/Transforms/InstCombine/InstCombineAddSub.cpp
@@ -2807,6 +2807,62 @@ Instruction *InstCombinerImpl::visitSub(BinaryOperator &I) {
if (Instruction *Res = foldBinOpOfSelectAndCastOfSelectCondition(I))
return Res;
+ // (sub[ nsw][ nuw] (sext (add nsw (X, Y)), sext (X))) --> (sext (Y))
+ {
+ Value *Add0;
+ if (match(Op0, m_SExt(m_Value(Add0))) &&
+ match(Add0, m_Add(m_Value(X), m_Value(Y))) &&
+ match(Op1, m_SExt(m_Specific(X)))) {
+ auto *OBO0 = cast<OverflowingBinaryOperator>(Add0);
+ if (OBO0->hasNoSignedWrap()) {
+ // Non-constant Y requires new SExt.
+ unsigned numOfNewInstrs = !isa<Constant>(Y) ? 1 : 0;
+ // Check if we can trade some of the old instructions for the new ones.
+ unsigned numOfDeadInstrs = 0;
+ numOfDeadInstrs += Op0->hasOneUse() ? 1 : 0;
+ numOfDeadInstrs += Op1->hasOneUse() ? 1 : 0;
+ numOfDeadInstrs += Add0->hasOneUse() ? 1 : 0;
+ if (numOfDeadInstrs >= numOfNewInstrs) {
+ Value *SExtY = Builder.CreateSExt(Y, I.getType());
+ return replaceInstUsesWith(I, SExtY);
+ }
+ }
+ }
+ }
+
+ // (sub[ nsw] (sext (add nsw (X, Y)), sext (add nsw (X, Z)))) -->
+ // --> (sub[ nsw] (sext (Y), sext(Z)))
+ {
+ Value *Z, *Add0, *Add1;
+ if (match(Op0, m_SExt(m_Value(Add0))) &&
+ match(Add0, m_Add(m_Value(X), m_Value(Y))) &&
+ match(Op1, m_SExt(m_Value(Add1))) &&
+ match(Add1, m_Add(m_Specific(X), m_Value(Z)))) {
+ auto *OBO0 = cast<OverflowingBinaryOperator>(Add0);
+ auto *OBO1 = cast<OverflowingBinaryOperator>(Add1);
+ if (OBO0->hasNoSignedWrap() && OBO1->hasNoSignedWrap()) {
+ unsigned numOfNewInstrs = 0;
+ // Non-constant Y, Z require new SExt.
+ numOfNewInstrs += !isa<Constant>(Y) ? 1 : 0;
+ numOfNewInstrs += !isa<Constant>(Z) ? 1 : 0;
+ // Check if we can trade some of the old instructions for the new ones.
+ unsigned numOfDeadInstrs = 0;
+ numOfDeadInstrs += Op0->hasOneUse() ? 1 : 0;
+ numOfDeadInstrs += Op1->hasOneUse() ? 1 : 0;
+ numOfDeadInstrs += Add0->hasOneUse() ? 1 : 0;
+ numOfDeadInstrs += Add1->hasOneUse() ? 1 : 0;
+ if (numOfDeadInstrs >= numOfNewInstrs) {
+ Value *SExtY = Builder.CreateSExt(Y, I.getType());
+ Value *SExtZ = Builder.CreateSExt(Z, I.getType());
+ Value *Sub = Builder.CreateSub(SExtY, SExtZ, "",
+ /* HasNUW */ false,
+ /* HasNSW */ I.hasNoSignedWrap());
+ return replaceInstUsesWith(I, Sub);
+ }
+ }
+ }
+ }
+
return TryToNarrowDeduceFlags();
}
diff --git a/llvm/test/Transforms/InstCombine/sub-sext-add.ll b/llvm/test/Transforms/InstCombine/sub-sext-add.ll
new file mode 100644
index 0000000000000..8b12acdf95ba5
--- /dev/null
+++ b/llvm/test/Transforms/InstCombine/sub-sext-add.ll
@@ -0,0 +1,89 @@
+; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
+; RUN: opt < %s -passes=instcombine -S | FileCheck %s
+
+define i64 @src_2add_2sext_sub(i32 %x, i32 %y, i32 %z) {
+; CHECK-LABEL: define i64 @src_2add_2sext_sub(
+; CHECK-SAME: i32 [[X:%.*]], i32 [[Y:%.*]], i32 [[Z:%.*]]) {
+; CHECK-NEXT: [[SEXT1:%.*]] = sext i32 [[Y]] to i64
+; CHECK-NEXT: [[SEXT2:%.*]] = sext i32 [[Z]] to i64
+; CHECK-NEXT: [[SUB:%.*]] = sub nsw i64 [[SEXT1]], [[SEXT2]]
+; CHECK-NEXT: ret i64 [[SUB]]
+;
+ %add1 = add nsw i32 %x, %y
+ %add2 = add nsw i32 %x, %z
+ %sext1 = sext i32 %add1 to i64
+ %sext2 = sext i32 %add2 to i64
+ %sub = sub i64 %sext1, %sext2
+ ret i64 %sub
+}
+
+define i64 @src_2add_2sext_sub_nsw(i32 %x, i32 %y, i32 %z) {
+; CHECK-LABEL: define i64 @src_2add_2sext_sub_nsw(
+; CHECK-SAME: i32 [[X:%.*]], i32 [[Y:%.*]], i32 [[Z:%.*]]) {
+; CHECK-NEXT: [[SEXT1:%.*]] = sext i32 [[Y]] to i64
+; CHECK-NEXT: [[SEXT2:%.*]] = sext i32 [[Z]] to i64
+; CHECK-NEXT: [[SUB:%.*]] = sub nsw i64 [[SEXT1]], [[SEXT2]]
+; CHECK-NEXT: ret i64 [[SUB]]
+;
+ %add1 = add nsw i32 %x, %y
+ %add2 = add nsw i32 %x, %z
+ %sext1 = sext i32 %add1 to i64
+ %sext2 = sext i32 %add2 to i64
+ %sub = sub nsw i64 %sext1, %sext2
+ ret i64 %sub
+}
+
+define i64 @src_2add_2sext_sub_nuw(i32 %x, i32 %y, i32 %z) {
+; CHECK-LABEL: define i64 @src_2add_2sext_sub_nuw(
+; CHECK-SAME: i32 [[X:%.*]], i32 [[Y:%.*]], i32 [[Z:%.*]]) {
+; CHECK-NEXT: [[TMP1:%.*]] = sext i32 [[Y]] to i64
+; CHECK-NEXT: [[TMP2:%.*]] = sext i32 [[Z]] to i64
+; CHECK-NEXT: [[SUB:%.*]] = sub nsw i64 [[TMP1]], [[TMP2]]
+; CHECK-NEXT: ret i64 [[SUB]]
+;
+ %add1 = add nsw i32 %x, %y
+ %add2 = add nsw i32 %x, %z
+ %sext1 = sext i32 %add1 to i64
+ %sext2 = sext i32 %add2 to i64
+ %sub = sub nuw i64 %sext1, %sext2
+ ret i64 %sub
+}
+
+define i64 @src_x_add_2sext_sub(i32 %x, i32 %y) {
+; CHECK-LABEL: define i64 @src_x_add_2sext_sub(
+; CHECK-SAME: i32 [[X:%.*]], i32 [[Y:%.*]]) {
+; CHECK-NEXT: [[SUB:%.*]] = sext i32 [[Y]] to i64
+; CHECK-NEXT: ret i64 [[SUB]]
+;
+ %add1 = add nsw i32 %x, %y
+ %sext1 = sext i32 %add1 to i64
+ %sext2 = sext i32 %x to i64
+ %sub = sub i64 %sext1, %sext2
+ ret i64 %sub
+}
+
+define i64 @src_x_add_2sext_sub_nsw(i32 %x, i32 %y) {
+; CHECK-LABEL: define i64 @src_x_add_2sext_sub_nsw(
+; CHECK-SAME: i32 [[X:%.*]], i32 [[Y:%.*]]) {
+; CHECK-NEXT: [[SUB:%.*]] = sext i32 [[Y]] to i64
+; CHECK-NEXT: ret i64 [[SUB]]
+;
+ %add1 = add nsw i32 %x, %y
+ %sext1 = sext i32 %add1 to i64
+ %sext2 = sext i32 %x to i64
+ %sub = sub nsw i64 %sext1, %sext2
+ ret i64 %sub
+}
+
+define i64 @src_x_add_2sext_sub_nuw(i32 %x, i32 %y) {
+; CHECK-LABEL: define i64 @src_x_add_2sext_sub_nuw(
+; CHECK-SAME: i32 [[X:%.*]], i32 [[Y:%.*]]) {
+; CHECK-NEXT: [[SUB:%.*]] = sext i32 [[Y]] to i64
+; CHECK-NEXT: ret i64 [[SUB]]
+;
+ %add1 = add nsw i32 %x, %y
+ %sext1 = sext i32 %add1 to i64
+ %sext2 = sext i32 %x to i64
+ %sub = sub nuw i64 %sext1, %sext2
+ ret i64 %sub
+}
|
topperc
reviewed
Jun 14, 2025
topperc
reviewed
Jun 14, 2025
topperc
reviewed
Jun 14, 2025
dtcxzyw
reviewed
Jun 14, 2025
dtcxzyw
reviewed
Jun 14, 2025
dtcxzyw
reviewed
Jun 16, 2025
| } | ||
|
|
||
| // (sub[ nsw] (sext (add nsw (X, Y)), sext (add nsw (X, Z)))) --> | ||
| // --> (sub[ nsw] (sext (Y), sext(Z))) |
There was a problem hiding this comment.
Suggested change
| // --> (sub[ nsw] (sext (Y), sext(Z))) | |
| // --> (sub[ nsw] (sext (Y), sext (Z))) |
Comment on lines
2841
to
2842
| /* HasNUW */ false, | ||
| /* HasNSW */ I.hasNoSignedWrap()); |
There was a problem hiding this comment.
Suggested change
| /* HasNUW */ false, | |
| /* HasNSW */ I.hasNoSignedWrap()); | |
| /*HasNUW=*/false, | |
| /*HasNSW=*/I.hasNoSignedWrap()); |
This was referenced Jun 16, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pattern can be often met in Flang generated LLVM IR,
for example, for the counts of the loops generated for array
expressions like:
a(x:x+y)ora(x+z:x+z)or their variations.In order to compute the loop count, Flang needs to subtract
the lower bound of the array slice from the upper bound
of the array slice. To avoid the sign wraps, it sign extends
the original values (that may be of any user data type)
to
i64.This peephole is really helpful in CPU2017/548.exchange2,
where we have multiple following statements like this:
While this is just a 2x3 iterations loop nest, LLVM cannot
figure it out, ending up vectorizing the inner loop really
hard (with a vector epilog and scalar remainder). This, in turn,
causes problems for LSR that ends up creating too many loop-carried
values in the loop containing the above statement, which are then
causing too many spills/reloads.
Alive2: https://alive2.llvm.org/ce/z/gLgfYX
Related to #143219.