KnownBits: refine high-bits of mul in signed case #113051
Open
+163
−12
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
KnownBits::mul suffers from the deficiency that it doesn't account for signed inputs. Fix it by refining known leading zeros when both inputs are signed, and setting known leading ones when one of the inputs is signed. The strategy we've used is to still use umul_ov, after adjusting for signed inputs, and setting known leading ones from the negation of the result, when it is known to be negative, noting that a possibly-zero result is a special case.
|
@llvm/pr-subscribers-llvm-support Author: Ramkumar Ramachandra (artagnon) ChangesKnownBits::mul suffers from the deficiency that it doesn't account for signed inputs. Fix it by refining known leading zeros when both inputs are signed, and setting known leading ones when one of the inputs is signed. The strategy we've used is to still use umul_ov, after adjusting for signed inputs, and setting known leading ones from the negation of the result, when it is known to be negative, noting that a possibly-zero result is a special case. -- 8< -- Full diff: https://github.com/llvm/llvm-project/pull/113051.diff 2 Files Affected:
diff --git a/llvm/lib/Support/KnownBits.cpp b/llvm/lib/Support/KnownBits.cpp
index 89668af378070b..b63945f202a34d 100644
--- a/llvm/lib/Support/KnownBits.cpp
+++ b/llvm/lib/Support/KnownBits.cpp
@@ -796,19 +796,26 @@ KnownBits KnownBits::mul(const KnownBits &LHS, const KnownBits &RHS,
assert((!NoUndefSelfMultiply || LHS == RHS) &&
"Self multiplication knownbits mismatch");
- // Compute the high known-0 bits by multiplying the unsigned max of each side.
- // Conservatively, M active bits * N active bits results in M + N bits in the
- // result. But if we know a value is a power-of-2 for example, then this
- // computes one more leading zero.
- // TODO: This could be generalized to number of sign bits (negative numbers).
- APInt UMaxLHS = LHS.getMaxValue();
- APInt UMaxRHS = RHS.getMaxValue();
-
- // For leading zeros in the result to be valid, the unsigned max product must
+ // Compute the high known-0 or known-1 bits by multiplying the max of each
+ // side. Conservatively, M active bits * N active bits results in M + N bits
+ // in the result. But if we know a value is a power-of-2 for example, then
+ // this computes one more leading zero or one.
+ APInt MaxLHS = LHS.isNegative() ? LHS.getMinValue().abs() : LHS.getMaxValue(),
+ MaxRHS = RHS.isNegative() ? RHS.getMinValue().abs() : RHS.getMaxValue();
+
+ // For leading zeros or ones in the result to be valid, the max product must
// fit in the bitwidth (it must not overflow).
bool HasOverflow;
- APInt UMaxResult = UMaxLHS.umul_ov(UMaxRHS, HasOverflow);
- unsigned LeadZ = HasOverflow ? 0 : UMaxResult.countl_zero();
+ APInt Result = MaxLHS.umul_ov(MaxRHS, HasOverflow);
+ bool NegResult = LHS.isNegative() ^ RHS.isNegative();
+ unsigned LeadZ = 0, LeadO = 0;
+ if (!HasOverflow) {
+ // Do not set leading ones unless the result is known to be non-zero.
+ if (NegResult && LHS.isNonZero() && RHS.isNonZero())
+ LeadO = (-Result).countLeadingOnes();
+ else if (!NegResult)
+ LeadZ = Result.countLeadingZeros();
+ }
// The result of the bottom bits of an integer multiply can be
// inferred by looking at the bottom bits of both operands and
@@ -873,8 +880,9 @@ KnownBits KnownBits::mul(const KnownBits &LHS, const KnownBits &RHS,
KnownBits Res(BitWidth);
Res.Zero.setHighBits(LeadZ);
+ Res.One.setHighBits(LeadO);
Res.Zero |= (~BottomKnown).getLoBits(ResultBitsKnown);
- Res.One = BottomKnown.getLoBits(ResultBitsKnown);
+ Res.One |= BottomKnown.getLoBits(ResultBitsKnown);
// If we're self-multiplying then bit[1] is guaranteed to be zero.
if (NoUndefSelfMultiply && BitWidth > 1) {
diff --git a/llvm/test/Analysis/ValueTracking/knownbits-mul.ll b/llvm/test/Analysis/ValueTracking/knownbits-mul.ll
new file mode 100644
index 00000000000000..37526c67f0d9e1
--- /dev/null
+++ b/llvm/test/Analysis/ValueTracking/knownbits-mul.ll
@@ -0,0 +1,143 @@
+; NOTE: Assertions have been autogenerated by utils/update_test_checks.py UTC_ARGS: --version 5
+; RUN: opt < %s -passes=instcombine -S | FileCheck %s
+
+define i8 @mul_low_bits_know(i8 %xx, i8 %yy) {
+; CHECK-LABEL: define i8 @mul_low_bits_know(
+; CHECK-SAME: i8 [[XX:%.*]], i8 [[YY:%.*]]) {
+; CHECK-NEXT: ret i8 0
+;
+ %x = and i8 %xx, 2
+ %y = and i8 %yy, 4
+ %mul = mul i8 %x, %y
+ %r = and i8 %mul, 6
+ ret i8 %r
+}
+
+define i8 @mul_low_bits_know2(i8 %xx, i8 %yy) {
+; CHECK-LABEL: define i8 @mul_low_bits_know2(
+; CHECK-SAME: i8 [[XX:%.*]], i8 [[YY:%.*]]) {
+; CHECK-NEXT: ret i8 0
+;
+ %x = or i8 %xx, -2
+ %y = and i8 %yy, 4
+ %mul = mul i8 %x, %y
+ %r = and i8 %mul, 2
+ ret i8 %r
+}
+
+define i8 @mul_low_bits_partially_known(i8 %xx, i8 %yy) {
+; CHECK-LABEL: define i8 @mul_low_bits_partially_known(
+; CHECK-SAME: i8 [[XX:%.*]], i8 [[YY:%.*]]) {
+; CHECK-NEXT: [[Y:%.*]] = or i8 [[YY]], 2
+; CHECK-NEXT: [[MUL:%.*]] = sub nsw i8 0, [[Y]]
+; CHECK-NEXT: [[R:%.*]] = and i8 [[MUL]], 2
+; CHECK-NEXT: ret i8 [[R]]
+;
+ %x = or i8 %xx, -4
+ %x.notsmin = or i8 %x, 3
+ %y = or i8 %yy, -2
+ %mul = mul i8 %x.notsmin, %y
+ %r = and i8 %mul, 6
+ ret i8 %r
+}
+
+define i8 @mul_low_bits_unknown(i8 %xx, i8 %yy) {
+; CHECK-LABEL: define i8 @mul_low_bits_unknown(
+; CHECK-SAME: i8 [[XX:%.*]], i8 [[YY:%.*]]) {
+; CHECK-NEXT: [[X:%.*]] = or i8 [[XX]], 4
+; CHECK-NEXT: [[Y:%.*]] = or i8 [[YY]], 6
+; CHECK-NEXT: [[MUL:%.*]] = mul i8 [[X]], [[Y]]
+; CHECK-NEXT: [[R:%.*]] = and i8 [[MUL]], 6
+; CHECK-NEXT: ret i8 [[R]]
+;
+ %x = or i8 %xx, -4
+ %y = or i8 %yy, -2
+ %mul = mul i8 %x, %y
+ %r = and i8 %mul, 6
+ ret i8 %r
+}
+
+define i8 @mul_high_bits_know(i8 %xx, i8 %yy) {
+; CHECK-LABEL: define i8 @mul_high_bits_know(
+; CHECK-SAME: i8 [[XX:%.*]], i8 [[YY:%.*]]) {
+; CHECK-NEXT: ret i8 0
+;
+ %x = and i8 %xx, 2
+ %y = and i8 %yy, 4
+ %mul = mul i8 %x, %y
+ %r = and i8 %mul, 16
+ ret i8 %r
+}
+
+define i8 @mul_high_bits_know2(i8 %xx, i8 %yy) {
+; CHECK-LABEL: define i8 @mul_high_bits_know2(
+; CHECK-SAME: i8 [[XX:%.*]], i8 [[YY:%.*]]) {
+; CHECK-NEXT: ret i8 -16
+;
+ %x = or i8 %xx, -2
+ %y = and i8 %yy, 4
+ %y.nonzero = or i8 %y, 1
+ %mul = mul i8 %x, %y.nonzero
+ %r = and i8 %mul, -16
+ ret i8 %r
+}
+
+define i8 @mul_high_bits_know3(i8 %xx, i8 %yy) {
+; CHECK-LABEL: define i8 @mul_high_bits_know3(
+; CHECK-SAME: i8 [[XX:%.*]], i8 [[YY:%.*]]) {
+; CHECK-NEXT: ret i8 0
+;
+ %x = or i8 %xx, -4
+ %y = or i8 %yy, -2
+ %mul = mul i8 %x, %y
+ %r = and i8 %mul, -16
+ ret i8 %r
+}
+
+define i8 @mul_high_bits_unknown(i8 %xx, i8 %yy) {
+; CHECK-LABEL: define i8 @mul_high_bits_unknown(
+; CHECK-SAME: i8 [[XX:%.*]], i8 [[YY:%.*]]) {
+; CHECK-NEXT: [[X:%.*]] = and i8 [[XX]], 2
+; CHECK-NEXT: [[Y:%.*]] = and i8 [[YY]], 4
+; CHECK-NEXT: [[MUL:%.*]] = mul nuw nsw i8 [[X]], [[Y]]
+; CHECK-NEXT: ret i8 [[MUL]]
+;
+ %x = and i8 %xx, 2
+ %y = and i8 %yy, 4
+ %mul = mul i8 %x, %y
+ %r = and i8 %mul, 8
+ ret i8 %r
+}
+
+define i8 @mul_high_bits_unknown2(i8 %xx, i8 %yy) {
+; CHECK-LABEL: define i8 @mul_high_bits_unknown2(
+; CHECK-SAME: i8 [[XX:%.*]], i8 [[YY:%.*]]) {
+; CHECK-NEXT: [[X:%.*]] = or i8 [[XX]], -2
+; CHECK-NEXT: [[Y:%.*]] = and i8 [[YY]], 4
+; CHECK-NEXT: [[MUL:%.*]] = mul nsw i8 [[X]], [[Y]]
+; CHECK-NEXT: [[R:%.*]] = and i8 [[MUL]], -16
+; CHECK-NEXT: ret i8 [[R]]
+;
+ %x = or i8 %xx, -2
+ %y = and i8 %yy, 4
+ %mul = mul i8 %x, %y
+ %r = and i8 %mul, -16
+ ret i8 %r
+}
+
+; TODO: This can be reduced to zero.
+define i8 @mul_high_bits_unknown3(i8 %xx, i8 %yy) {
+; CHECK-LABEL: define i8 @mul_high_bits_unknown3(
+; CHECK-SAME: i8 [[XX:%.*]], i8 [[YY:%.*]]) {
+; CHECK-NEXT: [[X:%.*]] = or i8 [[XX]], 28
+; CHECK-NEXT: [[Y:%.*]] = or i8 [[YY]], 30
+; CHECK-NEXT: [[MUL:%.*]] = mul i8 [[X]], [[Y]]
+; CHECK-NEXT: [[R:%.*]] = and i8 [[MUL]], 16
+; CHECK-NEXT: ret i8 [[R]]
+;
+ %x = or i8 %xx, -4
+ %y = or i8 %yy, -2
+ %mul = mul i8 %x, %y
+ %r = and i8 %mul, 16
+ ret i8 %r
+}
|
|
@llvm/pr-subscribers-llvm-analysis Author: Ramkumar Ramachandra (artagnon) ChangesKnownBits::mul suffers from the deficiency that it doesn't account for signed inputs. Fix it by refining known leading zeros when both inputs are signed, and setting known leading ones when one of the inputs is signed. The strategy we've used is to still use umul_ov, after adjusting for signed inputs, and setting known leading ones from the negation of the result, when it is known to be negative, noting that a possibly-zero result is a special case. -- 8< -- Full diff: https://github.com/llvm/llvm-project/pull/113051.diff 2 Files Affected:
diff --git a/llvm/lib/Support/KnownBits.cpp b/llvm/lib/Support/KnownBits.cpp
index 89668af378070b..b63945f202a34d 100644
--- a/llvm/lib/Support/KnownBits.cpp
+++ b/llvm/lib/Support/KnownBits.cpp
@@ -796,19 +796,26 @@ KnownBits KnownBits::mul(const KnownBits &LHS, const KnownBits &RHS,
assert((!NoUndefSelfMultiply || LHS == RHS) &&
"Self multiplication knownbits mismatch");
- // Compute the high known-0 bits by multiplying the unsigned max of each side.
- // Conservatively, M active bits * N active bits results in M + N bits in the
- // result. But if we know a value is a power-of-2 for example, then this
- // computes one more leading zero.
- // TODO: This could be generalized to number of sign bits (negative numbers).
- APInt UMaxLHS = LHS.getMaxValue();
- APInt UMaxRHS = RHS.getMaxValue();
-
- // For leading zeros in the result to be valid, the unsigned max product must
+ // Compute the high known-0 or known-1 bits by multiplying the max of each
+ // side. Conservatively, M active bits * N active bits results in M + N bits
+ // in the result. But if we know a value is a power-of-2 for example, then
+ // this computes one more leading zero or one.
+ APInt MaxLHS = LHS.isNegative() ? LHS.getMinValue().abs() : LHS.getMaxValue(),
+ MaxRHS = RHS.isNegative() ? RHS.getMinValue().abs() : RHS.getMaxValue();
+
+ // For leading zeros or ones in the result to be valid, the max product must
// fit in the bitwidth (it must not overflow).
bool HasOverflow;
- APInt UMaxResult = UMaxLHS.umul_ov(UMaxRHS, HasOverflow);
- unsigned LeadZ = HasOverflow ? 0 : UMaxResult.countl_zero();
+ APInt Result = MaxLHS.umul_ov(MaxRHS, HasOverflow);
+ bool NegResult = LHS.isNegative() ^ RHS.isNegative();
+ unsigned LeadZ = 0, LeadO = 0;
+ if (!HasOverflow) {
+ // Do not set leading ones unless the result is known to be non-zero.
+ if (NegResult && LHS.isNonZero() && RHS.isNonZero())
+ LeadO = (-Result).countLeadingOnes();
+ else if (!NegResult)
+ LeadZ = Result.countLeadingZeros();
+ }
// The result of the bottom bits of an integer multiply can be
// inferred by looking at the bottom bits of both operands and
@@ -873,8 +880,9 @@ KnownBits KnownBits::mul(const KnownBits &LHS, const KnownBits &RHS,
KnownBits Res(BitWidth);
Res.Zero.setHighBits(LeadZ);
+ Res.One.setHighBits(LeadO);
Res.Zero |= (~BottomKnown).getLoBits(ResultBitsKnown);
- Res.One = BottomKnown.getLoBits(ResultBitsKnown);
+ Res.One |= BottomKnown.getLoBits(ResultBitsKnown);
// If we're self-multiplying then bit[1] is guaranteed to be zero.
if (NoUndefSelfMultiply && BitWidth > 1) {
diff --git a/llvm/test/Analysis/ValueTracking/knownbits-mul.ll b/llvm/test/Analysis/ValueTracking/knownbits-mul.ll
new file mode 100644
index 00000000000000..37526c67f0d9e1
--- /dev/null
+++ b/llvm/test/Analysis/ValueTracking/knownbits-mul.ll
@@ -0,0 +1,143 @@
+; NOTE: Assertions have been autogenerated by utils/update_test_checks.py UTC_ARGS: --version 5
+; RUN: opt < %s -passes=instcombine -S | FileCheck %s
+
+define i8 @mul_low_bits_know(i8 %xx, i8 %yy) {
+; CHECK-LABEL: define i8 @mul_low_bits_know(
+; CHECK-SAME: i8 [[XX:%.*]], i8 [[YY:%.*]]) {
+; CHECK-NEXT: ret i8 0
+;
+ %x = and i8 %xx, 2
+ %y = and i8 %yy, 4
+ %mul = mul i8 %x, %y
+ %r = and i8 %mul, 6
+ ret i8 %r
+}
+
+define i8 @mul_low_bits_know2(i8 %xx, i8 %yy) {
+; CHECK-LABEL: define i8 @mul_low_bits_know2(
+; CHECK-SAME: i8 [[XX:%.*]], i8 [[YY:%.*]]) {
+; CHECK-NEXT: ret i8 0
+;
+ %x = or i8 %xx, -2
+ %y = and i8 %yy, 4
+ %mul = mul i8 %x, %y
+ %r = and i8 %mul, 2
+ ret i8 %r
+}
+
+define i8 @mul_low_bits_partially_known(i8 %xx, i8 %yy) {
+; CHECK-LABEL: define i8 @mul_low_bits_partially_known(
+; CHECK-SAME: i8 [[XX:%.*]], i8 [[YY:%.*]]) {
+; CHECK-NEXT: [[Y:%.*]] = or i8 [[YY]], 2
+; CHECK-NEXT: [[MUL:%.*]] = sub nsw i8 0, [[Y]]
+; CHECK-NEXT: [[R:%.*]] = and i8 [[MUL]], 2
+; CHECK-NEXT: ret i8 [[R]]
+;
+ %x = or i8 %xx, -4
+ %x.notsmin = or i8 %x, 3
+ %y = or i8 %yy, -2
+ %mul = mul i8 %x.notsmin, %y
+ %r = and i8 %mul, 6
+ ret i8 %r
+}
+
+define i8 @mul_low_bits_unknown(i8 %xx, i8 %yy) {
+; CHECK-LABEL: define i8 @mul_low_bits_unknown(
+; CHECK-SAME: i8 [[XX:%.*]], i8 [[YY:%.*]]) {
+; CHECK-NEXT: [[X:%.*]] = or i8 [[XX]], 4
+; CHECK-NEXT: [[Y:%.*]] = or i8 [[YY]], 6
+; CHECK-NEXT: [[MUL:%.*]] = mul i8 [[X]], [[Y]]
+; CHECK-NEXT: [[R:%.*]] = and i8 [[MUL]], 6
+; CHECK-NEXT: ret i8 [[R]]
+;
+ %x = or i8 %xx, -4
+ %y = or i8 %yy, -2
+ %mul = mul i8 %x, %y
+ %r = and i8 %mul, 6
+ ret i8 %r
+}
+
+define i8 @mul_high_bits_know(i8 %xx, i8 %yy) {
+; CHECK-LABEL: define i8 @mul_high_bits_know(
+; CHECK-SAME: i8 [[XX:%.*]], i8 [[YY:%.*]]) {
+; CHECK-NEXT: ret i8 0
+;
+ %x = and i8 %xx, 2
+ %y = and i8 %yy, 4
+ %mul = mul i8 %x, %y
+ %r = and i8 %mul, 16
+ ret i8 %r
+}
+
+define i8 @mul_high_bits_know2(i8 %xx, i8 %yy) {
+; CHECK-LABEL: define i8 @mul_high_bits_know2(
+; CHECK-SAME: i8 [[XX:%.*]], i8 [[YY:%.*]]) {
+; CHECK-NEXT: ret i8 -16
+;
+ %x = or i8 %xx, -2
+ %y = and i8 %yy, 4
+ %y.nonzero = or i8 %y, 1
+ %mul = mul i8 %x, %y.nonzero
+ %r = and i8 %mul, -16
+ ret i8 %r
+}
+
+define i8 @mul_high_bits_know3(i8 %xx, i8 %yy) {
+; CHECK-LABEL: define i8 @mul_high_bits_know3(
+; CHECK-SAME: i8 [[XX:%.*]], i8 [[YY:%.*]]) {
+; CHECK-NEXT: ret i8 0
+;
+ %x = or i8 %xx, -4
+ %y = or i8 %yy, -2
+ %mul = mul i8 %x, %y
+ %r = and i8 %mul, -16
+ ret i8 %r
+}
+
+define i8 @mul_high_bits_unknown(i8 %xx, i8 %yy) {
+; CHECK-LABEL: define i8 @mul_high_bits_unknown(
+; CHECK-SAME: i8 [[XX:%.*]], i8 [[YY:%.*]]) {
+; CHECK-NEXT: [[X:%.*]] = and i8 [[XX]], 2
+; CHECK-NEXT: [[Y:%.*]] = and i8 [[YY]], 4
+; CHECK-NEXT: [[MUL:%.*]] = mul nuw nsw i8 [[X]], [[Y]]
+; CHECK-NEXT: ret i8 [[MUL]]
+;
+ %x = and i8 %xx, 2
+ %y = and i8 %yy, 4
+ %mul = mul i8 %x, %y
+ %r = and i8 %mul, 8
+ ret i8 %r
+}
+
+define i8 @mul_high_bits_unknown2(i8 %xx, i8 %yy) {
+; CHECK-LABEL: define i8 @mul_high_bits_unknown2(
+; CHECK-SAME: i8 [[XX:%.*]], i8 [[YY:%.*]]) {
+; CHECK-NEXT: [[X:%.*]] = or i8 [[XX]], -2
+; CHECK-NEXT: [[Y:%.*]] = and i8 [[YY]], 4
+; CHECK-NEXT: [[MUL:%.*]] = mul nsw i8 [[X]], [[Y]]
+; CHECK-NEXT: [[R:%.*]] = and i8 [[MUL]], -16
+; CHECK-NEXT: ret i8 [[R]]
+;
+ %x = or i8 %xx, -2
+ %y = and i8 %yy, 4
+ %mul = mul i8 %x, %y
+ %r = and i8 %mul, -16
+ ret i8 %r
+}
+
+; TODO: This can be reduced to zero.
+define i8 @mul_high_bits_unknown3(i8 %xx, i8 %yy) {
+; CHECK-LABEL: define i8 @mul_high_bits_unknown3(
+; CHECK-SAME: i8 [[XX:%.*]], i8 [[YY:%.*]]) {
+; CHECK-NEXT: [[X:%.*]] = or i8 [[XX]], 28
+; CHECK-NEXT: [[Y:%.*]] = or i8 [[YY]], 30
+; CHECK-NEXT: [[MUL:%.*]] = mul i8 [[X]], [[Y]]
+; CHECK-NEXT: [[R:%.*]] = and i8 [[MUL]], 16
+; CHECK-NEXT: ret i8 [[R]]
+;
+ %x = or i8 %xx, -4
+ %y = or i8 %yy, -2
+ %mul = mul i8 %x, %y
+ %r = and i8 %mul, 16
+ ret i8 %r
+}
|
|
How does this compare to Jay's impl #86671 (comment)? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
KnownBits::mul suffers from the deficiency that it doesn't account for signed inputs. Fix it by refining known leading zeros when both inputs are signed, and setting known leading ones when one of the inputs is signed. The strategy we've used is to still use umul_ov, after adjusting for signed inputs, and setting known leading ones from the negation of the result, when it is known to be negative, noting that a possibly-zero result is a special case.
-- 8< --
Based on #113050.