-
Notifications
You must be signed in to change notification settings - Fork 12.9k
Add control flow complexity limiter #30931
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
Closed
Changes from all commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
f9a99a0
Add control flow complexity limiter
ahejlsberg f623325
Improve error message
ahejlsberg 93b1c53
Accept new baselines
ahejlsberg c01064e
Record flow analysis errors in NodeLinks
ahejlsberg 7dcb3ca
Add regression test
ahejlsberg 85a3b2c
Accept new baselines
ahejlsberg File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
163 changes: 163 additions & 0 deletions
163
tests/baselines/reference/complexControlFlowGraph.errors.txt
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,163 @@ | ||
tests/cases/compiler/complexControlFlowGraph.ts(4,3): error TS2563: The containing function or module body is too complex for control flow analysis. | ||
|
||
|
||
==== tests/cases/compiler/complexControlFlowGraph.ts (1 errors) ==== | ||
// Repro from #29926 | ||
|
||
const foo = function (this: any) { | ||
var a, b, c, d, ab, bc, cd, da, blocks = this.blocks; | ||
~~~ | ||
!!! error TS2563: The containing function or module body is too complex for control flow analysis. | ||
|
||
if (this.first) { | ||
a = blocks[0] - 1; | ||
a = (a << 3) | (a >>> 29); | ||
d = ((a & 0xefcdab89) | (~a & 0x98badcfe)) + blocks[1] + 271733878; | ||
d = (d << 7) | (d >>> 25); | ||
c = ((d & a) | (~d & 0xefcdab89)) + blocks[2] - 1732584194; | ||
c = (c << 11) | (c >>> 21); | ||
b = ((c & d) | (~c & a)) + blocks[3] - 271733879; | ||
b = (b << 19) | (b >>> 13); | ||
} else { | ||
a = this.h0; | ||
b = this.h1; | ||
c = this.h2; | ||
d = this.h3; | ||
a += ((b & c) | (~b & d)) + blocks[0]; | ||
a = (a << 3) | (a >>> 29); | ||
d += ((a & b) | (~a & c)) + blocks[1]; | ||
d = (d << 7) | (d >>> 25); | ||
c += ((d & a) | (~d & b)) + blocks[2]; | ||
c = (c << 11) | (c >>> 21); | ||
b += ((c & d) | (~c & a)) + blocks[3]; | ||
b = (b << 19) | (b >>> 13); | ||
} | ||
|
||
a += ((b & c) | (~b & d)) + blocks[4]; | ||
a = (a << 3) | (a >>> 29); | ||
d += ((a & b) | (~a & c)) + blocks[5]; | ||
d = (d << 7) | (d >>> 25); | ||
c += ((d & a) | (~d & b)) + blocks[6]; | ||
c = (c << 11) | (c >>> 21); | ||
b += ((c & d) | (~c & a)) + blocks[7]; | ||
b = (b << 19) | (b >>> 13); | ||
a += ((b & c) | (~b & d)) + blocks[8]; | ||
a = (a << 3) | (a >>> 29); | ||
d += ((a & b) | (~a & c)) + blocks[9]; | ||
d = (d << 7) | (d >>> 25); | ||
c += ((d & a) | (~d & b)) + blocks[10]; | ||
c = (c << 11) | (c >>> 21); | ||
b += ((c & d) | (~c & a)) + blocks[11]; | ||
b = (b << 19) | (b >>> 13); | ||
a += ((b & c) | (~b & d)) + blocks[12]; | ||
a = (a << 3) | (a >>> 29); | ||
d += ((a & b) | (~a & c)) + blocks[13]; | ||
d = (d << 7) | (d >>> 25); | ||
c += ((d & a) | (~d & b)) + blocks[14]; | ||
c = (c << 11) | (c >>> 21); | ||
b += ((c & d) | (~c & a)) + blocks[15]; | ||
b = (b << 19) | (b >>> 13); | ||
|
||
bc = b & c; | ||
a += (bc | (b & d) | (c & d)) + blocks[0] + 1518500249; | ||
a = (a << 3) | (a >>> 29); | ||
ab = a & b; | ||
d += (ab | (a & c) | bc) + blocks[4] + 1518500249; | ||
d = (d << 5) | (d >>> 27); | ||
da = d & a; | ||
c += (da | (d & b) | ab) + blocks[8] + 1518500249; | ||
c = (c << 9) | (c >>> 23); | ||
cd = c & d; | ||
b += (cd | (c & a) | da) + blocks[12] + 1518500249; | ||
b = (b << 13) | (b >>> 19); | ||
bc = b & c; | ||
a += (bc | (b & d) | cd) + blocks[1] + 1518500249; | ||
a = (a << 3) | (a >>> 29); | ||
ab = a & b; | ||
d += (ab | (a & c) | bc) + blocks[5] + 1518500249; | ||
d = (d << 5) | (d >>> 27); | ||
da = d & a; | ||
c += (da | (d & b) | ab) + blocks[9] + 1518500249; | ||
c = (c << 9) | (c >>> 23); | ||
cd = c & d; | ||
b += (cd | (c & a) | da) + blocks[13] + 1518500249; | ||
b = (b << 13) | (b >>> 19); | ||
bc = b & c; | ||
a += (bc | (b & d) | cd) + blocks[2] + 1518500249; | ||
a = (a << 3) | (a >>> 29); | ||
ab = a & b; | ||
d += (ab | (a & c) | bc) + blocks[6] + 1518500249; | ||
d = (d << 5) | (d >>> 27); | ||
da = d & a; | ||
c += (da | (d & b) | ab) + blocks[10] + 1518500249; | ||
c = (c << 9) | (c >>> 23); | ||
cd = c & d; | ||
b += (cd | (c & a) | da) + blocks[14] + 1518500249; | ||
b = (b << 13) | (b >>> 19); | ||
bc = b & c; | ||
a += (bc | (b & d) | cd) + blocks[3] + 1518500249; | ||
a = (a << 3) | (a >>> 29); | ||
ab = a & b; | ||
d += (ab | (a & c) | bc) + blocks[7] + 1518500249; | ||
d = (d << 5) | (d >>> 27); | ||
da = d & a; | ||
c += (da | (d & b) | ab) + blocks[11] + 1518500249; | ||
c = (c << 9) | (c >>> 23); | ||
b += ((c & d) | (c & a) | da) + blocks[15] + 1518500249; | ||
b = (b << 13) | (b >>> 19); | ||
|
||
bc = b ^ c; | ||
a += (bc ^ d) + blocks[0] + 1859775393; | ||
a = (a << 3) | (a >>> 29); | ||
d += (bc ^ a) + blocks[8] + 1859775393; | ||
d = (d << 9) | (d >>> 23); | ||
da = d ^ a; | ||
c += (da ^ b) + blocks[4] + 1859775393; | ||
c = (c << 11) | (c >>> 21); | ||
b += (da ^ c) + blocks[12] + 1859775393; | ||
b = (b << 15) | (b >>> 17); | ||
bc = b ^ c; | ||
a += (bc ^ d) + blocks[2] + 1859775393; | ||
a = (a << 3) | (a >>> 29); | ||
d += (bc ^ a) + blocks[10] + 1859775393; | ||
d = (d << 9) | (d >>> 23); | ||
da = d ^ a; | ||
c += (da ^ b) + blocks[6] + 1859775393; | ||
c = (c << 11) | (c >>> 21); | ||
b += (da ^ c) + blocks[14] + 1859775393; | ||
b = (b << 15) | (b >>> 17); | ||
bc = b ^ c; | ||
a += (bc ^ d) + blocks[1] + 1859775393; | ||
a = (a << 3) | (a >>> 29); | ||
d += (bc ^ a) + blocks[9] + 1859775393; | ||
d = (d << 9) | (d >>> 23); | ||
da = d ^ a; | ||
c += (da ^ b) + blocks[5] + 1859775393; | ||
c = (c << 11) | (c >>> 21); | ||
b += (da ^ c) + blocks[13] + 1859775393; | ||
b = (b << 15) | (b >>> 17); | ||
bc = b ^ c; | ||
a += (bc ^ d) + blocks[3] + 1859775393; | ||
a = (a << 3) | (a >>> 29); | ||
d += (bc ^ a) + blocks[11] + 1859775393; | ||
d = (d << 9) | (d >>> 23); | ||
da = d ^ a; | ||
c += (da ^ b) + blocks[7] + 1859775393; | ||
c = (c << 11) | (c >>> 21); | ||
b += (da ^ c) + blocks[15] + 1859775393; | ||
b = (b << 15) | (b >>> 17); | ||
|
||
if (this.first) { | ||
this.h0 = a + 1732584193 << 0; | ||
this.h1 = b - 271733879 << 0; | ||
this.h2 = c - 1732584194 << 0; | ||
this.h3 = d + 271733878 << 0; | ||
this.first = false; | ||
} else { | ||
this.h0 = this.h0 + a << 0; | ||
this.h1 = this.h1 + b << 0; | ||
this.h2 = this.h2 + c << 0; | ||
this.h3 = this.h3 + d << 0; | ||
} | ||
}; | ||
|
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Am I crazy for thinking that this input code doesn't actually look that complicated and that we're probably doing too much work somewhere? Like, this is all binary arithmetic - the result is quite literally always a number - I'm not even sure why control flow comes into play.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
eeeyyyeaaahhh; In reusing
checkExpression
verbatim within control flow, we're missing any fast-path-type-caluclation-only type savings we could get on statements like these. Without even looking at the operands we could pretty easily assume they're of typenumber
. I mean, technically we can't straight up assume, thanks to bigints; but if a binary expression tree contains number literals the result is either an error or number, and likewise for bigint literals (and if both are in a binary expression tree then the result is definitely an error), and the error case should get flagged by the normal statement walk. We could probably get serious savings by skipping calculation of a ton of references in control flow with a simple syntactic walk like that, since it'd mean that for any binary arithmetic expression with numeric literals in it, we'd always be able to avoid following the reference chain.Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We could even flag such trees during binding (like we do for other subtree containment checks), so all we need to do is check a flag on the node during checking, hmmmmm...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We're not using
checkExpression
verbatim, we're usinggetTypeOfExpression
, the intent of which is to obtain the type of an expression with as little work as possible (withcheckExpression
being the fallback). We already optimize for non-generic call expressions with no overloads and assertion expressions, and we could certainly explore more optimizations.The particular input code that led to this PR may indeed have many operations for which we can obtain a type just from looking at literals and operators, and, with some work, we might even be able to process the code without hitting the limiter. But of course someone could write the example to use named constants instead, and we'd be right back where we started. So, one way or the other, we need an overall work limiter.
The thing that could really make a difference is better caching of intermediate types. But knowing when it is safe to cache is tricky. In particular, code with back edges from loops is hard.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This code contains zero loops (and zero generics and zero overloads), however, which is why this code, specifically, being problematic to us seems so odd to me. I definitely understand the desire to have some sort of graceful-failure-like hard cap on work done here, however I don't think this example is a good justification for it - the actual complexity of the given code is relatively low; it just currently triggers an unoptimized worst case (deep chains of apparently dependent statements with uncached checks on them). IMO, that itself merits some fixing, at which point this test won't be that useful for testing this limiter. :(
Although maybe this is also just a good example of why we should have a merge-able hierarchical cache; we call
getTypeOfExpression
with caching disabled in control flow because we don't know if we're in an inferential/contextual context or not, and caching control flow results in those contexts does bad things to the types we produce - if we could simply "always cache" and have call resolution merge/toss the cache layer as appropriate we'd probably not have this problem, at least in this case.Really, the heart of it is that while I believe the limiter is probably needed for more graceful failure modes, I don't think it's really a "fix" - error'ing on completely valid input code should be a last resort for us, yeah? To be used when resources are truly stretched thin? This is just a poignant example of something we need to optimize, especially when the input code, to a human like myself, seems so simple.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Per our discussion earlier, I've opened #30945 with an optimization that allows us to actually check this input code. With that, we'll have to find another example that actually triggers this limiter for the test to be useful (probably not hard - just strip out all the literals). ❤️
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
#31003 is now a much more complete fix with minimal to no perf impact on non-degenerate cases. ❤️