Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactored node builder flags and tests #59440

Merged
merged 11 commits into from
Aug 8, 2024

Conversation

armanio123
Copy link
Member

@armanio123 armanio123 commented Jul 27, 2024

The NodeBuilderFlags enum has shifted more than the allowed signed int32 limit.

This refactors experiments with creating a type and constant using BigInt instead. This approach was discarded becuase of the performance of BigInt, and the limitations the type has.

Added an additional flag InternalNodeBuilderFlags, that releases a couple of bits on NodeBuilderFlag by moving all the internal flags.

Reference to this PR #58475.

@typescript-bot typescript-bot added Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug labels Jul 27, 2024
@typescript-bot
Copy link
Collaborator

Looks like you're introducing a change to the public API surface area. If this includes breaking changes, please document them on our wiki's API Breaking Changes page.

Also, please make sure @DanielRosenwasser and @RyanCavanaugh are aware of the changes, just as a heads up.

Armando Aguirre Sepulveda added 2 commits July 26, 2024 18:28
@armanio123
Copy link
Member Author

@typescript-bot test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jul 27, 2024

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
test top400 ✅ Started ✅ Results
user test this ✅ Started ✅ Results
run dt ✅ Started ✅ Results
perf test this faster ✅ Started 👀 Results

@typescript-bot
Copy link
Collaborator

@armanio123 Here are the results of running the user tests with tsc comparing main and refs/pull/59440/merge:

Everything looks good!

@typescript-bot
Copy link
Collaborator

Hey @armanio123, the results of running the DT tests are ready.

Everything looks the same!

You can check the log here.

@typescript-bot
Copy link
Collaborator

@armanio123
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - node (v18.15.0, x64)
Errors 30 30 ~ ~ ~ p=1.000 n=6
Symbols 62,153 62,153 ~ ~ ~ p=1.000 n=6
Types 50,242 50,242 ~ ~ ~ p=1.000 n=6
Memory used 192,432k (± 0.09%) 193,734k (± 0.96%) +1,302k (+ 0.68%) 192,386k 196,190k p=0.030 n=6
Parse Time 1.32s (± 0.75%) 1.31s (± 0.92%) ~ 1.29s 1.32s p=0.391 n=6
Bind Time 0.72s (± 0.72%) 0.71s (± 0.77%) ~ 0.71s 0.72s p=0.640 n=6
Check Time 9.59s (± 0.44%) 9.62s (± 0.33%) ~ 9.58s 9.67s p=0.172 n=6
Emit Time 2.78s (± 0.48%) 2.78s (± 0.67%) ~ 2.75s 2.80s p=0.617 n=6
Total Time 14.41s (± 0.32%) 14.43s (± 0.31%) ~ 14.39s 14.51s p=0.418 n=6
angular-1 - node (v18.15.0, x64)
Errors 7 7 ~ ~ ~ p=1.000 n=6
Symbols 945,532 945,532 ~ ~ ~ p=1.000 n=6
Types 409,507 409,507 ~ ~ ~ p=1.000 n=6
Memory used 1,221,163k (± 0.00%) 1,221,181k (± 0.00%) ~ 1,221,104k 1,221,215k p=0.230 n=6
Parse Time 6.57s (± 0.25%) 6.64s (± 0.87%) +0.07s (+ 1.01%) 6.59s 6.75s p=0.008 n=6
Bind Time 1.85s (± 0.22%) 1.86s (± 0.28%) ~ 1.85s 1.86s p=0.112 n=6
Check Time 31.05s (± 0.35%) 31.07s (± 0.37%) ~ 30.97s 31.28s p=0.873 n=6
Emit Time 15.02s (± 0.50%) 14.99s (± 0.46%) ~ 14.87s 15.06s p=0.748 n=6
Total Time 54.49s (± 0.29%) 54.56s (± 0.24%) ~ 54.46s 54.81s p=1.000 n=6
mui-docs - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,437,253 2,437,253 ~ ~ ~ p=1.000 n=6
Types 1,004,097 1,004,097 ~ ~ ~ p=1.000 n=6
Memory used 2,426,284k (± 0.01%) 2,426,249k (± 0.00%) ~ 2,426,159k 2,426,374k p=0.378 n=6
Parse Time 8.50s (± 0.51%) 8.45s (± 0.14%) -0.06s (- 0.67%) 8.43s 8.46s p=0.005 n=6
Bind Time 2.11s (± 0.57%) 2.11s (± 0.65%) ~ 2.09s 2.12s p=0.339 n=6
Check Time 75.67s (± 1.40%) 75.11s (± 0.33%) ~ 74.60s 75.27s p=0.575 n=6
Emit Time 0.28s (± 1.86%) 0.28s (± 1.86%) ~ 0.27s 0.28s p=1.000 n=6
Total Time 86.56s (± 1.22%) 85.94s (± 0.28%) ~ 85.45s 86.10s p=0.230 n=6
self-build-src - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,227,758 1,227,821 +63 (+ 0.01%) ~ ~ p=0.001 n=6
Types 265,136 265,006 -130 (- 0.05%) ~ ~ p=0.001 n=6
Memory used 2,346,580k (± 0.03%) 2,347,526k (± 0.02%) +946k (+ 0.04%) 2,346,927k 2,348,125k p=0.031 n=6
Parse Time 4.96s (± 0.76%) 5.03s (± 0.65%) +0.06s (+ 1.28%) 4.99s 5.08s p=0.036 n=6
Bind Time 1.90s (± 0.61%) 1.89s (± 0.40%) ~ 1.88s 1.90s p=0.150 n=6
Check Time 34.61s (± 0.36%) 34.53s (± 0.43%) ~ 34.27s 34.66s p=0.229 n=6
Emit Time 3.30s (± 0.85%) 3.34s (± 0.86%) ~ 3.32s 3.38s p=0.127 n=6
Total Time 44.79s (± 0.33%) 44.80s (± 0.31%) ~ 44.56s 44.94s p=0.810 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,227,758 1,227,821 +63 (+ 0.01%) ~ ~ p=0.001 n=6
Types 265,136 265,006 -130 (- 0.05%) ~ ~ p=0.001 n=6
Memory used 2,420,479k (± 0.04%) 2,420,875k (± 0.01%) ~ 2,420,517k 2,421,353k p=0.230 n=6
Parse Time 6.25s (± 1.19%) 6.19s (± 0.69%) ~ 6.15s 6.26s p=0.128 n=6
Bind Time 2.03s (± 0.40%) 2.07s (± 1.25%) +0.04s (+ 1.97%) 2.04s 2.11s p=0.008 n=6
Check Time 41.17s (± 0.76%) 41.16s (± 0.38%) ~ 41.02s 41.42s p=0.575 n=6
Emit Time 4.05s (± 3.63%) 4.06s (± 1.03%) ~ 4.01s 4.11s p=0.471 n=6
Total Time 53.53s (± 0.73%) 53.50s (± 0.25%) ~ 53.37s 53.70s p=0.936 n=6
self-compiler - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 255,951 255,989 +38 (+ 0.01%) ~ ~ p=0.001 n=6
Types 104,957 104,970 +13 (+ 0.01%) ~ ~ p=0.001 n=6
Memory used 427,801k (± 0.02%) 428,082k (± 0.05%) +281k (+ 0.07%) 427,908k 428,468k p=0.005 n=6
Parse Time 3.35s (± 0.65%) 3.35s (± 0.41%) ~ 3.34s 3.38s p=0.413 n=6
Bind Time 1.32s (± 1.31%) 1.31s (± 1.96%) ~ 1.27s 1.33s p=0.683 n=6
Check Time 17.94s (± 0.16%) 18.06s (± 0.35%) +0.12s (+ 0.64%) 17.99s 18.15s p=0.005 n=6
Emit Time 1.65s (± 1.15%) 1.70s (± 1.53%) +0.04s (+ 2.73%) 1.66s 1.73s p=0.013 n=6
Total Time 24.26s (± 0.05%) 24.41s (± 0.37%) +0.16s (+ 0.64%) 24.33s 24.53s p=0.005 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors 35 35 ~ ~ ~ p=1.000 n=6
Symbols 224,931 224,931 ~ ~ ~ p=1.000 n=6
Types 94,146 94,146 ~ ~ ~ p=1.000 n=6
Memory used 370,180k (± 0.05%) 370,218k (± 0.03%) ~ 370,019k 370,326k p=1.000 n=6
Parse Time 2.77s (± 0.50%) 2.76s (± 0.81%) ~ 2.73s 2.79s p=0.618 n=6
Bind Time 1.58s (± 0.67%) 1.58s (± 0.62%) ~ 1.57s 1.59s p=0.315 n=6
Check Time 15.66s (± 0.45%) 15.63s (± 0.45%) ~ 15.57s 15.73s p=0.518 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 20.00s (± 0.36%) 19.97s (± 0.33%) ~ 19.89s 20.05s p=0.572 n=6
vscode - node (v18.15.0, x64)
Errors 11 11 ~ ~ ~ p=1.000 n=6
Symbols 2,986,839 2,986,839 ~ ~ ~ p=1.000 n=6
Types 1,027,468 1,027,468 ~ ~ ~ p=1.000 n=6
Memory used 3,110,683k (± 0.00%) 3,110,635k (± 0.00%) ~ 3,110,558k 3,110,714k p=0.230 n=6
Parse Time 13.87s (± 0.34%) 13.85s (± 0.55%) ~ 13.75s 13.94s p=0.574 n=6
Bind Time 4.32s (± 1.58%) 4.31s (± 2.01%) ~ 4.27s 4.49s p=0.167 n=6
Check Time 79.27s (± 0.40%) 79.26s (± 0.52%) ~ 78.86s 79.86s p=0.936 n=6
Emit Time 20.46s (± 0.34%) 20.42s (± 0.66%) ~ 20.29s 20.60s p=0.423 n=6
Total Time 117.93s (± 0.32%) 117.83s (± 0.38%) ~ 117.32s 118.34s p=0.689 n=6
webpack - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 267,567 267,567 ~ ~ ~ p=1.000 n=6
Types 109,077 109,077 ~ ~ ~ p=1.000 n=6
Memory used 412,412k (± 0.01%) 412,534k (± 0.02%) +122k (+ 0.03%) 412,485k 412,690k p=0.005 n=6
Parse Time 3.83s (± 0.74%) 3.82s (± 0.83%) ~ 3.78s 3.86s p=0.627 n=6
Bind Time 1.71s (± 0.32%) 1.71s (± 0.44%) ~ 1.70s 1.72s p=0.476 n=6
Check Time 16.83s (± 0.40%) 16.88s (± 0.32%) ~ 16.81s 16.96s p=0.228 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 22.38s (± 0.35%) 22.41s (± 0.28%) ~ 22.30s 22.47s p=0.377 n=6
xstate-main - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 511,807 511,807 ~ ~ ~ p=1.000 n=6
Types 162,090 162,090 ~ ~ ~ p=1.000 n=6
Memory used 449,159k (± 0.07%) 449,208k (± 0.09%) ~ 448,705k 449,530k p=0.471 n=6
Parse Time 3.16s (± 0.53%) 3.17s (± 1.01%) ~ 3.12s 3.21s p=0.572 n=6
Bind Time 1.16s (± 0.71%) 1.16s (± 0.54%) ~ 1.15s 1.17s p=0.599 n=6
Check Time 17.12s (± 0.54%) 17.17s (± 0.43%) ~ 17.09s 17.30s p=0.298 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 21.44s (± 0.42%) 21.50s (± 0.29%) ~ 21.45s 21.60s p=0.228 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@typescript-bot
Copy link
Collaborator

@armanio123 Here are the results of running the top 400 repos with tsc comparing main and refs/pull/59440/merge:

Everything looks good!

@jakebailey
Copy link
Member

#59282 gives back a node flag, can we not just wait for that?

src/compiler/types.ts Outdated Show resolved Hide resolved
@armanio123
Copy link
Member Author

#59282 gives back a node flag, can we not just wait for that?

I was planning on adding a new flag to it, that's how I noticed this issue. So even if we reduce the number of flags, we're going to be dealing with this limitation.

Armando Aguirre Sepulveda added 3 commits July 29, 2024 17:57
@armanio123
Copy link
Member Author

Refactored as suggested during the design meeting. There's a couple of tests that looks suspicious to me so I appreciate extra set of eyes on them.

@jakebailey
Copy link
Member

Yeah, I suspect there's a bug lurking in there; most of the baseline changes are unexpected.

@jakebailey
Copy link
Member

Skimming the code locally, though, I can't find the place where things may have gone wrong... It all seems correct to me.

@@ -222,7 +223,7 @@ export function addNewNodeForMemberSymbol(
case SyntaxKind.PropertyDeclaration:
let flags = NodeBuilderFlags.NoTruncation;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't need AllowUnresolvedNames like the code below?

Sort of wondering if all old users of NoTruncation need that flag, but maybe we shouldn't add ones where we don't think it should matter except those where we would have set it manually...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently is not needed to pass the test cases. Thought we might be missing some, because adding it doesn't change any of them.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, our test suite is just not always exhaustive, so I suspect that there are some lurking bugs without this flag for these auto-import / codefix cases, given those seem to in other cases depend on this flag

@@ -506,6 +507,7 @@ export function createSignatureDeclarationFromCallExpression(
contextNode,
scriptTarget,
NodeBuilderFlags.NoTruncation,
/*internalFlags*/ undefined,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes me wonder if we are missing a test with an unresolved name for those who call this code. (Specifically fixAddMissingMember)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, that's the same thing I just responded to your last comment.

Copy link
Member

@jakebailey jakebailey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall this now seems good, though the title needs to be updated to not say it's an experiment, and I do think we may need to be adding the unresolved name thing in a couple of places that maybe we just don't have tests for?

I think that the tests that fail without the change to createSignatureDeclarationFromSignature may need to be recreated as tests which make the other cases where NoTruncation failed also fail without AllowUnresolvedNames.

@armanio123 armanio123 changed the title Experiment: Refactored node builder flags and tests Refactored node builder flags and tests Aug 6, 2024
@armanio123
Copy link
Member Author

If I understand the AllowUnresolvedNames flag correctly this tests cases should not use any correct? If so, there's no reason to add the flag as we don't handle it.

interface A {
    foo: import("./doesntexist.js").Foo;
    get bar(): import("./doesntexist.js").Foo;
}

class B implements A {
| <-- Trigger completion here
}

Completion returns:

foo: any;           // This should have been  `foo: import("./doesntexist.js").Foo;`
get bar(): any {    // and this `get bar(): import("./doesntexist.js").Foo;`
    throw new Error("Method not implemented.");
}

@jakebailey
Copy link
Member

I would expect those to not spit out any, no, though I'm confused because it sounds like you're saying it spits out any when the flag is set?

@armanio123
Copy link
Member Author

armanio123 commented Aug 7, 2024

That's correct, we spit out any regardless if the flag is set or not. Most likely we don't handle the flag on this code path.

@jakebailey
Copy link
Member

If you don't want to write a test for those other branches, can we just put that flag back on the places that used to have NoTruncation to maintain the existing behavior?

@armanio123
Copy link
Member Author

Yeah, I was going to propose the same as I don't know how to create all of the necessary tests. I'm going to put the flag in everyplace I find NoTruncation.

@@ -5962,7 +5964,7 @@ export function createTypeChecker(host: TypeCheckerHost): TypeChecker {

function typeToString(type: Type, enclosingDeclaration?: Node, flags: TypeFormatFlags = TypeFormatFlags.AllowUniqueESSymbolType | TypeFormatFlags.UseAliasDefinedOutsideCurrentScope, writer: EmitTextWriter = createTextWriter("")): string {
const noTruncation = compilerOptions.noErrorTruncation || flags & TypeFormatFlags.NoTruncation;
const typeNode = nodeBuilder.typeToTypeNode(type, enclosingDeclaration, toNodeBuilderFlags(flags) | NodeBuilderFlags.IgnoreErrors | (noTruncation ? NodeBuilderFlags.NoTruncation : 0));
const typeNode = nodeBuilder.typeToTypeNode(type, enclosingDeclaration, toNodeBuilderFlags(flags) | NodeBuilderFlags.IgnoreErrors | (noTruncation ? NodeBuilderFlags.NoTruncation : NodeBuilderFlags.None), noTruncation ? InternalNodeBuilderFlags.AllowUnresolvedNames : InternalNodeBuilderFlags.None);
Copy link
Member Author

@armanio123 armanio123 Aug 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This scenario seems weird to me as it depends on NoTruncation being set in TypeFormatFlags. Nonetheless, this is the current behavior, so I decided to change it but let me know if you disagree and I can rollback.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, I think it's a little weird, though I guess it's also true that nobody would have ever intentionally set AllowUnresolvedNames here because that was never a valid TypeFormatFlag.

Sort of leaning towards not doing this one and seeing what happens? I assume no test changes here either.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They don't change. Actually, the latest commit doesn't change any of the tests. I'll rollback this line.

Copy link
Member

@jakebailey jakebailey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this seems alright now, though it would be nice if we could figure out the rest.

Given we just added a flag everywhere, curious what @weswigham thinks, but it should at least maintain our old behavior which I feel is pretty safe.

@armanio123 armanio123 merged commit 5f79e16 into microsoft:main Aug 8, 2024
32 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

4 participants