-
-
Notifications
You must be signed in to change notification settings - Fork 721
perf(linter/plugins): lazy deserialize comments array #14637
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
perf(linter/plugins): lazy deserialize comments array #14637
Conversation
How to use the Graphite Merge QueueAdd either label to this PR to merge it via the merge queue:
You must have a Graphite account in order to use the merge queue. Sign up using this link. An organization admin has enabled the Graphite Merge Queue in this repository. Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue. |
CodSpeed Performance ReportMerging #14637 will not alter performanceComparing Summary
|
42d51a6 to
09bc1a8
Compare
1831c50 to
cd62cfb
Compare
6ba5770 to
f20c6f9
Compare
|
Not sure what's up with AST check CI. The generated code is up to date. CI's expected output is just different. |
a2dcd3c to
7a7949d
Compare
Looks like The problem on this PR was that the autofix CI job ran Or just sidestep the formatting problem, by shortening the error message: if (localAstId !== astId) throw new Error('Comments are only accessible while linting the file');This is an edge case which shouldn't be hit in practice, so I don't think we need to provide too much explanation. |
|
I tried that as well, dprint introduces a block in that case. |
|
Weird. I'm not seeing that locally. Anyway, it doesn't matter if it produces a block, only that it does so consistently. |
919bb6a to
ed4acbf
Compare
overlookmotel
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
All nits...
757c717 to
18d50de
Compare
|
Addressed the review comments in 09baebc. |
This works around `dprint`'s formatting bug.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
Lazy-loads the Program.comments array to reduce upfront deserialization cost, guarding access with an AST session id and restoring buffers on demand.
- Introduces a global astId to gate comment access to the active lint session.
- Saves buffer references (uint32, uint8, sourceText) and adds a lazy getter for comments that deserializes on first access.
Reviewed Changes
Copilot reviewed 2 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| tasks/ast_tools/src/generators/raw_transfer.rs | Adds astId initialization in the generated runtime to track the active AST session. |
| crates/oxc_ast/src/serialize/mod.rs | Implements lazy comments getter: captures buffer refs, increments astId per Program, restores buffers and deserializes comments on first access. |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
|
Thank you. I did warn you it might be tricky, but I underestimated! |
|
Thanks for guiding me through this. I did not imagine a project of oxc's scope would neat and tidy, and I enjoyed this welcome tour! |
Follow-on after #14637. Just a small stylistic thing - reorder a few lines.
…linted (#14727) Follow on after #14637. I now realize that there was a subtle race condition. We use `astId` counter in the getter for `program.comments`, to check that the getter is not called after that AST is done with. Buffers are reused, so the buffer held in local reference is the same, but the *contents* of the buffer could have changed - it may now contain a *different* AST. Previously, `astId` was incremented in `deserializeProgram`. So if `comments` getter is accessed after another AST is deserialized, it (rightly) throws. But there's a small period of time in between walking the AST on JS side and the *next* call to `deserializeProgram`. During that period, the buffer may be being mutated on Rust side (in another thread), but `astId` hasn't been incremented yet, so the `localAstId === astId` check in `comments` getter will pass, and it won't throw an error as it should. Fix this by incrementing `astId` as soon as traversal of *this* AST ends, instead of before deserializing the *next* AST begins.
We forgot to update this comment in #14637.
Part of #14564.
Deserialize
program.commentslazily upon first access, instead of eagerly, on assumption that most JS plugins won't access comments.deserializeWithcleans up the buffer once deserializing AST is complete, so we keep a local copy of the buffer in scope of thecommentsgetter, and restore it when the getter is triggered.Buffers are reused, so the
if (localAstId !== astId)check is required to make sure the AST in the buffer is same AST i.e. that the getter isn't accessed after the file has completed linting.