Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(transformer): transform explicit resource management #9310

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

camc314
Copy link
Contributor

@camc314 camc314 commented Feb 23, 2025

wanted to have a shot at implementing this, this is rough draft.

Dunqing or overlookmotel feel free to take over or close out this PR. not sure how much time i'll have to finish it off

@Dunqing @overlookmotel this should be ready when you guys have time 🙏

closes #9168

@github-actions github-actions bot added A-transformer Area - Transformer / Transpiler C-enhancement Category - New feature or request labels Feb 23, 2025
Copy link
Contributor Author

camc314 commented Feb 23, 2025


How to use the Graphite Merge Queue

Add either label to this PR to merge it via the merge queue:

  • 0-merge - adds this PR to the back of the merge queue
  • hotfix - for urgent hot fixes, skip the queue and merge this PR next

You must have a Graphite account in order to use the merge queue. Sign up using this link.

An organization admin has enabled the Graphite Merge Queue in this repository.

Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue.

This stack of pull requests is managed by Graphite. Learn more about stacking.

@camc314 camc314 marked this pull request as ready for review February 23, 2025 21:55
@camc314 camc314 marked this pull request as draft February 23, 2025 21:55
Copy link

codspeed-hq bot commented Feb 23, 2025

CodSpeed Performance Report

Merging #9310 will degrade performances by 5.43%

Comparing c/02-23-feat_transformer_transform_explicit_resource_management (beed50c) with main (4ca62ab)

Summary

❌ 3 regressions
✅ 36 untouched benchmarks

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

Benchmark BASE HEAD Change
transformer[antd.js] 53.3 ms 55 ms -3.15%
transformer[checker.ts] 24 ms 25.4 ms -5.43%
transformer[pdf.mjs] 10.6 ms 11.1 ms -4.42%

@camc314 camc314 force-pushed the c/02-23-feat_transformer_transform_explicit_resource_management branch 2 times, most recently from c4b4c55 to 0dd6726 Compare February 27, 2025 17:02
@camc314 camc314 force-pushed the c/02-23-feat_transformer_transform_explicit_resource_management branch 2 times, most recently from 6b624bd to 081e4d7 Compare February 27, 2025 17:43
@camc314 camc314 marked this pull request as ready for review February 27, 2025 17:43
@camc314 camc314 force-pushed the c/02-23-feat_transformer_transform_explicit_resource_management branch from 081e4d7 to 6716335 Compare February 28, 2025 09:30
@Dunqing
Copy link
Member

Dunqing commented Feb 28, 2025

Thank you for working on this, I will review this next week, before that, can you add some documentation like other plugins do at the top of file?

For example:

//! ES2020: Nullish Coalescing Operator
//!
//! This plugin transforms nullish coalescing operators (`??`) to a series of ternary expressions.
//!
//! > This plugin is included in `preset-env`, in ES2020
//!
//! ## Example
//!
//! Input:
//! ```js
//! var foo = object.foo ?? "default";
//! ```
//!
//! Output:
//! ```js
//! var _object$foo;
//! var foo =
//! (_object$foo = object.foo) !== null && _object$foo !== void 0
//! ? _object$foo
//! : "default";
//! ```
//!
//! ## Implementation
//!
//! Implementation based on [@babel/plugin-transform-nullish-coalescing-operator](https://babeljs.io/docs/babel-plugin-transform-nullish-coalescing-operator).
//!
//! ## References:
//! * Babel plugin implementation: <https://github.com/babel/babel/tree/v7.26.2/packages/babel-plugin-transform-nullish-coalescing-operator>
//! * Nullish coalescing TC39 proposal: <https://github.com/tc39-transfer/proposal-nullish-coalescing>

@camc314 camc314 force-pushed the c/02-23-feat_transformer_transform_explicit_resource_management branch from 6716335 to 305c2ab Compare February 28, 2025 10:59
@Dunqing Dunqing self-assigned this Mar 3, 2025
@camc314 camc314 force-pushed the c/02-23-feat_transformer_transform_explicit_resource_management branch from 305c2ab to 69a7a41 Compare March 3, 2025 13:36
Copy link
Contributor

@overlookmotel overlookmotel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks very much for tackling this. And bravo for getting all the tests to pass, including all the semantic IDs (don't worry about that 1 weird one).

I've made comments below about some small things.

The bigger thing is the performance hit. My assumption is it's due to looping over statements. Usually we try to avoid that by setting some state in enter_statements, then let the traversal run through the children with smaller visitors which update that state as they go, and then you check the state in exit_statements and act on it if there's work to be done. This avoids visiting every statement twice.

However, I'm not familiar with this transform, and found it hard to follow what the code here is doing, so can't judge how amenable it'd be to that approach. So I suggest we merge this, and then work on the performance in later PRs.

What would help a lot is if you'd be able to add more comments, so the logic is easier to follow. The AstBuilder calls are so verbose (bad API, sorry!) that it helps a lot to have a comment before a bunch of ctx.ast.blah_blah calls saying what the code it produces. e.g. add a comment before this:

// `var var_id = <expr>;`

inner_block.push(Statement::VariableDeclaration(ctx.ast.alloc(
ctx.ast.variable_declaration(
span,
VariableDeclarationKind::Var,
ctx.ast.vec_from_array([ctx.ast.variable_declarator(
span,
VariableDeclarationKind::Var,
ctx.ast.binding_pattern(
BindingPatternKind::BindingIdentifier(
ctx.ast.alloc(var_id.create_binding_identifier(ctx)),
),
NONE,
false,
),
Some(expr),
false,
)]),
false,
),
)));

Also methods should ideally have a high-level comment with a before vs after example (exactly like you did on enter_for_of_statement).

Once the logic of the transform is clearer, it'll be easier to see what perf optimization is possible.

Sorry for the deluge of feedback. I hope it's helpful, rather than annoying!

@@ -45,6 +45,7 @@ fn main() {
let ret = SemanticBuilder::new()
// Estimate transformer will triple scopes, symbols, references
.with_excess_capacity(2.0)
.with_scope_tree_child_ids(true)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change seems extraneous.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i need to think more here, i'm using it

let child_ids = scopes.get_child_ids(current_scope_id);
, but there might be a better way.

will look properly tommorow

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't explain myself well. This file is an example. Did you mean to change the example?

node.body = Statement::BlockStatement(ctx.ast.alloc_block_statement_with_scope_id(
SPAN,
new_body,
ctx.create_child_scope(ctx.current_scope_id(), ScopeFlags::empty()),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is quite correct. current_scope_id isn't updated until after enter_for_of_statement is called:

unsafe fn walk_for_of_statement<'a, Tr: Traverse<'a>>(
traverser: &mut Tr,
node: *mut ForOfStatement<'a>,
ctx: &mut TraverseCtx<'a>,
) {
traverser.enter_for_of_statement(&mut *node, ctx);
let previous_scope_id = ctx.current_scope_id();
let current_scope_id = (*((node as *mut u8).add(ancestor::OFFSET_FOR_OF_STATEMENT_SCOPE_ID)
as *mut Cell<Option<ScopeId>>))
.get()
.unwrap();
ctx.set_current_scope_id(current_scope_id);

So the correct parent scope is for_of_stmt_scope_id, not ctx.current_scope_id().

This is rather unintutive, I know! If I remember right, the reason was that for some types the scope is entered late/exited early (scope doesn't start before the 1st field e.g. Class). So we chose to always call enter_* before entering the scope, and exit_* after exiting the scope, so you don't have to remember which types have this "late entry" / "early exit" behavior, and which don't.

If making that change doesn't fix/fail any tests, I guess there's no tests for the case where for statement doesn't already have a statement block. If so, we should add one.

@camc314 camc314 force-pushed the c/02-23-feat_transformer_transform_explicit_resource_management branch 4 times, most recently from fc2520a to 6bcf4b2 Compare March 5, 2025 22:50
@camc314
Copy link
Contributor Author

camc314 commented Mar 5, 2025

Thank you for the detailed review! I appreciate it.

For the perf hit, i had this thought. Would it be something like:

  1. enter var decl.
  2. if using, add address of parent (e.g. block stmt) to a hash map
  3. when leaving a block, check if the address is in the hash map, if so, do some work. else skip.

If that makes sense, perhapse we merge this once the other feedback has been resolved and fixed in a foillowup?

I've actioned most of your comments just now. the more complex ones i need to think about more before actioning, but i'll try to do tomorrow/friday.

NOTE: still need to add some more code comments about ast builder changes

@camc314 camc314 force-pushed the c/02-23-feat_transformer_transform_explicit_resource_management branch from 6bcf4b2 to beed50c Compare March 5, 2025 22:56
@Boshen
Copy link
Member

Boshen commented Mar 6, 2025

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-transformer Area - Transformer / Transpiler C-enhancement Category - New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

transformer: explicit resource management
4 participants