transformer: Sync AST nodes and scopes

[overlookmotel]

It's common in transformer that you need to:

1. Remove a statement from AST.
2. Replace a statement with multiple statements.
3. Insert a statement at top of a statement block (e.g. import React from 'react').

All of these cause us problems because the contents of Vec<Statement> needs to be shuffled up/down, which can be expensive, especially at top level where there may be many statements.

We currently have various workarounds for this, including delaying inserting statements until later in transform, or rebuilding the Vec<Statement>, copying all the contents. Multiple transforms may repeat this "shuffle" on the same set of statements.

Could we add a variant Statement::CompositeStatement?

struct CompositeStatement<'a> {
  stmts: Vec<'a, Statement<'a>>
}

During transform, you can perform any insert/replace/delete action by replacing a Statement with a CompositeStatement. Then when exiting the statement block, if it contains any CompositeStatements, the Vec<Statement> would be shuffled to flatten all the CompositeStatements. This could happen in place and in a single pass, enacting all the changes made by multiple transforms all in one go.

i.e. CompositeStatement would only be used temporarily in the transformer, and they'd be gone in final AST, when transformation is complete.

The other advantage is that we can keep AST and scopes tree in sync at all times. Currently we add bindings to scope tree for new insertions before the actual AST nodes which produce those bindings are inserted, so there's a period where the 2 are out of sync. This may cause us problems further down the line.

The downside of this idea is that whenever iterating over a Vec<Statement> in transformer, you now need to handle CompositeStatements and step into them.

[Boshen]

We need a way to detect broken scopes in transform-ci.

[overlookmotel]

Replacing Vec<Statement> with a chunked linked list would be another way to solve these problems. Probably better than the CompositeStatement solution suggested above.

[Dunqing]

I created a #4581 to add a test to verify that ScopeTree and SymbolTable are correct after the transformation

[overlookmotel]

Closing in favour of oxc-project/backlog#35.