Skip to content

Porting changes from Babel's parser

Jump to bottom
Alan Pierce edited this page Jul 7, 2021 · 5 revisions

Background

Sucrase's parser is based off of Babel's, with some significant modifications to specialize it toward's Sucrase's smaller scope. An ongoing maintenance task for Sucrase is to explicitly consider each code change to the Babel parser and apply it to Sucrase's parser if it's useful. Most code changes do not apply, but this process helps incorporate parser bug fixes and new ECMAScript and TypeScript features as they are introduced.

Ideally, this porting happens on a regular basis, but is especially motivated by new TypeScript syntax. Some past examples:

How to port a batch of changes

  1. Determine which commits remain to be ported. Look in Sucrase's commit history or pull request history for the most recent change starting with "Port babel-parser changes". Go to that commit message and find the bottom-most commit. All work after that point still needs to be ported.
  2. Get an up-to-date Babel repo checkout. If you haven't yet, clone the Babel monorepo using git clone https://github.com/babel/babel.git. If you have it already cloned, run git fetch and git checkout origin/main.
  3. List all babel-parser changes. This can be done with git log --reverse --format="%h %s" packages/babel-parser, then save the results to a file or text editor. From there, you can remove the lines that have already been ported and consider limiting the number of changes to a reasonable amount for a single PR. Remove all usages of the [skip ci] directive in commit titles, since that will cause Sucrase's CI to be skipped. To view the actual changes for porting, it's best to use the GitHub UI or a git viewer like gitk packages/babel-parser.
  4. Go through each change and consider porting it. If no action is necessary, leave a line starting with "🚫" and a rationale about why no action was needed. If the change applies to Sucrase, leave a "✅" with an explanation of the work done (e.g. ported, implemented independently, filed an issue for later).
  5. Put the change up for code review. The PR title should have the date range of the Babel work, and the PR description should have the list of Babel commits and what action was taken. It's fine to limit the date range to a smaller amount for the sake of keeping work more incremental, but every commit in that range should be considered (even if it just means filing an issue).

Considerations when porting a change

The Sucrase parser at src/parser started off as a simple fork of the Babel parser, but has undergone a number of changes since then:

  • It was ported to TypeScript.
  • The parser folder was renamed to traverser to better reflect Sucrase's approach.
  • The class hierarchy was changed to instead use top-level functions, with explicit if statements when behavior needs to vary due e.g. with TypeScript enabled or disabled.
  • The traverser steps were simplified to never produce a syntax tree, only tokens (with some additional useful information attached to tokens) and a flat list of scopes.
  • Almost all validation was removed. Sucrase is designed to operate on valid code, and most error checking (like disallowing two exports with the same name) should be left to other tools.
  • Almost all configuration options were removed. Sucrase biases toward accepting upcoming syntax rather than gating it behind a flag.
  • The "token context" system was completely removed, since it was redundant with the information provided by the parser.
  • The estree parser was removed, since it is not needed for Sucrase.
  • Support for sloppy mode (non-strict mode) was removed.

When porting a babel-parser commit, many types of work can be immediately skipped:

  • Any releases, changes to tooling, documentation or other chore-like tasks.
  • Any new error reporting or fixes to existing error reporting.
  • Any changes to the AST format or fixes to AST generation.
  • Fixes for bugs that aren't present in Sucrase.

For changes that may apply, they need to be considered case-by-case in the context of Sucrase. Here are some thoughts to consider for those changes:

  • For new TypeScript features, it's often easy enough to translate the code change directly to the relevant part of Sucrase. If that's a challenge, you can use the Babel commit for inspiration and to get a sense of what test cases need to be covered, then implement the change independently. It's also sometimes helpful to look at the TypeScript parser implementation for inspiration.
  • Some changes are useful but lower priority, like new proposed syntax that isn't close to being standardized, or fixes for code that seems unrealistic. In those cases, it's best to port the change if possible, but if it's challenging, it's also fine to file an issue as a follow-up and skip porting the change for now.
  • New tests may or may not make sense to port directly; the question to ask is whether the test feels useful for Sucrase. Sometimes, for example, tests are designed to cover complexity in Babel that doesn't exist in Sucrase. If they do make sense for Sucrase, they should usually be written as an end-to-end transform test and added to sucrase-test.ts or a related test file. Sucrase does not use the same test fixture system that Babel's parser has.
  • Refactors often won't make sense for Sucrase (especially if they're motivated by code goals that aren't in Sucrase's scope), but may be worth considering.
Clone this wiki locally