Skip to content

Commit

Permalink
misc: Add a guide to update grammars
Browse files Browse the repository at this point in the history
  • Loading branch information
Luni-4 committed Dec 19, 2023
1 parent 33f6f26 commit cc1a5a5
Show file tree
Hide file tree
Showing 3 changed files with 69 additions and 0 deletions.
5 changes: 5 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,11 @@ cargo insta test --review

Will run the tests, generate the new snapshot references and let you review them.

### Updating grammars
Have a look at
<a href="https://mozilla.github.io/rust-code-analysis/developers/update-grammars.html" target="_blank">Update grammars guide</a>
to learn how to update languages grammars.

# Contributing

If you want to contribute to the development of this software, have a look at the
Expand Down
1 change: 1 addition & 0 deletions rust-code-analysis-book/src/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,4 @@
- [Developers Guide](developers/README.md)
- [How-to: Add a new language](developers/new-language.md)
- [How-to: Implement LoC](developers/loc.md)
- [How-to: Update grammars](developers/update-grammars.md)
63 changes: 63 additions & 0 deletions rust-code-analysis-book/src/developers/update-grammars.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
# Update grammars

Each programming language needs to be parsed in order to extract its syntax and semantic: the so-called grammar of a language.
In `rust-code-analysis`, we use [tree-sitter](https://github.com/tree-sitter) as parsing library since it provides a set of distinct grammars for each of our
supported programming languages. But a grammar is not a static monolith, it changes over time, and it can also be affected by bugs,
hence it is necessary to update it every now and then.

As now, grammars can be update **only** on `Linux` and `MacOS` systems because we have used `bash` scripts to
bring together and reduce the sequence of operations.

In `rust-code-analysis` we use both **third-party** and **internal** grammars.
The first one are published on `crates.io` and maintained by external developers,
while the second one have been thought and defined inside the project to manage variant of some languages
used in `Firefox`.
We are going to explain how to update both of them in the following sections.

## Third-party grammars

1. Update the grammar version in `Cargo.toml` and `enums/Cargo.toml`. Below an example for the `tree-sitter-java` grammar

```toml
tree-sitter-java = "x.xx.x"
```

where `x` represents a digit.

2. Run `./recreate-grammars` to recreate and refresh all grammars structures and data

```bash
./recreate-grammars
```

3. Once the script above has finished its execution, you need to fix, if there are any, all failed tests and problems
introduced by changes in the grammars.

4. Commit your changes and create a new pull request

## Internal grammars

1. Update dependency `version` field in `Cargo.toml` and `enums/Cargo.toml`. Below an example for the `tree-sitter-ccomment` grammar

```bash
tree-sitter-ccomment = { path = "./tree-sitter-ccomment", version = "=x.xx.x" }
```
where `x` represents a digit.
2. Open the `Cargo.toml` file of the chosen grammar and:
- Set its version to the **same** value present in the main `Cargo.toml` file
- Increase the `tree-sitter` version to the most recent one
3. Run `./generate-grammars/generate-grammar.sh` which updates the grammar recreating and refreshing every file and script.
This script requires the name of the grammar as mandatory argument.
Below an example always using the `tree-sitter-ccomment` grammar
```bash
./generate-grammars/generate-grammar.sh tree-sitter-ccomment
```
4. Once the script above has finished its execution, you need to fix, if there are any, all failed tests and problems
introduced by changes in the grammars.
5. Commit your changes and create a new pull request

0 comments on commit cc1a5a5

Please sign in to comment.