-
Notifications
You must be signed in to change notification settings - Fork 50
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
misc: Add a guide to update grammars
- Loading branch information
Showing
3 changed files
with
69 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,63 @@ | ||
# Update grammars | ||
|
||
Each programming language needs to be parsed in order to extract its syntax and semantic: the so-called grammar of a language. | ||
In `rust-code-analysis`, we use [tree-sitter](https://github.com/tree-sitter) as parsing library since it provides a set of distinct grammars for each of our | ||
supported programming languages. But a grammar is not a static monolith, it changes over time, and it can also be affected by bugs, | ||
hence it is necessary to update it every now and then. | ||
|
||
As now, grammars can be update **only** on `Linux` and `MacOS` systems because we have used `bash` scripts to | ||
bring together and reduce the sequence of operations. | ||
|
||
In `rust-code-analysis` we use both **third-party** and **internal** grammars. | ||
The first one are published on `crates.io` and maintained by external developers, | ||
while the second one have been thought and defined inside the project to manage variant of some languages | ||
used in `Firefox`. | ||
We are going to explain how to update both of them in the following sections. | ||
|
||
## Third-party grammars | ||
|
||
1. Update the grammar version in `Cargo.toml` and `enums/Cargo.toml`. Below an example for the `tree-sitter-java` grammar | ||
|
||
```toml | ||
tree-sitter-java = "x.xx.x" | ||
``` | ||
|
||
where `x` represents a digit. | ||
|
||
2. Run `./recreate-grammars` to recreate and refresh all grammars structures and data | ||
|
||
```bash | ||
./recreate-grammars | ||
``` | ||
|
||
3. Once the script above has finished its execution, you need to fix, if there are any, all failed tests and problems | ||
introduced by changes in the grammars. | ||
|
||
4. Commit your changes and create a new pull request | ||
|
||
## Internal grammars | ||
|
||
1. Update dependency `version` field in `Cargo.toml` and `enums/Cargo.toml`. Below an example for the `tree-sitter-ccomment` grammar | ||
|
||
```bash | ||
tree-sitter-ccomment = { path = "./tree-sitter-ccomment", version = "=x.xx.x" } | ||
``` | ||
where `x` represents a digit. | ||
2. Open the `Cargo.toml` file of the chosen grammar and: | ||
- Set its version to the **same** value present in the main `Cargo.toml` file | ||
- Increase the `tree-sitter` version to the most recent one | ||
3. Run `./generate-grammars/generate-grammar.sh` which updates the grammar recreating and refreshing every file and script. | ||
This script requires the name of the grammar as mandatory argument. | ||
Below an example always using the `tree-sitter-ccomment` grammar | ||
```bash | ||
./generate-grammars/generate-grammar.sh tree-sitter-ccomment | ||
``` | ||
4. Once the script above has finished its execution, you need to fix, if there are any, all failed tests and problems | ||
introduced by changes in the grammars. | ||
5. Commit your changes and create a new pull request |