Improve contribution and PR process

[keynmol]

Aim of this is to increase contribution velocity on this repository without impacting users.

Current situation

1. If you have 2 PRs changing any two parts of the grammar, they will immediately conflict with each other because of the dramatic changes in generated code (usually those changes are not mergeable)
2. This means parallel PRs are in constant queue and need to be updated after each merge to the master

Suggestion

1. Grammar PRs only bring in changes to grammar.js, corpus/* or src/scanner.c - not parser.c or any other generated code
   • This will dramatically reduce the conflicts and will make all PRs more likely to be mergeable without manual updates
2. CI still checks for parser code being generated from grammar alone, and run tests against that
3. When we feel like a release is due, we actually generate parser code and commit it as a separate PR

Because of the CI checks, the above process should be safe, at the cost of not containing the latest-est generated code in master. Which IMO is not a big deal.

[eed3si9n]

I like the nature that this repository is ready-to-use from the consumer's perspective, even though it is kind of weird that there's a generated code committed inside. The "release" approach I think requires human attention going forward, so I am not sure if that's a sustainable approach. I don't want to rely too much on high energy.

Alternatively, given the "current situation" mostly happens between you and me for some span of time, maybe we can come up with a convention for ourselves?:

Suggestion 2

1. Our grammar PRs only bring in changes to grammar.js, corpus/* or src/scanner.c - not parser.c or any other generated code
   • This will dramatically reduce the conflicts and will make all PRs more likely to be mergeable without manual updates
2. CI still checks for parser code being generated from grammar alone, and run tests against that
3. Per PR, we immediately send a second PR with generated code inside, and merge it upon the grammar PR merge.
   • Since other grammar PRs do not conflict, this should only conflict the other codegen PRs only
4. Run a weekly cron job to check if the generated code is fresh

What do you think?

[keynmol]

Since other grammar PRs do not conflict, this should only conflict the other codegen PRs only

I think this gets us back to the same situation - if each grammar PR has a corresponding codegen PR, then you have the same number of conflicts and same linear queue of PRs, each of them needs to be updated manually (even if update is literally npm run build && git commit --amend --no-edit && git push -f.

Plus I think it introduces more process to avoid forgetting to send the codegen PR for the right change.

For the "immediate" nature of this repo, you can always have requires_generate_from_grammar = false in your nvim-treesitter config for Scala parser. Slow, but deadly.

What we really want here is a github action that auto-commits generated code after merge to master, but I think that will require setting up a bot account, right? No other way I think. And that's definitely too much effort

[eed3si9n]

If codegen PR conflicts, we can ignore them until the grammar PR gets merges, so we don't have to worry about it during the PR iteration.

For the "immediate" nature of this repo, you can always have requires_generate_from_grammar = false in your nvim-treesitter config for Scala parser. Slow, but deadly.

Not sure if that's still a good tradeoff since now right when I want to use vim, it could go into 20 min of codegen on Mac laptop. Also if I want most users to take advantage of the new things soon, then they'd have to go through the same torture.

How about this?

Suggestion 3

3. At least once a week, we actually generate parser code and commit it as a separate PR

For now I'd be happy to take on this task using my Linux box.

[keynmol]

Honestly I was thinking of doing it almost daily 😅 As long as there are PRs of course
As we get through the phase of tackling low-hanging fruit in both Scala 2 and Scala 3.

I'm also happy to do it, so let's coordinate when necessary.

[eed3si9n]

Here's #86, which generates parser on macOS as well so we can omit the parser.c.

[keynmol]

I think we're doing pretty well, and most of the process is automated and we even have this documented thanks to @ckipp01. Lots of PRs, lots of syncing, no conflicts.

Spectacular. Vivacious. Joyful and opulent.

@eed3si9n @ckipp01 if you don't have any further suggestions on this, I think we can close this.

[ckipp01]

Sounds good to me! I think if there are individual things related tot the process we want to improve on we can create issue for them.