Description
Did you check existing issues?
- I have read all the tree-sitter docs if it relates to using the parser
- I have searched the existing issues of tree-sitter-python
Tree-Sitter CLI Version, if relevant (output of tree-sitter --version)
tree-sitter 0.25.2 (6e0618704ad758ba2ea5822faa80bcd36fbeba3d)
Describe the bug
The current python grammar has issues properly separating compound statements such as if statements, function definitions, and class definitions when these statements are only partially complete, leading to a lot of flickering colors when additional syntax highlighting is enabled (for instance with Neovim).
My tests seem to indicate that this is due to the tree-sitter-python grammar over binding compound statements such as if (expression) and def f() which are missing a ":" with the next ":" in the file instead of producing a local error for that if statement.
This means that if one is writing new code above existing text, the parse tree will connect new if, def, and class statements (or other statements that require a ":') with subsequent ones producing a lot of erroneous highlights.
I have made some initial attempts at modifying the grammar; however, I have not made much progress. It seems that a stronger link needs to be established between the keywords if, def, class and the next subsequent ":".
I tried adding in a right associativity rule to the if_statement and function_definition rules but that did not seem to help.
Steps To Reproduce/Bad Parse Tree
Run tree-sitter parse (repro code) to get:
(module [0, 0] - [3, 0]
(if_statement [0, 0] - [2, 8]
condition: (call [0, 3] - [2, 7]
function: (true [0, 3] - [0, 7])
(ERROR [2, 0] - [2, 5]
(identifier [2, 0] - [2, 3])
(identifier [2, 4] - [2, 5]))
arguments: (argument_list [2, 5] - [2, 7]))
consequence: (block [2, 8] - [2, 8])))
Expected Behavior/Parse Tree
Something like:
(ERROR [0, 0] - [0, 8]
condition: (true [0, 3] - [0, 7])
consequence: (block [0, 8] - [0, 8]))
(function_definition [2, 0] - [2, 8]
name: (identifier [2, 4] - [2, 5])
parameters: (parameters [2, 5] - [2, 7])
body: (block [2, 8] - [2, 8])))
Repro
if True:
def a():
Activity