Skip to content

Improve modeling token nodes #2010

Open
@Matejkob

Description

@Matejkob

With @ahoppen we identified potential confusion and overcomplication in code relating to modeling token nodes. Specifically, we're currently trying to collapse multiple token kinds into a single one without indicating this in our variable names.

The only non-trivial place that needs to be updated is Child.type in SyntaxBuildableChild and, IMHO, the confusion stems from the fact that we try to collapse multiple token kinds into a single one without giving any indication of doing so in the variable names.

I think the way to go about fixing this is to change the associated value of SyntaxOrTokenNodeKind.token to [TokenChoice]. Then we won’t need to call Child.tokenKind in Child.type anymore and could shift the computation that’s currently done in this property down to uses where there’s a lot more context. Potentially, the computation might also just become a lot simpler once we see that context.

Originally posted by @ahoppen in #2006 (comment)

In relation to issue #1978, changes were made in PR #2006 to the Child.tokenKind implementation. However, this solution still appears to be incorrect and needs further attention, which this issue aims to address.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions