Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Labels don't show unless the EOI is in it's place #631

Open
mnbjhu opened this issue May 1, 2024 · 5 comments
Open

Labels don't show unless the EOI is in it's place #631

mnbjhu opened this issue May 1, 2024 · 5 comments

Comments

@mnbjhu
Copy link

mnbjhu commented May 1, 2024

Suppose I have the following parser:

    let ident = select! {
        Token::Ident(ident) => ident,
    };
    just(Token::Keyword(Keyword::Select))
        .ignore_then(ident.labelled("column"))
        .then_ignore(just(Token::Keyword(Keyword::From)))
        .then(ident.labelled("table"))
        .map(|(column, from)| Statement::Select(Select { column, from }))

When I parse this input:

(Keyword(Select), 0..6)
(Ident("some"), 7..11)
(Keyword(From), 12..16)
(Newline, 16..17)

Then I get an error: 'found newline expected something else'. However if I use:

(Keyword(Select), 0..6)
(Ident("some"), 7..11)
(Keyword(From), 12..16)

Then I get the error: 'found end of input expected table' which is what I'd expect. Is this a bug or am I missing something? (1.0.0-alpha.7)

@zesterer
Copy link
Owner

Hmm, this odd and seems unintended. Do you have a minimal working example of this I can play with?

@mnbjhu
Copy link
Author

mnbjhu commented May 10, 2024

Of course, I've created an example here: https://github.com/mnbjhu/label_test. The test 'error_at_token' (main.rs) fails

ojkelly added a commit to ojkelly/chumsky that referenced this issue Jun 22, 2024
When a source does not contain a newline,  labels are lost/ignored and replaced with the fallback `expected something else`.

This reverts the change made in this commit which appears to fix this zesterer@6837537#diff-3d2dfe7e70ee2398dd0941e8f6ea855652695c3f82b7816f649668fb2d56b93dR60
@ojkelly
Copy link

ojkelly commented Jun 22, 2024

I noticed this after attempting to update, and found it only occurs on sources without a newline. I'm working on a project with a few hundred tests without newlines (unit tests for individual parsers for example).

I had a dig through the diff from what I had been using and found what I think might be the cause #645

I'm not fully across the details of that area of code, so it may not be the ideal fix but hopefully it's useful.

@mnbjhu
Copy link
Author

mnbjhu commented Jun 25, 2024

Thanks for this, I'll make sure to check it out this weekend. I've been working on a language server recently and this will be really useful :)

@zesterer
Copy link
Owner

Sorry, this got buried in my notifications. Thanks for the PR!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants