Skip to content

Lrlex codegen - #661

Open
ratmice wants to merge 9 commits into
softdevteam:masterfrom
ratmice:lrlex_codegen
Open

Lrlex codegen#661
ratmice wants to merge 9 commits into
softdevteam:masterfrom
ratmice:lrlex_codegen

Conversation

@ratmice

@ratmice ratmice commented Aug 19, 2026

Copy link
Copy Markdown
Collaborator

This is an attempt at lrlex codegen, in the style of the recently added lrpar codegen module.
One thing this doesn't do is change the CTTokenMapBuilder.

This tries to sequence the move over a couple of commits, with the first commit just generating module and ownership structure. With each subsequent commit moving one aspect of code generation, until it's all in the codegen module.

One slightly suprising thing was the placement of lexerdef within the LexerBuildEnv, because we need it to generate the rule_ids_map, with that we can wait to generate the LexerCodegen util the rule_ids_map is built, and pass it in to the code_generator function, this is a slight departure from the lrpar api, which merely took a timestamp.

Edit: One thing that the additional rule_ids_map parameter I immediately notice is that unlike BuildEnvArgs and other builder patterns, it's hard for us to add add an additional parameter should we need one. I wonder if we should consider LexerCodegenArgs?

Comment thread lrlex/src/lib/codegen.rs
}
}

pub(crate) enum LexerBuildEnvError {}

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had forgotten to double check this, for some reason I thought a variant had been added to this,
but that was LexerCodegenError.

)
let code_gen = build_env
.code_generator(self.rule_ids_map, env!("VERGEN_BUILD_TIMESTAMP"))
.map_err(|e| match e {})?;

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This wacky map_err/match was necessary at the time because ? couldn't convert the error.
I figured by the time I was done we'd have a variant added. Although I must admit I'm a bit uncertain what the type returned is, I assume it's inferring ! or some such.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a new one on me! I imagine it's inferring the unit type ().

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd be surprised if () would convert into Box<dyn Error> via ? though, which is why I think it must be !
or Infallible.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see what you mean. I guess the ultimate point here is that the closure passed to map_err can't ever be called because e is of a type that cannot be constructed.

Comment thread lrlex/src/lib/codegen.rs
Comment thread lrlex/src/lib/codegen.rs
Comment thread lrlex/src/lib/codegen.rs Outdated
f.write_str(&match self {
Self::InvalidRustIdentifierModName { mod_name, error } => {
format!(
"Invalid mod_name '{mod_name}' is not a valid rust identifier due to '{error}'"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I'd drop the "Invalid " prefix: the rest of the message makes clear it's an error.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 741cecf

I checked lrpar/codegen, and the latter didn't include "Invalid" already, so the two errors looked in sync.

)
let code_gen = build_env
.code_generator(self.rule_ids_map, env!("VERGEN_BUILD_TIMESTAMP"))
.map_err(|e| match e {})?;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a new one on me! I imagine it's inferring the unit type ().

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants