Skip to content

Completion in macros #3513

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Mar 9, 2020
Merged

Conversation

flodiebold
Copy link
Member

I experimented a bit with completion in macros. It's kind of working, but there are a lot of rough edges.

  • I'm trying to expand the macro call with the inserted fake token. This requires some hacky additions on the HIR level to be able to do "hypothetical" expansions. There should probably be a nicer API for this, if we want to do it this way. I'm not sure whether it's worth it, because we still can't do a lot if the original macro call didn't expand in nearly the same way. E.g. if we have something like println!("", x<|>) the expansions will look the same and everything is fine; but in that case we could maybe have achieved the same result in a simpler way. If we have something like m!(<|>) where m!() doesn't even expand or expands to something very different, we don't really know what to do anyway.
  • Relatedly, there are a lot of cases where this doesn't work because either the original call or the hypothetical call doesn't expand. E.g. if we have m!(x.<|>) the original token tree doesn't parse as an expression; if we have m!(match x { <|> }) the hypothetical token tree doesn't parse. It would be nice if we could have better error recovery in these cases.

@flodiebold
Copy link
Member Author

Peek 2020-03-07 16-32

It works in the most important macros, at least 😄 You can see I have to type one extra letter for dot completions to work in dbg or vec because they take an expr fragment.

let mut fake_ident_token = fake_ident_token;

// Are we inside a macro call?
while let (Some(actual_macro_call), Some(macro_call_with_fake_ident)) = (
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Originally I've thought that recusively expanding the macros is exactly how we should do IDE features. However, @edwin0cheng had an idea which seems like it kinda can be better? In syntax highlighting, we descend to the bottom of macro expansion in one step:

https://github.com/rust-analyzer/rust-analyzer/blob/aff82cf7ac172f213cb5dcca637cb2c5332294c1/crates/ra_ide/src/syntax_highlighting.rs#L109

I think ideally this is what we want to do for completion as well. Although I am not sure if expand_hypothetical can be made to work this way, especially if we switch from "resolve" to "record" strategy for SourceAnalyzer

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some vague thoughs for negative durability in salsa for such hypothetical cases...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Support in salsa for throwaway 'alternative universes' would be cool...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But about the recursive expanding, I wanted to avoid a situation where the 'hypothetical' expansion is too different; I think we need to 'follow along' both to keep track. There are probably a lot of ways this can currently break if the inserted token changes the expansion too much.

match self.token.kind() {
// workaroud when completion is triggered by trigger characters.
IDENT => self.token.text_range(),
IDENT => self.original_token.text_range(),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, this seems like a preexisting bug? Perhaps we need to flip the naming? original_x => x, x => fake_x?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a newly introduced distinction between the token in the original source file and the token in the macro expansion, not between real/fake 😅

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

... and in most situations we do want the macroexpanded token.

Two uses only needed the crate; one was wrong and should use the module from the
scope instead.
Comment on lines 74 to 104
pub fn expand_hypothetical(
&self,
actual_macro_call: &ast::MacroCall,
hypothetical_call: &ast::MacroCall,
token_to_map: SyntaxToken,
) -> Option<(SyntaxNode, SyntaxToken)> {
let macro_call =
self.find_file(actual_macro_call.syntax().clone()).with_value(actual_macro_call);
let sa = self.analyze2(macro_call.map(|it| it.syntax()), None);
let macro_call_id = macro_call
.as_call_id(self.db, |path| sa.resolver.resolve_path_as_macro(self.db, &path))?;
let macro_file = macro_call_id.as_file().macro_file().unwrap();
let (tt, tmap_1) =
hir_expand::syntax_node_to_token_tree(hypothetical_call.token_tree().unwrap().syntax())
.unwrap();
let range = token_to_map
.text_range()
.checked_sub(hypothetical_call.token_tree().unwrap().syntax().text_range().start())?;
let token_id = tmap_1.token_by_range(range)?;
let macro_def = hir_expand::db::expander(self.db, macro_call_id)?;
let (node, tmap_2) = hir_expand::db::parse_macro_with_arg(
self.db,
macro_file,
Some(std::sync::Arc::new((tt, tmap_1))),
)?;
let token_id = macro_def.0.map_id_down(token_id);
let range = tmap_2.range_by_token(token_id)?.by_kind(token_to_map.kind())?;
let token = algo::find_covering_element(&node.syntax_node(), range).into_token()?;
Some((node.syntax_node(), token))
}

Copy link
Member

@edwin0cheng edwin0cheng Mar 8, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Neat!
So what this function does is, replace the original macro call with the hypothetical_call arguments. And I see the limitation here: we don't want to pollute the salsa database such that we can't convert the hypothetical_call to an actual MacroCallId. So we have to expand it manually.

One small nit: we might change the arguments to reflect it only replace the arguments :

    pub fn expand_hypothetical(
        &self,
        actual_macro_call: &ast::MacroCall,
        hypotheticall_args: &ast::TokenTree,
        token_to_map: SyntaxToken,
)

And I can imagine we could move some of the logic here to hir_expand but overall it is awesome !

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good points, I'll move most of it to hir_expand, then I think it can be pretty contained there.

@flodiebold flodiebold force-pushed the completion-in-macros branch 2 times, most recently from cb06162 to 59da22f Compare March 8, 2020 10:08
@flodiebold flodiebold marked this pull request as ready for review March 8, 2020 10:09
@flodiebold flodiebold force-pushed the completion-in-macros branch from 59da22f to afdf08e Compare March 8, 2020 10:10
@flodiebold
Copy link
Member Author

Here's something cool:

tested_by

Or this:

dot

😄

@edwin0cheng
Copy link
Member

LGTM !

@matklad
Copy link
Member

matklad commented Mar 9, 2020

bors r+

@bors
Copy link
Contributor

bors bot commented Mar 9, 2020

@bors bors bot merged commit beb4f49 into rust-lang:master Mar 9, 2020
@flodiebold flodiebold deleted the completion-in-macros branch March 9, 2020 09:11
bors added a commit that referenced this pull request Jun 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants