Allow access to location information from proc_macro TokenStreams

I would like to build a template engine that leverages custom derive to attach a template rendering function to a struct at compile time. I was planning on attaching an attribute to the struct to specify the path to the template file, which could then be read and transpiled into Rust. However, it seems it is currently not possible to get at the path to the source file where the TokenStream originated. As such, it seems I would have to rely on absolute paths to template files or on the current working directory. Even assuming cargo compiles (including workspaces), this does not seem reliable.

The Items that are part of the TokenStream contain a Span, but I don't think the CodeMap they index into is currently available to a proc_macro. Would that be a viable implementation strategy?

My experimental code can be found here: https://github.com/djc/askama

@jseyfried

Is there any update on this? I need this feature, too: I'm writing a macro that has its own "module system" so to speak. Thus I need to load files form disk relative to the file in which the macro is called.

@jseyfried @alexcrichton I could try implementing this feature for proc_macro. What do you think? Are there reasons not to expose the current path to the macro author?

There's some work in #43604 but AFAIK there's no movement towards manufacturing custom spans. I know it's something we'd like to support eventually!

#43604 is mainly just waiting on review right now.

This is Span::call_site, right? That's stable: https://doc.rust-lang.org/stable/proc_macro/struct.Span.html#method.call_site

I have a hard time remembering exactly my intent with this issue from 5 years ago, but I think this would depend on getting proc_macro_span stabilized (for the Span::source_file() method).

@petrochenkov

What would actually be more useful, less work to stabilize and have additional utility, is to basically just be able to resolve a file path the way include!() or include_str!() does, given a span or SourceFile.

Then like the new proc_macro_tracked_path feature, the compiler could know to watch that file for changes and trigger recompilation.

@petrochenkov I feel like it would actually make a lot of sense to just combine these. Maybe something like:

impl SourceFile {
    /// Load a file relative to this source file and parse it as a `TokenStream`.
    ///
    /// The file path will be included in the resulting dep-info and will trigger recompilation when it changes.
    pub fn include(&self, path: impl AsRef<Path>) -> Result<TokenStream, IncludeError> { ... }

    /// Load a file relative to this source file as a UTF-8 string.
    ///
    /// The file path will be included in the resulting dep-info and will trigger recompilation when it changes.
    pub fn include_str(path: impl AsRef<Path>) -> Result<String, IncludeError> { ... }
}

This would actually fix a long-outstanding issue in SQLx (although I think include_str!() would fix it, just haven't gotten around to trying it): transact-rs/sqlx#663 (comment)

Yeah, that would be pretty great to have for my use case.

Navigation Menu

Repository navigation

Allow access to location information from proc_macro TokenStreams #38546

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Allow access to location information from proc_macro TokenStreams #38546

Description

Activity

djc commented on Dec 22, 2016

LukasKalbertodt commented on Aug 25, 2017

alexcrichton commented on Aug 25, 2017

abonander commented on Aug 25, 2017

jyn514 commented on Jul 14, 2021

djc commented on Jul 14, 2021

abonander commented on Jul 14, 2021

djc commented on Jul 14, 2021

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions