Skip to content

Improve Debug representation of proc_macro::TokenStream #49720

Closed
@dtolnay

Description

@dtolnay

Currently the Debug representation of the TokenStream corresponding to tokens [a + 1] is as follows. This is just delegating to the derived Debug impls of the underlying syntax::tokenstream::TokenStream.

Show current Debug representation of [a + 1]...
TokenStream {
    kind: Stream(
        [
            TokenStream {
                kind: Tree(
                    Delimited(
                        Span {
                            lo: BytePos(
                                79
                            ),
                            hi: BytePos(
                                86
                            ),
                            ctxt: #0
                        },
                        Delimited {
                            delim: Bracket,
                            tts: ThinTokenStream(
                                Some(
                                    [
                                        TokenStream {
                                            kind: Tree(
                                                Token(
                                                    Span {
                                                        lo: BytePos(
                                                            80
                                                        ),
                                                        hi: BytePos(
                                                            81
                                                        ),
                                                        ctxt: #0
                                                    },
                                                    Ident(
                                                        a#0,
                                                        false
                                                    )
                                                )
                                            )
                                        },
                                        TokenStream {
                                            kind: Tree(
                                                Token(
                                                    Span {
                                                        lo: BytePos(
                                                            82
                                                        ),
                                                        hi: BytePos(
                                                            83
                                                        ),
                                                        ctxt: #0
                                                    },
                                                    BinOp(
                                                        Plus
                                                    )
                                                )
                                            )
                                        },
                                        TokenStream {
                                            kind: Tree(
                                                Token(
                                                    Span {
                                                        lo: BytePos(
                                                            84
                                                        ),
                                                        hi: BytePos(
                                                            85
                                                        ),
                                                        ctxt: #0
                                                    },
                                                    Literal(
                                                        Integer(
                                                            1
                                                        ),
                                                        None
                                                    )
                                                )
                                            )
                                        }
                                    ]
                                )
                            )
                        }
                    )
                )
            }
        ]
    )
}

It would be far more helpful to have the Debug for proc_macro::TokenStream line up better with the TokenStream public API. Something like:

TokenStream [
    Group {
        delimiter: Bracket,
        stream: TokenStream [
            Term {
                name: "a",
                span: #0 bytes(80..81)
            },
            Op {
                op: '+',
                spacing: Alone,
                span: #0 bytes(82..83)
            },
            Literal {
                value: `1`,
                span: #0 bytes(84..85)
            }
        },
        span: #0 bytes(79..86)
    }
]

Metadata

Metadata

Assignees

Labels

A-decl-macros-2-0Area: Declarative macros 2.0 (#39412)

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions