Stabilize -Zunpretty=expanded

[joshtriplett]

--pretty=expanded is an incredibly useful debugging tool for macros. The change in rustc 1.19 to stop supporting unstable command-line options on the stable/beta compilers makes it all the more important to stabilize unstable options that people use, and this seems like it easily qualifies.

Much like --emit mir, this should not make any commitments about the output, only that the option should continue to exist. In particular, I'd even suggest making it explicitly clear that there's no commitment for the output to compile.

[shepmaster]

For usage in the playground, it would be lovely if it was actually --emit something, perhaps --emit macro-expanded?

[nrc]

I strongly don't want to do this. The generated code is not guaranteed to compile (because hygienic names are not reflected in the generated source text), it means a big chunk of compiler code (pretty printing) is now exposed to stable users, and I worry about bugs, etc. in that code.

I worry that this will encourage tools to try and use that output code, which is absolutely the wrong thing to do.

For debugging macros, we should be able to use expansion info in spans, plus some output saved from macro expansion to give a much better tool - look at every stage of expansion, not just the start and end, have def/ref info for each stage, etc.

[joshtriplett]

@nrc I'd love to see an even more capable mechanism, absolutely, and the idea of stepping through macro expansion step-by-step sounds incredible. But in the interim, --pretty=expanded is what we have for macro debugging, which means stable users have approximately nothing.

If you want to discourage tools from using it, stick an intentional line at the top explaining that the output format isn't stable and prevent it from compiling. But I don't think it's reasonable to take away the only mechanism stable users have for macro debugging, offer no recourse, and not yet have another solution available.

Now, that said, that does mean this will still break cbindgen and other tools which apparently process this code. And that seems quite unfortunate as well, but perhaps more justifiable. But right now, I'm a lot more worried about what to tell stable users who ask "how do I debug my macros?".

[kennytm]

@nrc Every-stage-expansion is useful, but just like we have both -Zdump-mir and --emit mir, I don't think that precludes the introduction of --emit expanded/--print expanded.

[nrc]

But right now, I'm a lot more worried about what to tell stable users who ask "how do I debug my macros?".

We tell them to use nightly. Switching compiler with rustup is so easy, that I don't think a stable requirement is a reason to stabilise an otherwise unstable feature.

[nrc]

I feel like expanded AST is different from mir, because no-one parses the MIR code and expects it to compile (at least as Rust, it would compile as its own language).

[kennytm]

@nrc using nightly doesn't always work, as the involving macro may change between rust versions (consider the difference of thread_local! between stable and nightly.)

[ehuss]

I was wondering if the output isn't compilable, could it at least be parseable? Related to #47287 I came across an issue where a macro expansion results in unparseable output. In particular, I was trying to use cargo expand on the cargo source tree, and rustfmt chokes on this idiom. Not a big deal, one can work around it by not using rustfmt, but unformatted macro expansion can be difficult to read.

// Works
fn f1() -> bool {
    macro_rules! try(( $ e : expr ) => (
        match $ e {
            Some ( e ) => e ,
            None => return false ,
        }
    ));

    try!(Some(42)) == 42
}

// Expanded version doesn't work
fn f2() -> bool {
    match Some(42) {
        Some ( e ) => e ,
        None => return false ,
    } == 42
}

// Could be OK with parenthesis (at least in this case)
fn f3() -> bool {
    (match Some(42) {
        Some ( e ) => e ,
        None => return false ,
    }) == 42
}

[fedochet]

For debugging macros, we should be able to use expansion info in spans, plus some output saved from macro expansion to give a much better tool - look at every stage of expansion, not just the start and end, have def/ref info for each stage, etc.

@nrc can you please provide more details about how we can retrieve such information (if this is possible)?

[kvark]

Note that the situation needs some kind of solution at the moment. --pretty=expanded is used by cbindgen, which helps gluing together WebRender with Gecko as well as powering other crates, like wgpu-bindings.
If stabilizing the pretty output is not an option, what would be the right way to proceed with making cbindgen robust? cc @eqrion