proc macro API: Making `enum Delimiter` non-exhaustive

[petrochenkov]

... with an eye on extending it with angle brackets < TOKEN_TREES > in the future.

Yes, </> may be unbalanced, but I suspect that we can always consider balanced angle brackets as a group with minimal breakage (based on estimates from #53578).

Lang team is seriously intending to remove turbofish from value paths (value::<Type> => value<Type>) despite all the grammar implications, and angle-bracketed groups may be the only way to do it without requiring backtracing from parsers.

Is it realistic to make this change now, when 1.29 with stabilized proc macro API was just released?
This stabilization was never announced in release notes or 1.29 blog post, so it may be possible that only infrastructural crates like proc-macro2/syn/... are affected.
What are the alternatives if we want to make < ... > a group?
cc @dtolnay @alexcrichton @varkor

[alexcrichton]

Changing the stable API now that it's shipped is a non-starter, and unfortunately I don't know how we could retroactively add this.

[petrochenkov]

I guess, as a last resort we could

• Keep the true delimiter internally in Group
• Report Delimiter::None in fn delimiter if the true delimiter is angle brackets.
• Add Group::delim_is_angle_bracket, or enum Delimiter2 + Group::delimiter2.
• Paper all this over in proc-macro2/syn and friends.

[dtolnay]

If the intention is to make parsing easier, that doesn't require <...> to be a delimiter. For example representing it as a None-delimited group inside of which the first and last token are < and > would work just as well.

[petrochenkov]

I thought about this, but assumed that there's a difference between a "proper group" and "None-delimited group that just happens to start/end with angle brackets".
But perhaps that's not actually so important? Hmm.

[dtolnay]

I believe those would be equivalently easy for a parser to handle (at least Syn's parser).

[alexcrichton]

Removing nomination due to discussion in libs triage, @dtolnay will follow-up with some more specifics

[petrochenkov]

I'm still interested in what @dtolnay was planning to write, but I'm going to close this issue because balanced angle brackets turned out to not be an acceptable solution for the turbofish problem (rust-lang/rfcs#2544 (comment)).