Use a proper lexer for hiding code in doctests

[jyn514]

Right now, rustdoc uses line-by-line heuristics to tell if a line should be hidden. That has false positives with multi-line strings, e.g.

let s = "hello
# because why not?";

and soon, after #84445,

let s = "hello
#because why not?";

It would be nice to use a proper lexer so we can detect where strings start and end instead of having to do custom "parsing".

Originally posted by @GuillaumeGomez in #84445 (comment)

[SergioBenitez]

Please note that this is a breaking change. What's more, this change would make it impossible to hide a multi-line string in a doc comment:

# let s = "hello\
# there";

Please see my suggestion in #84502 (comment) which doesn't suffer from this issue.

[jyn514]

I don't think this is important enough to spend time on an RFC.