rustdoc only: duplicate lang item `panic_halt`

[TimNN]

This is a regression from rustdoc 1.40.0 (73528e339 2019-12-16) to rustdoc 1.41.0-beta.3 (fb3056216 2020-01-19).

If rustdoc attempts to document a crate that has at least two dependencies which implement a panic_handler, it fails with an error similar to

error: duplicate lang item in crate `panic_halt` (which `panic2` depends on): `panic_impl`.
  |
  = note: first defined in crate `std` (which `panic2` depends on).

Note that the crate builds fine, it's just documenting that is broken.

---

Steps to reproduce

1. Create a new project, e.g. cargo new panic2
2. Add a dependency on panic-halt = "0.2.0", e.g. cargo add panic-halt
3. Attempt to document the crate, e.g. cargo doc

An example of the different behavior between `stable`/`beta` and `build`/`doc`.

$ cargo +stable clean

$ cargo +beta clean

$ cargo +stable build
   Compiling panic-halt v0.2.0
   Compiling panic2 v0.1.0 (/Users/logic/Projects/tmp/panic2)
    Finished dev [unoptimized + debuginfo] target(s) in 0.35s

$ cargo +stable doc
 Documenting panic-halt v0.2.0
    Checking panic-halt v0.2.0
 Documenting panic2 v0.1.0 (/Users/logic/Projects/tmp/panic2)
    Finished dev [unoptimized + debuginfo] target(s) in 1.40s

$ cargo +beta build
   Compiling panic-halt v0.2.0
   Compiling panic2 v0.1.0 (/Users/logic/Projects/tmp/panic2)
    Finished dev [unoptimized + debuginfo] target(s) in 0.27s

$ cargo +beta doc
 Documenting panic-halt v0.2.0
    Checking panic-halt v0.2.0
 Documenting panic2 v0.1.0 (/Users/logic/Projects/tmp/panic2)
error: duplicate lang item in crate `panic_halt` (which `panic2` depends on): `panic_impl`.
  |
  = note: first defined in crate `std` (which `panic2` depends on).

error: aborting due to previous error

error: Could not document `panic2`.

Caused by:
  process didn't exit successfully: `rustdoc --edition=2018 --crate-type bin --crate-name panic2 src/main.rs -o /Users/logic/Projects/tmp/panic2/target/doc --error-format=json --json=diagnostic-rendered-ansi --document-private-items -L dependency=/Users/logic/Projects/tmp/panic2/target/debug/deps --extern panic_halt=/Users/logic/Projects/tmp/panic2/target/debug/deps/libpanic_halt-6d0ca720c3c355fb.rmeta` (exit code: 1)

[Mark-Simulacrum]

cc @rust-lang/rustdoc

[GuillaumeGomez]

I'll try to take a look in the next days.

[pietroalbini]

Note that the stable release will be prepared on monday, so the patch should be merged by then.

[ollie27]

It looks like this was triggered by #66211 (cc. @kinnison). rustdoc now seems to load all crates passed by --extern even if they're unused in the code whereas rustc just ignores them. Adding extern crate panic_halt; or use panic_halt; to panic2 causes stable rustdoc and rustc to produce the same error message.

Is this issue affecting any real world code? It seems weird to list a dependency on panic-halt but not actually use it.

#66211 was to fix an ICE using the intra-doc-links feature but that's only available on nightly so it should be safe to revert #66211 on beta for now if needed.

[TimNN]

I guess it is unlikely that this will be triggered in many real world scenarios. I came across this problem because I was experimenting with embedded programming: I had multiple panic implementations in the dependencies and then multiple binaries which were using different panic implementations (to see their different behaviors).

[kinnison]

If there's a way to get the metadata into rustdoc without fully loading the --extern crates then that'd satisfy rustdoc's usecase without causing this. I do not object to backing this out of beta so long as there's some idea of how we'd fix it without entirely backing it out and reintroducing the ICE. @GuillaumeGomez how do you feel about that?

[GuillaumeGomez]

We can revert it on the beta to prevent the crash but we definitely need a way to get the metadata from the crates.

[GuillaumeGomez]

So after a discussion with @eddyb, it appears that @petrochenkov is one of the last person working on this part.

cc @petrochenkov (at @eddyb's suggestion)

[petrochenkov]

If you don't want to resolve all crates passed with --extern (with which I agree), then resolve paths in intra-doc links, the crates will be used on demand then.

All the paths need to be resolved before the Resolver data is frozen (to_resolver_outputs is called), so rustdoc should organize its passes accordingly.

[petrochenkov]

In other words, if you remove Resolver::clone_outputs while keeping rustdoc working, it will fix this issue and some other issues as well.