`cargo test --all-targets` does not work exactly as advertised

[infinity0]

Dummy crate, from test::test_then_build:

$ cat src/lib.rs 
#[test]
fn foo() {}
$ cat Cargo.toml 
[package]
name = "foo"
version = "0.0.1"
authors = []

Running cargo test --all-targets seems to test even less than cargo test.

$ cargo test
   Compiling foo v0.0.1 (file://$PWD)
    Finished dev [unoptimized + debuginfo] target(s) in 0.36 secs
     Running target/debug/deps/foo-32de61ba59a5bc99
running 1 test
[..]
   Doc-tests foo
running 0 tests
$ cargo test --all-targets
    Finished dev [unoptimized + debuginfo] target(s) in 0.0 secs
     Running target/debug/deps/foo-32de61ba59a5bc99
running 1 test
[..]
     Running target/debug/deps/foo-32de61ba59a5bc99
running 1 test
[..]
     Running target/debug/deps/foo-32de61ba59a5bc99

Affects at least cargo 0.24.0 all the way up to current HEAD.

Uncovered by proposed additions to #5146.

[infinity0]

Looks like the default case (no --all-targets) is handled by generate_auto_targets whereas the --all-target case is handled as a special case of CompileFilter::Only branch in generate_targets. Not sure of the best way to fix this, a clean solution probably needs a significant re-architecturing of the surrounding code and data structures.

[stale]

As there hasn't been any activity here in over 6 months I've marked this as stale and if no further activity happens for 7 days I will close it.

I'm a bot so this may be in error! If this issue should remain open, could someone (the author, a team member, or any interested party) please comment to that effect?

The team would be especially grateful if such a comment included details such as:

• Is this still relevant?
• If so, what is blocking it?
• Is it known what could be done to help move this forward?

Thank you for contributing!

If you're reading this comment from the distant future, fear not if this was closed automatically. If you believe it's still an issue please leave a comment and a team member can reopen this issue. Opening a new issue is also acceptable!

[ehuss]

Yea, --all-targets is not working very well. Running the lib test three times is fairly straightforward (it is incorrectly picked up as a --lib, --tests, and --benches). Adding doc-tests should also be easier to do now since I recently changed how CompileMode::Doctest units are created. I'll try to get to this soon (and double-check --all-targets for other commands).