`cargo tree` panics with "activated_features for invalid package: features did not find PackageId" when a binary dependency is specified with an explicit `target` key

[bstrie]

Problem

For convenience I have also uploaded the failing code here: https://github.com/bstrie/bindeperror4

For the following code:

# Cargo.toml
[package]
name = "mycrate"
version = "0.0.0"
edition = "2021"

[dependencies]
mybindep = { path = "mybindep", artifact = "bin", target = "x86_64-unknown-linux-gnu" }

// src/main.rs
fn main() {
    env!("CARGO_BIN_FILE_MYBINDEP");
}

# mybindep/Cargo.toml
[package]
name = "mybindep"
version = "0.0.0"
edition = "2021"

// mybindep/src/main.rs
fn main() {}

Attempting to use cargo tree produces the following:

thread 'main' panicked at 'activated_features for invalid package: features did not find PackageId { name: "mybindep", version: "0.0.0", source: "/home/ben/code/scrap/bindeperror4/mybindep" } NormalOrDevOrArtifactTarget(None)', src/tools/cargo/src/cargo/core/resolver/features.rs:318:14

Expand for full `cargo tree` backtrace.

thread 'main' panicked at 'activated_features for invalid package: features did not find PackageId { name: "mybindep", version: "0.0.0", source: "/home/ben/code/scrap/bindeperror4/mybindep" } NormalOrDevOrArtifactTarget(None)

Stack backtrace:
   0: <anyhow::Error>::msg::<alloc::string::String>
   1: <cargo::core::resolver::features::ResolvedFeatures>::activated_features_int
   2: cargo::ops::tree::graph::add_pkg
   3: cargo::ops::tree::graph::add_pkg
   4: cargo::ops::tree::graph::build
   5: cargo::ops::tree::build_and_print
   6: cargo::commands::tree::exec
   7: cargo::cli::main
   8: cargo::main
   9: std::sys_common::backtrace::__rust_begin_short_backtrace::<fn(), ()>
  10: std::rt::lang_start::<()>::{closure#0}
  11: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once
             at /rustc/de1bc0008be096cf7ed67b93402250d3b3e480d0/library/core/src/ops/function.rs:280:13
  12: std::panicking::try::do_call
             at /rustc/de1bc0008be096cf7ed67b93402250d3b3e480d0/library/std/src/panicking.rs:492:40
  13: std::panicking::try
             at /rustc/de1bc0008be096cf7ed67b93402250d3b3e480d0/library/std/src/panicking.rs:456:19
  14: std::panic::catch_unwind
             at /rustc/de1bc0008be096cf7ed67b93402250d3b3e480d0/library/std/src/panic.rs:137:14
  15: std::rt::lang_start_internal::{{closure}}
             at /rustc/de1bc0008be096cf7ed67b93402250d3b3e480d0/library/std/src/rt.rs:128:48
  16: std::panicking::try::do_call
             at /rustc/de1bc0008be096cf7ed67b93402250d3b3e480d0/library/std/src/panicking.rs:492:40
  17: std::panicking::try
             at /rustc/de1bc0008be096cf7ed67b93402250d3b3e480d0/library/std/src/panicking.rs:456:19
  18: std::panic::catch_unwind
             at /rustc/de1bc0008be096cf7ed67b93402250d3b3e480d0/library/std/src/panic.rs:137:14
  19: std::rt::lang_start_internal
             at /rustc/de1bc0008be096cf7ed67b93402250d3b3e480d0/library/std/src/rt.rs:128:20
  20: main
  21: <unknown>
  22: __libc_start_main
  23: <unknown>', src/tools/cargo/src/cargo/core/resolver/features.rs:318:14
stack backtrace:
   0: rust_begin_unwind
             at /rustc/de1bc0008be096cf7ed67b93402250d3b3e480d0/library/std/src/panicking.rs:584:5
   1: core::panicking::panic_fmt
             at /rustc/de1bc0008be096cf7ed67b93402250d3b3e480d0/library/core/src/panicking.rs:142:14
   2: core::result::unwrap_failed
             at /rustc/de1bc0008be096cf7ed67b93402250d3b3e480d0/library/core/src/result.rs:1785:5
   3: cargo::ops::tree::graph::add_pkg
   4: cargo::ops::tree::graph::add_pkg
   5: cargo::ops::tree::graph::build
   6: cargo::ops::tree::build_and_print
   7: cargo::commands::tree::exec
   8: cargo::cli::main
   9: cargo::main

Curiously the panic message is very similar to the panic reported in #10431 . Since that issue has already been fixed, it's possible that the solution for this issue may be similar to the solution for that one.

Note that the following diff causes cargo tree to run as expected:

- mybindep = { path = "mybindep", artifact = "bin", target = "x86_64-unknown-linux-gnu" }
+ mybindep = { path = "mybindep", artifact = "bin" }

So the presence of the explicit target key is crucial here. This is similar to the circumstances of #10526 and #10525 , so it further suggests something interesting about how the target key is being handled. It may be that fixing one of these will fix all of these.

cc @Byron

Version

cargo 1.62.0-nightly (edffc4a 2022-04-19)
release: 1.62.0-nightly
commit-hash: edffc4ada3d77799e5a04eeafd9b2f843d29fc23
commit-date: 2022-04-19
host: x86_64-unknown-linux-gnu
libgit2: 1.4.2 (sys:0.14.2 vendored)
libcurl: 7.80.0-DEV (sys:0.4.51+curl-7.80.0 vendored ssl:OpenSSL/1.1.1m)
os: Pop!_OS 21.10 (impish) [64-bit]

[bstrie]

For now, note that the original cargo-tree tool still works and doesn't have this problem: https://github.com/sfackler/cargo-tree

[weihanglo]

This issue can be fixed by duplicating the logic of resolver to determine the FeatureFor artifact deps. The following patch demonstrates it:

Click to view the patch

diff --git a/src/cargo/ops/tree/graph.rs b/src/cargo/ops/tree/graph.rs
index 20a9ca0b6..c975e6fcc 100644
--- a/src/cargo/ops/tree/graph.rs
+++ b/src/cargo/ops/tree/graph.rs
@@ -387,10 +387,38 @@ fn add_pkg(
         let dep_pkg = graph.package_map[&dep_id];
 
         for dep in deps {
-            let dep_features_for = if dep.is_build() || dep_pkg.proc_macro() {
-                FeaturesFor::HostDep
-            } else {
-                features_for
+            // This is somehow similar to
+            // <https://github.com/rust-lang/cargo/blob/5181f995d08e72da457bf8a2e6c9c8d0bba09088/src/cargo/core/resolver/features.rs#L800-L852>
+            // without the consideration of `dep = { …, lib = true }`
+            //
+            // TODO: decide how to display artifact deps within cargo-tree,
+            // including handling artifact deps with `{ …, lib = true }`
+            let dep_features_for = match dep
+                .artifact()
+                .and_then(|a| a.target())
+                .and_then(|t| t.to_resolved_compile_target(requested_kind))
+            {
+                // Dependency has a `{ …, target = <triple> }`
+                Some(target) => FeaturesFor::ArtifactDep(target),
+                // Get the information of the dependent crate from `features_for`.
+                // If a dependent crate is
+                //
+                // * specified as an artifact dep with a `target`, or
+                // * a host dep,
+                //
+                // its transitive deps, including build-deps, need to be built on that target.
+                None if features_for != FeaturesFor::default() => features_for,
+                // Dependent crate is a normal dep, then back to old rules:
+                //
+                // * normal deps, dev-deps -> inherited target
+                // * build-deps -> host
+                None => {
+                    if dep.is_build() || dep_pkg.proc_macro() {
+                        FeaturesFor::HostDep
+                    } else {
+                        features_for
+                    }
+                }
             };
             let dep_index = add_pkg(
                 graph,

However, I found another panic in cargo tree when it meets all the followings conditions at once:

• a dependency of an artifact dependency is platform specified
• artfiact dep itself is { target = <specified> } with the same platform as its own dependency
• the platform is not activated.

Here is a reproducer in Cargo's own test dialect:

Click to view the reproducer

#[cargo_test]
fn artifact_dep_target_specified() {
    if cross_compile::disabled() {
        return;
    }
    let alt = cross_compile::alternate();
    let p = project()
        .file(
            "Cargo.toml",
            &format!(
                r#"
                    [package]
                    name = "foo"
                    version = "0.1.0"

                    [dependencies]
                    bar = {{ path = "bar", artifact = "bin", target = "{alt}" }}
                "#,
            ),
        )
        .file("src/lib.rs", "")
        .file(
            "bar/Cargo.toml",
            &format!(
                r#"
                    [package]
                    name = "bar"
                    version = "0.1.0"

                    [target.{alt}.dependencies]
                    baz = {{ path = "../baz" }}
                "#,
            ),
        )
        .file("bar/src/lib.rs", "")
        .file(
            "baz/Cargo.toml",
            r#"
                [package]
                name = "baz"
                version = "0.1.0"
            "#,
        )
        .file("baz/src/lib.rs", "")
        .build();

    p.cargo("tree -Z bindeps")
        .masquerade_as_nightly_cargo(&["bindeps"])
        .with_stdout("") 
        .run();
    // thread 'main' panicked at 'no entry found for key', src/cargo/ops/tree/graph.rs:387:23
}

Personally, I don't think this should panic, as cargo tree is used only for displaying dependency information. However, I won't fix it and instead pause here until a broader issue gets resolved: How does Cargo display artifact dependencies in cargo tree properly? Some questions might be:

• Should we annotate artifact dependency nodes with a special (artifact deps) suffix?
• Should we show the information about which target platform an artifact dependency will be built on?
• How do we want to display the node of an artifact dependency with { …, lib = true }?

I haven't thought about it much. Will be back when I get more time. If anyone comes up with great ideas to show artifact dependencies in cargo tree, I am all ears!

[bstrie]

@weihanglo , my opinions for each question:

Should we annotate artifact dependency nodes with a special (artifact deps) suffix?

There are multiple possible values for the artifact field, so I wouldn't mind having a (bin) or (staticlib) or (cdylib) suffix.

Should we show the information about which target platform an artifact dependency will be built on?

No, I think that's a detail that can be easily answered by checking Cargo.toml. But if you did decide to display this information, I would include it as part of the above annotation, e.g. (wasm32-wasi bin) or (x86_64-unknown-linux-musl staticlib).

How do we want to display the node of an artifact dependency with { …, lib = true }?

I think it should show up multiple times in the tree graph. The lib node should show up as an entirely normal dependency, whereas the artifact node should be annotated as mentioned above.

[bstrie]

To elaborate on the above, I think it would be nice if users could think of this:

bar = { version = "*", artifact = "bin", lib = true }

...as though it were this:

bar_1 = { package = "bar", version = "*" }
bar_2 = { package = "bar", version = "*", artifact = "bin" }

...with the only difference that using the lib = "true" form doesn't force you to use renames via package = to avoid a collision. And since it's natural that the second example would have two lines in the cargo tree output, it seems natural that the first example should as well, and the (bin) bit is a convenient disambiguator.

[bstrie]

(And to elaborate even further, I think that #10837 suggests that the lib and bin artifacts from the exact same line can actually have different dependencies as a result of per-target feature unification, which is something that we'd want to reflect in cargo tree's output.)