Cargo fails to compile a crate if it's included multiple times at different paths

[sam-ka]

Problem

Cargo fails to compile a crate if it is dependent on a crate that is included multiple times at different paths. For example, you have crate a which depends on crates b and c. Both crates b and c depend on crate d but each includes it from a different path.

/a
/a/b
/a/b/d (identical to /a/c/d)
/a/c
/a/c/d (identical to /a/b/d)

The error shown is:

 error: package collision in the lockfile: packages  d v0.1.0 (/tmp/tmp.TfZoQkYrMp/a/b/d) and  d v0.1.0 (/tmp/tmp.TfZoQkYrMp/a/c/d) are different,  but only one can be written to lockfile unambiguously

This scenario manifests when you recursively include crates as git submodules and a single crate is indirectly included multiple times as a sub dependency (same crate, same version, different paths).

Steps

I've attached a bash script that can reproduce this issue automatically (see note # 3). It's short enough that it can also serve as a step-by-step guide for reproducing the issue manually. The steps needed to reproduce the bug are:

1. Setup crate A.
2. Setup crate B and C under A.
3. Have A depend on B and C.
4. Setup crate D under B.
5. Have B depend on D.
6. Copy crate D to C.
7. Have C depend on D.
8. Build crate A.

Possible Solution(s)

I'm not privy to Cargo's internals unfortunately. Sorry.

Notes

1. Using cargo's git directive (instead of path) is not an option due to the way our project is structured.
2. rustc handles this issue correctly: if the duplicate crates are identical, they are compiled without issue. If they differ, rustc asks for them to be differentiated appart by using -C metadata=.... This opens up the possibility of using rustc together with GNU make as a workaround, but I'd rather continue using Cargo instead.
3. In addition to reproducing the bug, the attached script also demonstrates how rustc handles the issue when used directly.

Output of cargo version: cargo 1.45.1 (f242df6 2020-07-22)
Host Platform : x86_64-unknown-linux-gnu

Attachments

• cargo-bug.zip

[sam-ka]

If you need more information about this bug or the bug reproduction script, please let me know.

[stephenctw]

Hi @sam-ka I am having exact the same problem. Do you currently have a workaround? Thanks.

[sam-ka]

@stephenctw While still using cargo? Not currently, Unfortunately.

A solution used to exist by presenting false information to cargo when it tries to access the lock file (using LD_PRELOAD to hijack open and read syscalls), but this workaround stopped working before I opened this issue.

The only solution I found was to side-step cargo and use rustc directly. It's not the most elegant solution, but it works.

[gitmalong]

The bug still exists with cargo 1.60.0-nightly.