`cargo vendor --respect-source-config` deletes previously vendored sources

[lambdageek]

Problem

Running cargo vendor > .cargo/config.toml followed by running cargo vendor --respect-source-config results in a failure. Moreover cargo vendor deletes the previous contents of the vendor folder.

% cargo vendor --versioned-dirs --respect-source-config --no-delete
error: failed to sync

Caused by:
  failed to load pkg lockfile

Caused by:
  no matching package named `bytemuck` found
  location searched: directory source `/Users/alklig/work/bad_vendor/vendor` (which is replacing registry `crates-io`)
  required by package `bad_vendor v0.1.0 (/Users/alklig/work/bad_vendor)`

Steps

1. clone https://github.com/lambdageek/bad-vendor
2. cargo vendor --versioned-dirs --respect-source-config --no-delete > .cargo/config.toml
3. cargo vendor --versioned-dirs --respect-source-config --no-delete

Possible Solution(s)

I suspect something in the loop trying to deal with #5956 is not expecting to see replaced sources. In particular, I wonder if this line is deleting files in the "vendor" directory

cargo/src/cargo/ops/vendor.rs

Line 137 in 3fe24ab

drop(fs::remove_dir_all(pkg.root()));

Notes

No response

Version

cargo 1.84.1
release: 1.84.1
host: aarch64-apple-darwin
libgit2: 1.8.1 (sys:0.19.0 vendored)
libcurl: 8.7.1 (sys:0.4.74+curl-8.9.0 system ssl:(SecureTransport) LibreSSL/3.3.6)
ssl: OpenSSL 1.1.1w  11 Sep 2023
os: Mac OS 15.3.1 [64-bit]

[weihanglo]

I suspect something in the loop trying to deal with #5956 is not expecting to see replaced sources. In particular, I wonder if this line is deleting files in the "vendor" directory

The observation is mostly correct, but I don't think it is intended to use like that. It tried to vendor crates from a source location that is exactly the same as the destination. It sounds pretty awkward. However I do think it is a bug. Cargo shouldn't delete anything that is not a cache. That means Cargo should only delete crates from registry sources but not other kinds of sources.

[lambdageek]

It tried to vendor crates from a source location that is exactly the same as the destination.

There's something more going on. The following also deletes files from vendor_dist1:

$ cargo vendor --versioned-dirs --respect-source-config --no-delete vendor_dist1 > .cargo/config.toml
# ... success ...
$ ls vendor_dist1
bytemuck-1.21
$ cargo vendor --versioned-dirs --respect-source-config --no-delete vendor_dist2 
rror: failed to sync

Caused by:
  failed to load pkg lockfile

Caused by:
  no matching package named `bytemuck` found
  location searched: directory source `/tmp/bad-vendor/vendor_dist1` (which is replacing registry `crates-io`)
  required by package `bad_vendor v0.1.0 (/tmp/bad-vendor)`

after cargo vendor ... vendor_dist2 the directory cargo_dist1 is empty

[weihanglo]

Yeah it has pretty much the same root cause. As I said,

Cargo shouldn't delete anything that is not a cache. That means Cargo should only delete crates from registry sources but not other kinds of sources.

Cargo doesn't check that, so adding a check for it should suffice.

[weihanglo]

Hmm… maybe not as easy as I thought. Might also need to check if it is a source replacement.