`cargo package --list` shows different whether `.git` is present or not

[marcoieni]

Problem

cargo package --list behavior is inconsistent:

• If .git is present, then it contains files starting with . (i.e. hidden files)
• If .git is not present, then it doesn't contain hidden files

Steps

1. cargo new hello
2. cd hello
3. touch .emptyfile optional step to show that even hidden files unrelated to git are ignored
4. git add . && git commit -m "init"
5. cargo package --list

warning: manifest has no description, license, license-file, documentation, homepage or repository.
See https://doc.rust-lang.org/cargo/reference/manifest.html#package-metadata for more info.
.cargo_vcs_info.json
.emptyfile
.gitignore
Cargo.lock
Cargo.toml
Cargo.toml.orig
src/main.rs

As you can see, hidden files are present in the output. Now let's remove the git directory.

7. rm -rf .git
8. cargo package --list

warning: manifest has no description, license, license-file, documentation, homepage or repository.
See https://doc.rust-lang.org/cargo/reference/manifest.html#package-metadata for more info.
Cargo.lock
Cargo.toml
Cargo.toml.orig
src/main.rs

I would expect the output of the two commands to be the same. Why do we ignore hidden files if .git is not present? It is confusing.

Possible Solution(s)

Always include hidden files, even if .git is not present. I.e. don't differentiate between the two cases (.git present and not present): the behavior should be the same.

Notes

I care about this because, for release-plz, I need to compare the git repository with the crate downloaded from the cargo registry, which doesn't contain the .git directory. I use cargo package --list to compare if the two versions of the crate are the same.

As a workaround I'm thinking of running git init && git add . && git commit "init" in the crate downloaded from the registry, so that the outcome of cargo package --list is the same.

Version

$ cargo version --verbose
cargo 1.77.1 (e52e36006 2024-03-26)
release: 1.77.1
commit-hash: e52e360061cacbbeac79f7f1215a7a90b6f08442
commit-date: 2024-03-26
host: aarch64-apple-darwin
libgit2: 1.7.2 (sys:0.18.2 vendored)
libcurl: 8.4.0 (sys:0.4.70+curl-8.5.0 system ssl:(SecureTransport) LibreSSL/3.3.6)
ssl: OpenSSL 1.1.1w  11 Sep 2023
os: Mac OS 14.4.1 [64-bit]

[weihanglo]

The root cause is pretty much the same as this one: #13662.

Without a VCS presence, Cargo walks through the file system with some heuristics like ignoring hidden files.

[marcoieni]

I see.
Personally, I don't like this heuristics. I would love to see it removed as it is confusing. 😕

However, If this is the intended behavior or you don't want to change the current behavior, then let's document it in the cargo book and close this issue.

[weihanglo]

That is well understood. Personally I also don't like Cargo depending on one VCS that much, though removing the behavior is kinda a breaking change to some workflows. When things are about hidden files and publish, that become more interesting. We don't want to accidentally publish users' private data or even private keys.

The filesystem walk also relies on user-level .gitignore. That was also discussed during the integration of gitoxide #13592 (comment), which might exacerbate the situation you're in.

We might need a complete deprecation plan pushing people away from this behavior, yet I haven't thought too hard on this.

[marcoieni]

ok, I understand the situation, thanks. Feel free to close this issue if you don't need it open 👍

[ehuss]

The cargo team discussed this today and decided to close. We would prefer to not change the behavior, since that would be a breaking change, and had risks as mentioned above. Some comments:

• If you have the crate from the registry, then listing the contents of the crate should be the list of files that is equivalent to cargo package --list. We weren't sure if that would help with your use case.
• The behavior of with or without git is documented at https://doc.rust-lang.org/cargo/reference/manifest.html#the-exclude-and-include-fields.