Provide a json output format of `cargo package --list`

[messense]

Problem

Currently cargo package --list outputs filenames line by line, it works fine normally. But when Cargo.toml contains metadata that uses relative path to point to parent directory, there is no way to get that information. For example readme = "../README.md:

[package]
readme = "../README.md"

cargo package --list outputs

README.md

instead of

../README.md

It makes sense because it's the filesystem layout in the .crate file, but for downstream tooling like maturin it doesn't know that the READMD.md is actually coming from ../README.md, which causes PyO3/maturin#1431

Proposed Solution

Add a json output format, for example cargo package --list=json outputs something like

{
  "README.md": "../README.md",
  "src/lib.rs": "src/lib.rs"
}

as proposed by @konstin in PyO3/maturin#1442 (comment)

Notes

No response

[messense]

Related to #11597, but CARGO_PKG_README isn't usable outside of Cargo build process.

cc @weihanglo

[weihanglo]

Thanks for the proposal!

Although this doesn't sound too hard to implement, we should still make sure that we have thought of different workarounds and designs before moving on.

For workarounds, off the top of my head there are at least two:

• cargo metadata | jq '.packages[] | select(.name == "<package-name>") | .readme' to get the path.
• Make the README.md file a symlink. Cargo takes care of symlinks and copies over.

The second one is definitely a feasible option I feel like.

For the design of this proposal, I'd like to know apart from the issue from maturin, is there any other issue that this proposal can resolve? One bug could probably be fixed with this proposal: cargo package --list doesn't escape or quote filenames. If a path contains newlines it becomes hard to parse by machines. The other question is that, do people want to know more on filesize, file permission, and many more? What information should be included in that JSON output? We should also make sure that we don't push the responsibility of cargo package too far, as it is first served as “listing files being packaged”.

[konstin]

For me, this isn't specifically about the readme, but more generally about allowing an external to create a .crate like package. We need to e.g. include the readme, license files, Cargo.toml and Cargo.toml.orig. It would be really neat if cargo would provide a json interface to get everything you need for that similar to what cargo metadata currently does really nicely for the dependency tree; Basically, prepare writing the .crate, write all the generated files to the target directory and dump a path mapping as json. I don't think this needs any more information than just the path, when in doubt we can still stat the files.

While it would technically be possible to just read the .crate, that seems both rather wasteful and makes it difficult to combine with the files we need to add for python source distributions.

[messense]

While it would technically be possible to just read the .crate

We can't if there are path dependencies without a version key, Cargo refuses to package them.