feat(publish): Stabilize multi-package publishing #15636
Merged
+156
−860
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
r? @weihanglo rustbot has assigned @weihanglo. Use |
rustbot
added
A-cli
|
@rfcbot fcp merge See the PR description for details |
|
Team member @epage has proposed to merge this. The next step is review by the rest of the tagged team members: No concerns currently listed. Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up! See this document for info about what commands tagged team members can give me. |
rfcbot
added
proposed-final-comment-period
|
I have gotten a reproduction case for 1 of the 2 checksum issues, see #15647. |
epage
force-pushed
the
stabilize-workspace-package
branch
from
a6c8597 to
5a2230e
Compare
|
As an update, #15647 is being fixed in #15711. Both are scoped only for Also, I forgot to note #15622 in the stabilization report. While it can be reproduced with consecutive |
rfcbot
added
final-comment-period
|
🔔 This is now entering its final comment period, as per the review above. 🔔 |
A user will now be able to use flags like `--workspace` with `cargo publish`. `cargo package` will now also work with those flags without having to pass `--no-verify --exclude-lockfile`. Many release tools have come out that solve this problem. They will still need a lot of the logic that went into that for other parts of the release process. However, a cargo-native solution allows for: - Verification during dry-run - Better strategies for waiting for the publish timeout `cargo publish` is non-atomic at this time. If there is a server side error, network error, or rate limit during the publish, the workspace will be left in a partially published state. Verification is done before any publishing so that won't affect things. There are multiple strategies we can employ for improving this over time, including - atomic publish - `--idempotent` (rust-lang#13397) - leave this to release tools to manage This includes support for `--dry-run` verification. As release tools didn't have a way to do this before, users may be surprised at how slow this is because a `cargo build` is done instead of a `cargo check`. This is being tracked in rust-lang#14941. This adds to `cargo package` the `--registry` and `--index` flags to help with resolving dependencies when depending on a package being packaged at that moment. These flags are only needed when a `cargo package --workspace` operation would have failed before due to inability to find a locally created dependency. Regarding the publish timeout, `cargo publish --workspace` publishes packages in batches and we only timeout if nothing in the batch has finished being published within the timeout, deferring the rest to the next wait-for-publish. So for example, if you have packages `a`, `b`, `c` then we'll wait up to 60 seconds and if only `a` and `b` were ready in that time, we'll then wait another 60 seconds for `c`. During testing, users ran into issues with `.crate` checksums that we've not been able to reproduce since: - rust-lang#1169 (comment) - rust-lang#14396 By stabilizing this, Cargo's behavior becomes dependent on an overlay registry. When generating a lockfile or verifying a package, we overlay the locally generated `.crate` files on top of the registry so the registry appears as it would and everything works. If there is a conflict with a version, the local version wins which is important for the dry-run mode of release tools as they won't have bumped the version yet. Our concern for the overlay registry is dependency confusion attacks. Considering this is not accessible for general user operations, this should be fine. Fixes rust-lang#1169 Fixes rust-lang#10948
epage
force-pushed
the
stabilize-workspace-package
branch
2 times, most recently
from
262ffc6 to
d2f7220
Compare
rfcbot
added
finished-final-comment-period
|
The final comment period, with a disposition to merge, as per the review above, is now complete. As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed. This will be merged soon. |
weihanglo
approved these changes
Jul 7, 2025
github-merge-queue
bot
removed this pull request from the merge queue due to failed status checks
epage
force-pushed
the
stabilize-workspace-package
branch
2 times, most recently
from
14f4057 to
c8bf409
Compare
|
@weihanglo CI is green again |
rustbot
removed
the
S-waiting-on-review
bors
added a commit
to rust-lang/rust
that referenced
this pull request
Jul 12, 2025
Update cargo 14 commits in 930b4f62cfcd1f0eabdb30a56d91bf6844b739bf..eabb4cd923deb73e714f7ad3f5234d68ca284dbe 2025-06-28 14:58:43 +0000 to 2025-07-09 22:07:55 +0000 - feat: Implementation and tests for `multiple-build-scripts` (rust-lang/cargo#15704) - perf: Speed up TOML parsing by upgrading toml (rust-lang/cargo#15736) - Mark cachelock tests that rely on interprocess blocking behaviour as unsupported on AIX. (rust-lang/cargo#15734) - feat(publish): Stabilize multi-package publishing (rust-lang/cargo#15636) - Update to Rust 2024 (rust-lang/cargo#15732) - Clarify package ID specifications in SBOMs are fully qualified (rust-lang/cargo#15731) - chore(deps): update cargo-semver-checks to v0.42.0 (rust-lang/cargo#15730) - test: Switch config tests to use snapshots (rust-lang/cargo#15729) - implement package feature unification (rust-lang/cargo#15684) - chore: Upgrade dependencies (rust-lang/cargo#15722) - Report valid file name when we can't find a build target for `name = "foo.rs"` (rust-lang/cargo#15707) - chore(release): Publish build-rs on release (rust-lang/cargo#15708) - Override `Cargo.lock` checksums when doing a dry-run `publish` (rust-lang/cargo#15711) - test(rustfix): Update for nightly (rust-lang/cargo#15717) r? ghost
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR try to resolve?
A user will now be able to use flags like
--workspacewithcargo publish.cargo packagewill now also work with those flags without having to pass--no-verify --exclude-lockfile.Many release tools have come out that solve this problem. They will still need a lot of the logic that went into that for other parts of the release process.
However, a cargo-native solution allows for:
cargo publishis non-atomic at this time.If there is a server side error, network error, or rate limit during the publish, the workspace will be left in a partially published state. Verification is done before any publishing so that won't affect things. There are multiple strategies we can employ for improving this over time, including
--idempotent(Want cargo publish --idempotent #13397)This includes support for
--dry-runverification. As release tools didn't have a way to do this before, users may be surprised at how slow this is because acargo buildis done instead of acargo check. This is being tracked in #14941.This adds to
cargo packagethe--registryand--indexflags to help with resolving dependencies when depending on a package being packaged at that moment.These flags are only needed when a
cargo package --workspaceoperation would have failed before due to inability to find a locally created dependency.Regarding the publish timeout,
cargo publish --workspacepublishes packages in batches and we only timeout if nothing in the batch has finished being published within the timeout, deferring the rest to the next wait-for-publish. So for example, if you have packagesa,b,cthen we'll wait up to 60 seconds and if onlyaandbwere ready in that time, we'll then wait another 60 seconds forc.During testing, users ran into issues with
.cratechecksums:cargo publish --dry-run -Zpackage-workspacereports the checksum has changed #15647cargo publish --dry-runin OverrideCargo.lockchecksums when doing a dry-runpublish#15711cargo packagestill has the problem-Zpackage-workspace#14396 (not been able to reproduce)cargo publishcalls)Fixes #1169
Fixes #10948
How to test and review this PR?
By stabilizing this, Cargo's behavior becomes dependent on an overlay registry.
When generating a lockfile or verifying a package, we overlay the locally generated
.cratefiles on top of the registry so the registry appears as it would and everything works.If there is a conflict with a version, the local version wins which is important for the dry-run mode of release tools as they won't have bumped the version yet.
Our concern for the overlay registry is dependency confusion attacks. Considering this is not accessible for general user operations, this should be fine.