Tracking Issue for garbage collection

[ehuss]

Summary

Original proposal: https://hackmd.io/@rust-cargo-team/SJT-p_rL2
Implementation: #12634
Documentation: https://doc.rust-lang.org/nightly/cargo/reference/unstable.html#gc
Issues: Z-gc Nightly: garbage collection

The -Zgc flag enable garbage collection for deleting old, unused files in cargo's cache.

Status

• Tracking of last-use data started in Rust 1.78 via Stabilize global cache data tracking. #13492.
• Automatic cleaning of unused data started in Rust 1.88 via Stabilize automatic garbage collection. #14287.
• Manual cleaning is tracked in gc: Determine CLI design for manual cleaning #13060.
• What kind of config options to expose is tracked in gc: Determine which config options should be exposed and their defaults #13061.
• Build artifact tracking and cleaning is tracked in Garbage collect whole target/ #13136.

Unresolved Issues

• Finalize the CLI
  • Enumerate prior art from other tools with cache management (poetry, docker, git, nix, zig, dune, etc)
    • Apparently buck2 and Turborepo grow unbounded

Future Extensions

No response

About tracking issues

Tracking issues are used to record the overall progress of implementation.
They are also used as hubs connecting to other relevant issues, e.g., bugs or open design questions.
A tracking issue is however not meant for large scale discussion, questions, or bug reports about a feature.
Instead, open a dedicated issue for the specific matter and add the relevant feature gate label.

[epage]

For when we get to target/:

From @bjorn3 at https://hachyderm.io/@bjorn3/111047792430714997

The incr comp cache is GCed on every compilation by only copying artifacts that were used for the current compilation session to the new incr comp cache dir and then removing the old incr comp cache dir entirely. In effect it is a maximally eager semi-space garbage collector. This is not suitable for cargo at all because alternating between cargo build -p foo and cargo build -p bar is guaranteed to rebuild things every time as each build would delete the artifacts of the other build.

[epage]

Quick scan of brew

• autoremove
• cleanup
  • "Removes all downloads more than 120 days old. This can be adjusted with HOMEBREW_CLEANUP_MAX_AGE_DAYS."
• HOMEBREW_CLEANUP_PERIODIC_FULL_DAYS (default 30 days)

One complaint that came up was " "brew cleanup has not been run in 30 days, running now" ... and then proceeds to run an interminable process in the middle of you attempting to do something else." (mastadon)

[epage]

From https://hachyderm.io/@nathanhammond@mastodon.social/111048319933010803

Also: global vs. local? Just keep all things local. Errant cache hits pulling from a non-scoped global cache are a disaster. Errant cache hits from the same local cache are a "whoops." For shared artifacts (e.g. a singular tar downloaded from the registry), global is fine, and should just grow unbounded. (Seriously: nobody cares.)

pnpm, bun, npm, yarn: package cache is intended to be shared global immutable, unbounded growth.

I'm assuming our global package cache is to help with CI but we should probably explicitly document out priority use cases.

[mcclure]

Is -Zgc the final planned CLI interface?

To my mind clearing the global cache is a "verb" so it being something like "cargo clean-cache" would be discoverable but "-Zgc" (to what command?) would be less so.

In addition, it would be nice to have the option of fully deleting the global cache through the CLI in addition to just GCing it (you may say "but you just delete ~/.cargo", but if one is using Windows it is not intuitively obvious whether a cache would be stored in \Users\username, %APPDATA%, or one of a couple other locations)

[epage]

Unstable cargo features require two things

• Nightly rust
• Being enabled

-Zgc is the flag for enabling support for this unstable feature but it isn't the interface for the unstable feature (in some cases, we do make them overlap but that isn't that common).

#12634 has an interface but my plan is to avoid bike shedding that when I review it just so we get the infrastructure in and then we can worry about both the infrastructure and the CLI as we work towards stabilization.

In addition, it would be nice to have the option of fully deleting the global cache through the CLI in addition to just GCing it (you may say "but you just delete ~/.cargo", but if one is using Windows it is not intuitively obvious whether a cache would be stored in \Users\username, %APPDATA%, or one of a couple other locations)

#3289 is the more specific issue for that. Depending on the controls offered, this is usually supported directly or by saying things like "reduce cache size to 0B".

[mcclure]

Thank you for explaining, I apologize for my confusion.