Feature request: exact-only restore mode for target caches

[garysassano]

Summary

Add an opt-in exact-only restore mode for target/, using a separate cache object from Cargo home inputs.

The current broad restore-key fallback is useful for reuse, but it can repeatedly copy an older target/ tree into a new immutable cache entry after lockfile or manifest changes. Because Cargo artifacts encode versions, features, compiler options, fingerprints, and build-script state in hashed paths, the existing name-based cleanup can retain multiple generations for packages that remain in the dependency graph.

This can make each partial restore produce a larger cache object even though only one cache entry is restored per job.

Observed behavior

I observed a compressed cache grow through this copy-forward lineage:

206 MB -> 2.86 GB -> 4.15 GB -> 8.88 GB -> 12.79 GB -> 13.80 GB -> 13.89 GB

Subsequent exact hits restored an approximately 17.82 GB object before target caching was disabled. The intermediate staircase shows repeated partial restores producing progressively larger immutable objects rather than the backend merging entries.

In the 14 instrumented jobs from this incident lineage, there was one initial miss. Of the 13 subsequent restores, 3 were exact hits (23.1%) and 10 were prefix matches (76.9%). This is a small, incident-specific sample rather than a stable repository-wide hit rate, but it shows the trade-off clearly: exact-only target restoration would have avoided copy-forward in this period while providing target reuse on only 3 of those 13 restores.

In one representative job, restore and save took 9m54s while the build took 3m50s.

Exact hits do not enlarge an immutable entry, but a later lockfile or manifest change can partially restore it and materialize another combined target tree under a new exact key.

What appears to happen

The effective lifecycle is:

Cargo.lock or relevant manifests change
        |
        v
exact cache key changes
        |
        v
the environment-level restore key selects the newest older cache
        |
        v
Cargo writes artifacts for the new dependency/configuration state
        |
        v
cleanup retains multiple hashed generations for still-used package names
        |
        v
the combined target tree is saved under a new immutable exact key

The storage backend is not merging cache entries. It restores one object. The growth occurs inside the restored filesystem tree before that tree is archived as the next object.

Why the cleanup fix isn't sufficient

#375 correctly identified that timestamp cleanup stopped after the first immediate entry. PR #377 changed that return to continue and is now present on master.

That is a useful correctness fix, but it does not address this failure mode completely:

• the observed cache grew from 206 MB to 17.82 GB in less than the one-week age threshold
• age cleanup is heuristic rather than Cargo-fingerprint-aware
• nested hashed generations may be younger than the threshold
• there is no archive-size or generation limit
• the previous target tree is still copied forward after a partial-key restore

Proposed behavior

Split the cache into two independently restored objects and apply different restore policies:

1. Cargo home inputs such as ~/.cargo/registry and ~/.cargo/git retain the current exact key plus prefix fallback behavior.
2. Workspace target/ artifacts use the same exact dependency/configuration key but do not provide a restore key when exact-only mode is enabled.

Conceptually:

with:
  cache-targets: true
  target-restore-mode: exact

With exact-only target restoration:

lockfile/manifests unchanged -> exact target hit
lockfile/manifests changed   -> clean target, compile, save fresh target object

An exact-only target miss is a clean compilation, not a completely cold start. The separately cached Cargo registry and Git inputs still use prefix fallback, so dependency sources remain warm.

The existing prefix behavior can remain the default for compatibility:

with:
  target-restore-mode: prefix

Implementing this as two cache-provider restore/save operations is important. Keeping Cargo home and target/ in one cache object would not allow a partial Cargo-input restore without also restoring the old target tree.

The split should use new cache versions or namespaces so exact-only target restores cannot match existing combined Cargo-home-plus-target cache objects.

Why split objects are important

Using one object for Cargo inputs and target/ makes it difficult to apply different restore policies. Registry and Git data are naturally reusable across nearby lockfile states, while target/ is a mutable, fingerprinted build tree whose internal consistency is harder to establish after generations are combined.

An exact-only target policy avoids requiring the action to infer which hashed .rlib, .rmeta, fingerprint, or build-script generation is currently valid. The trade-off is a clean target compilation after relevant key changes, while Cargo registry and Git inputs remain reusable through their broader fallback policy.

This eliminates cross-key target copy-forward. It does not bound one exact target object, remove old backend objects, or eliminate archive processing on exact hits.

Additional diagnostics and safeguards

Independent of the restore-mode proposal, these would make growth easier to detect and contain:

• log target size and file count before and after cleanup
• report aggregate bytes and file counts removed by each cleanup rule at info level (individual deletion paths are already available through core.debug)
• surface cleanup failures as warnings: cleanup.ts currently contains multiple empty catch {} blocks, while save-phase failures are otherwise easy to miss at default verbosity
• provide a configurable maximum target archive size and skip saving when exceeded
• document that prefix-restored target caches can accumulate artifact generations after dependency, feature, or compiler-configuration changes

Existing workaround and alternatives

Plain actions/cache

Users can approximate exact-only target restoration today with a separate actions/cache step keyed by the exact dependency/configuration hash and no restore-keys. Native support would keep key construction, workspace discovery, target cleanup, cache-version migration, and Cargo-input caching within rust-cache.

Smarter fingerprint-aware pruning

This could preserve more reuse, but safely identifying active Cargo artifact generations requires understanding fingerprints, features, compiler settings, build scripts, dep-info, and nested hash layouts. Exact-only restoration avoids that inference.

Age or size pruning only

These policies can bound storage, but still require restoring the old object before pruning it.

Disable target caching

This eliminates target copy-forward but gives up all compiled dependency reuse. It remains the safest fallback when target archives are not economical.

AI disclosure

AI was used to analyze cache logs and draft this issue. The findings were manually reviewed against rust-cache v2.9.2 and current master.