Skip to content

no-op remap_path_prefix change invalidates incremental cache #132132

Open

Description

I'm building rust code in a CI environment, setting -Cincremental to a shared location. CI allocates a new working directory for each build. To prevent leaking that info to the debug info, we set --remap_path_prefix=${PWD}= to eliminate the unwanted prefix from the debug info.

rustc src/lib.rs \
  --crate-name=xxx \
  --crate-type=rlib \
  --error-format=human \
  --out-dir=${OUTDIR} \
  --remap-path-prefix=${PWD}= \
  --emit=dep-info,link \
  -Ccodegen-units=256 \
  -Zincremental_info \
  -Cincremental=${CACHEDIR}

I expected this will reuse the incremental build cache, but it's not:

[incremental] session directory: 259 files hard-linked
[incremental] session directory: 0 files copied
[incremental] completely ignoring cache because of differing commandline arguments

Further inspection of how the hash being generated, I found:

/// Remap source path prefixes in all output (messages, object files, debug, etc.).
remap_path_prefix: Vec<(PathBuf, PathBuf)> [TRACKED_NO_CRATE_HASH],
/// Base directory containing the `src/` for the Rust standard library, and
/// potentially `rustc` as well, if we can find it. Right now it's always
/// `$sysroot/lib/rustlib/src/rust` (i.e. the `rustup` `rust-src` component).
///
/// This directory is what the virtual `/rustc/$hash` is translated back to,
/// if Rust was built with path remapping to `/rustc/$hash` enabled
/// (the `rust.remap-debuginfo` option in `config.toml`).
real_rust_source_base_dir: Option<PathBuf> [TRACKED_NO_CRATE_HASH],
edition: Edition [TRACKED],
/// `true` if we're emitting JSON blobs about each artifact produced
/// by the compiler.
json_artifact_notifications: bool [TRACKED],
/// `true` if we're emitting a JSON blob containing the unused externs
json_unused_externs: JsonUnusedExterns [UNTRACKED],
/// `true` if we're emitting a JSON job containing a future-incompat report for lints
json_future_incompat: bool [TRACKED],
pretty: Option<PpMode> [UNTRACKED],
/// The (potentially remapped) working directory
working_dir: RealFileName [TRACKED],
color: ColorConfig [UNTRACKED],

for our case, working_dir always uses remapped_path "", which is stable over build. but the remap_path_prefix is different everytime. and the logic for populating working_dir is at:

let working_dir = std::env::current_dir().unwrap_or_else(|e| {
early_dcx.early_fatal(format!("Current directory is invalid: {e}"));
});
let file_mapping = file_path_mapping(remap_path_prefix.clone(), &unstable_opts);
let working_dir = file_mapping.to_real_filename(&working_dir);

So to get correct remapped working_dir, I must set --remap-path-prefix, but setting it will change remap_path_prefix and causing hash mismatch:

if prev_commandline_args_hash != expected_hash {
if sess.opts.unstable_opts.incremental_info {
eprintln!(
"[incremental] completely ignoring cache because of \
differing commandline arguments"
);

Meta

rustc --version --verbose:

rustc 1.82.0-nightly (f6e511eec 2024-10-15)
binary: rustc
commit-hash: f6e511eec7342f59a25f7c0534f1dbea00d01b14
commit-date: 2024-10-15
host: aarch64-apple-darwin
release: 1.82.0-nightly
LLVM version: 19.1.1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

No one assigned

    Labels

    A-debuginfoArea: Debugging information in compiled programs (DWARF, PDB, etc.)A-incr-compArea: Incremental compilationA-path-remappingArea: path remapping, --remap-path-prefix, --remap-cwd-prefix, --remap-diagnostics-scope etc.C-feature-requestCategory: A feature request, i.e: not implemented / a PR.T-compilerRelevant to the compiler team, which will review and decide on the PR/issue.WG-incr-compWorking group: Incremental compilation

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions