Paths emitted by build script are not corrected when cross-compiling

[ExpHP]

Posted on behalf of @Phaiax: (originally rust-lang/rust#55072)

I am trying some embedded rust programming and figured out that the cargo/rustc cache does not survive a move of the project folder. (This is the cortex-m-quickstart project but I think that does not really matter.)

There are no absolute paths neither in .cargo/config nor in Cargo.toml. After a cargo clean I can do a build again.

PS C:\Users\danie_000\Code\RE\ABC> rustc --version
rustc 1.30.0-beta.15 (590121930 2018-10-12)
PS C:\Users\danie_000\Code\RE\ABC> cargo --version
cargo 1.30.0-beta (05e9b01fc 2018-09-21)

The rust version is set by rustup default beta and not overwritten for any of these folders.

After the rename of the Project folder, cargo summons rustc with some references to the ABC path that does not exist anymore.

PS C:\Users\danie_000\Code\RE> cd .\ABC\
PS C:\Users\danie_000\Code\RE\ABC> cargo clean
PS C:\Users\danie_000\Code\RE\ABC> cargo build
   Compiling proc-macro2 v0.4.20
   Compiling unicode-xid v0.1.0
   Compiling rand_core v0.3.0
   Compiling cortex-m-rt v0.6.4
   Compiling cortex-m-semihosting v0.3.1
   Compiling cortex-m v0.5.7
   Compiling vcell v0.1.0
   Compiling myapp v0.1.0 (C:\Users\danie_000\Code\RE\ABC)
   Compiling bare-metal v0.2.3
   Compiling r0 v0.2.2
   Compiling aligned v0.2.0
   Compiling panic-halt v0.2.0
   Compiling rand_core v0.2.2
   Compiling volatile-register v0.2.0
   Compiling rand v0.5.5
   Compiling quote v0.6.8
   Compiling syn v0.15.11
   Compiling cortex-m-rt-macros v0.1.2
    Finished dev [unoptimized + debuginfo] target(s) in 28.77s
PS C:\Users\danie_000\Code\RE\ABC> cd ..
PS C:\Users\danie_000\Code\RE> mv ABC XYZ
PS C:\Users\danie_000\Code\RE> cd .\XYZ\
PS C:\Users\danie_000\Code\RE\XYZ> cargo -v build
       Fresh unicode-xid v0.1.0
       Fresh rand_core v0.3.0
       Fresh vcell v0.1.0
       Fresh r0 v0.2.2
       Fresh aligned v0.2.0
       Fresh bare-metal v0.2.3
   Compiling myapp v0.1.0 (C:\Users\danie_000\Code\RE\XYZ)
       Fresh panic-halt v0.2.0
     Running `rustc --edition=2018 --crate-name build_script_build build.rs --color always --crate-type bin --emit=dep-info,link -C debuginfo=2 -C metadata=3a754d4070599644 -C extra-filename=-3a754d4070599644 --out-dir C:\Users\danie_000\Code\RE\XYZ\target\debug\build\myapp-3a754d4070599644 -C incremental=C:\Users\danie_000\Code\RE\XYZ\target\debug\incremental -L dependency=C:\Users\danie_000\Code\RE\XYZ\target\debug\deps`
       Fresh rand_core v0.2.2
       Fresh volatile-register v0.2.0
       Fresh proc-macro2 v0.4.20
       Fresh rand v0.5.5
       Fresh cortex-m-semihosting v0.3.1
       Fresh cortex-m v0.5.7
       Fresh quote v0.6.8
       Fresh syn v0.15.11
       Fresh cortex-m-rt-macros v0.1.2
       Fresh cortex-m-rt v0.6.4
     Running `C:\Users\danie_000\Code\RE\XYZ\target\debug\build\myapp-3a754d4070599644\build-script-build`
     Running `rustc --edition=2018 --crate-name myapp src\main.rs --color always --crate-type bin --emit=dep-info,link -C debuginfo=2 -C metadata=7f9cdf275b22bd50 -C extra-filename=-7f9cdf275b22bd50 --out-dir C:\Users\danie_000\Code\RE\XYZ\target\thumbv7m-none-eabi\debug\deps --target thumbv7m-none-eabi -C incremental=C:\Users\danie_000\Code\RE\XYZ\target\thumbv7m-none-eabi\debug\incremental -L dependency=C:\Users\danie_000\Code\RE\XYZ\target\thumbv7m-none-eabi\debug\deps -L dependency=C:\Users\danie_000\Code\RE\XYZ\target\debug\deps --extern cortex_m=C:\Users\danie_000\Code\RE\XYZ\target\thumbv7m-none-eabi\debug\deps\libcortex_m-6d95ef6e4550c4c8.rlib --extern cortex_m_rt=C:\Users\danie_000\Code\RE\XYZ\target\thumbv7m-none-eabi\debug\deps\libcortex_m_rt-03e440f58d1c0555.rlib --extern cortex_m_semihosting=C:\Users\danie_000\Code\RE\XYZ\target\thumbv7m-none-eabi\debug\deps\libcortex_m_semihosting-26e2b9dfbd5c7e3b.rlib --extern panic_halt=C:\Users\danie_000\Code\RE\XYZ\target\thumbv7m-none-eabi\debug\deps\libpanic_halt-8d085ea03dce307c.rlib -C link-arg=-Tlink.x -L C:\Users\danie_000\Code\RE\XYZ\target\thumbv7m-none-eabi\debug\build\myapp-b0e16e01c22428a8\out -L C:\Users\danie_000\Code\RE\ABC\target\thumbv7m-none-eabi\debug\build\cortex-m-765dfe6812e223ce\out -L C:\Users\danie_000\Code\RE\ABC\target\thumbv7m-none-eabi\debug\build\cortex-m-rt-1387b5ad4b26cfa2\out -L C:\Users\danie_000\Code\RE\ABC\target\thumbv7m-none-eabi\debug\build\cortex-m-semihosting-cb2469f22b42a1d7\out`
error: linking with `rust-lld` failed: exit code: 1
  |
  = note: "rust-lld" "-flavor" "gnu" "-L" "C:\\Users\\danie_000\\.rustup\\toolchains\\beta-x86_64-pc-windows-msvc\\lib\\rustlib\\thumbv7m-none-eabi\\lib" "C:\\Users\\danie_000\\Code\\RE\\XYZ\\target\\thumbv7m-none-eabi\\debug\\deps\\myapp-7f9cdf275b22bd50.57siomzwzet2ykzc.rcgu.o" "-o" "C:\\Users\\danie_000\\Code\\RE\\XYZ\\target\\thumbv7m-none-eabi\\debug\\deps\\myapp-7f9cdf275b22bd50" "--gc-sections" "-L" "C:\\Users\\danie_000\\Code\\RE\\XYZ\\target\\thumbv7m-none-eabi\\debug\\deps" "-L" "C:\\Users\\danie_000\\Code\\RE\\XYZ\\target\\debug\\deps" "-L" "C:\\Users\\danie_000\\Code\\RE\\XYZ\\target\\thumbv7m-none-eabi\\debug\\build\\myapp-b0e16e01c22428a8\\out" "-L" "C:\\Users\\danie_000\\Code\\RE\\ABC\\target\\thumbv7m-none-eabi\\debug\\build\\cortex-m-765dfe6812e223ce\\out" "-L" "C:\\Users\\danie_000\\Code\\RE\\ABC\\target\\thumbv7m-none-eabi\\debug\\build\\cortex-m-rt-1387b5ad4b26cfa2\\out" "-L" "C:\\Users\\danie_000\\Code\\RE\\ABC\\target\\thumbv7m-none-eabi\\debug\\build\\cortex-m-semihosting-cb2469f22b42a1d7\\out" "-L" "C:\\Users\\danie_000\\.rustup\\toolchains\\beta-x86_64-pc-windows-msvc\\lib\\rustlib\\thumbv7m-none-eabi\\lib" "-Bstatic" "C:\\Users\\danie_000\\Code\\RE\\XYZ\\target\\thumbv7m-none-eabi\\debug\\deps\\libcortex_m_rt-03e440f58d1c0555.rlib" "C:\\Users\\danie_000\\Code\\RE\\XYZ\\target\\thumbv7m-none-eabi\\debug\\deps\\libr0-e3b5285d82ebc9ad.rlib" "--start-group" "C:\\Users\\danie_000\\Code\\RE\\XYZ\\target\\thumbv7m-none-eabi\\debug\\deps\\libpanic_halt-8d085ea03dce307c.rlib" "C:\\Users\\danie_000\\.rustup\\toolchains\\beta-x86_64-pc-windows-msvc\\lib\\rustlib\\thumbv7m-none-eabi\\lib\\libcore-47bba88e05b648a7.rlib" "--end-group" "C:\\Users\\danie_000\\.rustup\\toolchains\\beta-x86_64-pc-windows-msvc\\lib\\rustlib\\thumbv7m-none-eabi\\lib\\libcompiler_builtins-1eaa5842cd3f5019.rlib" "-Tlink.x" "-Bdynamic"
  = note: rust-lld: error: cannot find linker script link.x


error: aborting due to previous error

error: Could not compile `myapp`.

Caused by:
  process didn't exit successfully: `rustc --edition=2018 --crate-name myapp src\main.rs --color always --crate-type bin --emit=dep-info,link -C debuginfo=2 -C metadata=7f9cdf275b22bd50 -C extra-filename=-7f9cdf275b22bd50 --out-dir C:\Users\danie_000\Code\RE\XYZ\target\thumbv7m-none-eabi\debug\deps --target thumbv7m-none-eabi -C incremental=C:\Users\danie_000\Code\RE\XYZ\target\thumbv7m-none-eabi\debug\incremental -L dependency=C:\Users\danie_000\Code\RE\XYZ\target\thumbv7m-none-eabi\debug\deps -L dependency=C:\Users\danie_000\Code\RE\XYZ\target\debug\deps --extern cortex_m=C:\Users\danie_000\Code\RE\XYZ\target\thumbv7m-none-eabi\debug\deps\libcortex_m-6d95ef6e4550c4c8.rlib --extern cortex_m_rt=C:\Users\danie_000\Code\RE\XYZ\target\thumbv7m-none-eabi\debug\deps\libcortex_m_rt-03e440f58d1c0555.rlib --extern cortex_m_semihosting=C:\Users\danie_000\Code\RE\XYZ\target\thumbv7m-none-eabi\debug\deps\libcortex_m_semihosting-26e2b9dfbd5c7e3b.rlib --extern panic_halt=C:\Users\danie_000\Code\RE\XYZ\target\thumbv7m-none-eabi\debug\deps\libpanic_halt-8d085ea03dce307c.rlib -C link-arg=-Tlink.x -L C:\Users\danie_000\Code\RE\XYZ\target\thumbv7m-none-eabi\debug\build\myapp-b0e16e01c22428a8\out -L C:\Users\danie_000\Code\RE\ABC\target\thumbv7m-none-eabi\debug\build\cortex-m-765dfe6812e223ce\out -L C:\Users\danie_000\Code\RE\ABC\target\thumbv7m-none-eabi\debug\build\cortex-m-rt-1387b5ad4b26cfa2\out -L C:\Users\danie_000\Code\RE\ABC\target\thumbv7m-none-eabi\debug\build\cortex-m-semihosting-cb2469f22b42a1d7\out` (exit code: 1)
PS C:\Users\danie_000\Code\RE\XYZ>

---

I (@ExpHP) tried it myself and can easily reproduce it on linux, so it is not just a Windows thing. The build scripts involved are also innocuous, making correct use of std::env::var("OUT_DIR") and emitting basic cargo linker flags.

To reproduce:

• rustup target add thumbv7m-none-eabi
• generate a project from the cortex-m-quickstart template (or clone it and fix up the Cargo.toml)
• ^^ do what the OP did in the above quote ^^

---

Thoughts:

To my best understanding, cargo normally has no problem with the absolute paths that appear in build script cargo: directives because it stores the current path at the time of build to target/build/(unique-name)/root-output; future builds use this to correct the paths that are in the cached stdout from the script.

Looking at the build artefacts for clues, I notice that the build scripts' output is saved at

/home/lampam/asd/clone/original-name/target/thumbv7m-none-eabi/debug/build

but the root-output files for the build scripts contain

/home/lampam/asd/clone/original-name/target/debug

which is not a prefix of that path.

[ehuss]

@alexcrichton I was looking at fixing this. It's a little convoluted, so I just wanted to verify some assumptions:

The fix renaming PR introduced the fix that should address it, but it does it in a weird way. It uses the root_output variable which is .../target/debug for the host. I'm pretty certain that build scripts should only use target paths from their target-specific "out" directory, correct? So it should only be necessary to rewrite paths to that .../target/TRIPLE/debug/build/PKG-HASH/out directory correct? If so, then almost all the uses of root_output should actually be build_output. I'm a bit confused why the PR didn't do it that way in the first place, so I wanted to clarify. Maybe build scripts can emit paths to other things in the target dir outside of their "OUT_DIR"?

I think there is one exception that should still use the host directory, and that is this line which does some funky stuff for linking the build script itself.

[alexcrichton]

@ehuss nah you're spot on, build scripts only have supported output to $OUT_DIR which is indeed not in target/debug but rather in target/TRIPLE/debug/.... I suspect that the original PR may have just been mistaken, causing bugs like this!

Definitely feel free to correct any mistakes there :)

[ehuss]

@alexcrichton sorry, two more questions for fixing this.

• Changing the format of the root-output file may confuse cargo and cause it to use corrupted paths in existing projects. Normally this wouldn't be a problem because new cargos typically come with a new rustc which will cause the hash and fingerprint to change, and everything will be rebuilt with new hashes. However, if you swap between different cargos while using the same rustc, it has a chance to cause failures. Should the fix handle this? One option is to use a new filename for root-output (root-output2?).

• I noticed this doesn't fix paths for extra metadata values. For example, it seems to be common to have cargo:root=$OUT_DIR or cargo:include=$OUT_DIR/include. Should it handle these? I was thinking it could just do a replace of all lines containing $OUT_DIR, but either way sounds brittle.

[alexcrichton]

No worries! Changing the filename is fine by me, although I'd also be up for throwing in a schema-like version baked into Cargo into the hash to bust the cache (and we could increment it on changes like this to bust the cache)

That's a good point that we don't handle the other items. I think it's probably ok to handle it as if the absolute path to the output shows up in the various lines I can't really imagine a case where you wouldn't want it remapped!