Add the ability to provide build flags for the build-script-build

[cardoe]

My interest is in this is for Yocto. For Yocto the --build and --target are different but they're the same for Rust (I'm using x86_64 Ubuntu and building an x86_64 Yocto image). So what that means is that Yocto uses the distro compiler to produce a build sysroot using the triple x86_64-linux and the target sysroot uses the triple x86_64-poky-linux. A binary built for the build sysroot needs different libraries and a different rpath then what a binary built for the target sysroot. Now the Rust built-in target that would match both of these would be x86_64-unknown-linux-gnu and that's where the problem lies. Since these need different sysroots and rpath's then I need to be able to pass flags to the linker (or what I really use is a custom linker that's a wrapper to provide the right args). Now gcc-rs has the ability to pass HOST_CC and TARGET_CC along with other HOST_ and TARGET_ options but Cargo does not. Now if I built from x86_64 for an arm target and have the following config:

[target.x86_64-unknown-linux-gnu]
linker = "custom-build-linker"
[target.aarch64-unknown-linux-gnu]
linker = "custom-target-linker"

Cargo will build build-script-build with the custom-build-linker but when I'm building from x86_64 for x86_64 then there's no way for me to supply two different linkers. I focused my examples on linkers here but rustflags work the same way as well.

I propose adding support to Cargo to read a [host] section that would behave like a [target] section but instead would be used for any items that Cargo needs to build to run on the build machine (e.g. compiler plugins?). It would look something like:

[host]
linker = "custom-build-linker"
[target.aarch64-unknown-linux-gnu]
linker = "custom-target-linker"

[alexcrichton]

Shouldn't the poky-linux have a separate target? Using the same target for two different systems is likely to lead to a long tail of bugs to fix (like this)...

[cardoe]

That's actually how I'm doing things now. x86_64-linux and x86_64-poky-linux are just clones of Rust's x86_64-unknown-linux-gnu with the appropriate linker set. I was hoping to get away from custom targets because there are a slew of issues associated with that. The target specs changing (rust-lang/rust#38061 aims to help me with that) and Rust not having a default lookup path for target specs breaks things for me as well (we discussed this in rust-lang/rust#31117).

I can stick with the existing mechanisms of defining a custom target if that's better. I just noticed that I was able to use the built-in target except for this one issue (but then again there might be more like you said).

[alexcrichton]

Oh yeah I'd definitely believe that custom target specs have bugs, but if this is a target we'd want to officially support and whatnot we should consider adding it to rustc proper perhaps rather than relying on custom target specs?

[cardoe]

Well no. You don't want these upstreamed because they aren't real. Yocto does everything with sysroots. Which means we need a --sysroot flag and -L flags to where the right libraries are. These directories aren't fixed because they're relative to people's build directories. Because of this when you are targeting x86_64 for Yocto and building from x86_64 there are different target triples used to prevent HOST / TARGET contamination. So HOST will be x86_64-linux while the TARGET will be x86_64-poky-linux. I've wanted to map these over to the internal x86_64-unknown-gnu-linux while being able to pass additional flags but due to this issue I am unable to pass the right flags in to build.rs. Which means I am basically taking the internal x86_64-unknown-linux-gnu target and dumping it out to x86_64-linux.json and x86_64-poky-linux.json and injecting the necessary --sysroot and -L flags.

[cardoe]

I should note that I wrap up the path to the gcc from the sysroot (instead of the native OS's) and the --sysroot and -L flags with a custom linker script.

[cardoe]

I'd love to see rust-lang/rust#38338 stabilized to make this transformation easier.

[saurik]

I ran into a situation (filed as #9089) that seems related to this issue: build-script-build is insisting on using my CARGO_TARGET_..._LINKER override, but is ignoring my CARGO_TARGET_..._RUSTFLAGS, where I have a number of -C link-arg= flags that are required for my linker to work: in some sense, my core thesis is that I feel cargo should be consistent in either using both my special/magic target-specific linker and considering my special/magic target-specific link arguments, or it should ignore both.

While one fix for this would be to apply some form of RUSTFLAGS to the build-script-build--I'd have expected CARGO_BUILD_RUSTFLAGS to be precisely this, but somehow it isn't?--per #4423, another solution to this problem that would equally satisfy my specific scenario would for Cargo to avoid confusing the host with the target (related to my #8147), as it just seems wrong to use the "target" linker to build something for the "host"; in such a case, Cargo could decide to instead provide a CARGO_BUILD_LINKER (or maybe CARGO_HOST_LINKER, as I seem to not know what "BUILD" means? ;P) in a manner similar to @cardoe's request (which I could honestly just leave unset, as I have no need to provide a special build-time linker: I have special requirements on my target that require me to use a custom toolchain to compile my project, and the fact that my host happens to be the same triple as my target is somewhat irrelevant).