Linking against musl libc causes linker error undefined symbol: posix_spawn_file_actions_addchdir_np

[NuLL3rr0r]

We have a proprietry Rust library we have been using for the past few years. We compile it to a .a file on GNU/Linux linked against musl libc, and then link it to our C++ libraries or applications.

This used to work fine until Rust version 1.83.0. But, with any newer versions we get the following linker error:

 ld.lld: error: undefined symbol: posix_spawn_file_actions_addchdir_np
>>> referenced by unix.rs:706 (library/std/src/sys/process/unix/unix.rs:706)
>>>               std-35c5a11e4fef3042.std.b3e601a09ed2eba9-cgu.0.rcgu.o:(std::sys::process::unix::unix::_$LT$impl$u20$std..sys..process..unix..common..Command$GT$::spawn::h08060e6e1d5b59e5) in archive /home/mamadou/dev/SomeProject/stage/lib/linux/rustc/x86-64/debug/libSomeRustLib.a
clang++: error: linker command failed with exit code 1 (use -v to see invocation)

I've noticed issue #99740 and pull request 3949 has introduced something related to this.

Code

I expected to see this happen:

The code should

Instead, this happened: explanation

Version it worked on

It most recently worked on: Rust 1.83.0

Version with regression

Any version newer than v1.83.0 fails to link.

rustc --version --verbose:

rustc 1.87.0 (17067e9ac 2025-05-09)
binary: rustc
commit-hash: 17067e9ac6d7ecb70e50f92c1944e545188d2359
commit-date: 2025-05-09
host: x86_64-unknown-linux-gnu
release: 1.87.0
LLVM version: 20.1.1

[NuLL3rr0r]

Just to clarify further, we use the Epic Native Toolchain versions v22, v23, and v25 since we use this library in Unreal Engine. Unreal Engine links against libc++ and not libstdc++.

v22 is built against CentOS 7 and v23 and v25 against Rocky Linux 8.

_LIBCPP_VERSION version for v22 is reported as 15001 while for the other two as 180100.

[NuLL3rr0r]

Also, we cross compule the Rust lib using cross-rs:

$ cross build --package SomeLib \
    --manifest-path=/Some/Path/Cargo.toml \
    --target aarch64-unknown-linux-musl

$ cross build --package SomeLib \
    --manifest-path=/Some/Path/Cargo.toml \
    --target aarch64-unknown-linux-musl --release

$ cross build --package SomeLib \
    --manifest-path=/Some/Path/Cargo.toml \
    --target x86_64-unknown-linux-musl

$ cross build --package SomeLib \
    --manifest-path=/Some/Path/Cargo.toml \
    --target x86_64-unknown-linux-musl --release

[NuLL3rr0r]

Furthermore, since Unreal Engine's build system UBT (Unreal Build Tool) is written in C#, you can see here it invokes the toolchain like this:

	private static string GetBundledLinuxLibCxxFlags(string ToolchainPath)
	{
		string ToolchainFlags = string.Format("--sysroot={0} -B{0}/usr/lib/ -B{0}/usr/lib64", ToolchainPath);
		string CFlags = "\"" + ToolchainFlags + "\"";
		string CxxFlags = "\"-I " + ThirdPartySourceDirectory + "/Linux/LibCxx/include -I " + ThirdPartySourceDirectory + "/Linux/LibCxx/include/c++/v1 " + ToolchainFlags + "\"";
		string CxxLinkerFlags = "\"-stdlib=libc++ -nodefaultlibs -L " + ThirdPartySourceDirectory + "/Linux/LibCxx/lib/Linux/x86_64-unknown-linux-gnu/ " + ThirdPartySourceDirectory + "/Linux/LibCxx/lib/Linux/x86_64-unknown-linux-gnu/libc++.a " + ThirdPartySourceDirectory + "/Linux/LibCxx/lib/Linux/x86_64-unknown-linux-gnu/libc++abi.a -lm -lc -lgcc_s -target x86_64-unknown-linux-gnu " + ToolchainFlags + "\"";

		return "-DCMAKE_C_FLAGS=" + CFlags + " -DCMAKE_CXX_FLAGS=" + CxxFlags + " -DCMAKE_EXE_LINKER_FLAGS=" + CxxLinkerFlags + " -DCAMKE_MODULE_LINKER_FLAGS=" + CxxLinkerFlags + " -DCMAKE_SHARED_LINKER_FLAGS=" + CxxLinkerFlags + " ";
	}

[Urgau]

posix_spawn_file_actions_addchdir_np was changed from a weak symbol to a strong symbol, see 7f74c89:

Now that Rust has a minimum musl version of 1.2.3, all supported musl versions
now include this symbol, so we can unconditionally expect it to be there.

The update of the minimum required version of musl to 1.2.3 was two years ago in #107129.

[NuLL3rr0r]

Thank you for the reference! But, prior to issue #99740 which seems to not make it to Rust v1.83.0 (release date: Nov 28 2024), our libraries were linking fine. Also, I just checked cross-rs musl Docker image and they specify they use v1.2.3.

So, not sure why we get undefined symbol since libSomeRustLib.a is generated by cross-rs Docker image. If that Docker image is using musl libc v1.2.3, then I guess the symbol should end up there? Or, am I missing something?

Could it be because our C++ libraries link against libc++ (this is something we cannot avoid, since Unreal Engine does it, that's why we ship two different libcs; if we use for example glibc we get all kind of duplicate symbol errors).

[workingjubilee]

So, not sure why we get undefined symbol since libSomeRustLib.a is generated by cross-rs Docker image. If that Docker image is using musl libc v1.2.3

...well, is it?

I notice that cross-rs has its last official release on crates.io or on GitHub releases in early 2023, but the diff that lands the musl libc 1.2.3 change is in late 2023, well after that point.

[NuLL3rr0r]

@workingjubilee nice catch with cross-rs release date. But, I've just checked our repo, and our Makefile, always installs the latest git commit of cross-rs:

	@cargo install cross --git https://github.com/cross-rs/cross

And, running --version confirms the commit hash.

$ cross --version                                         
cross 0.2.5 (51f46f2 2025-05-24)
[cross] warning: unable to get metadata for package
[cross] note: Falling back to `cargo` on the host.
cargo 1.87.0 (99624be96 2025-05-06)

Though, I'm not sure where those docker images are installed from. Their Supported targets section list musl libc as 1.2.3.

Probably, it's best to open an issue there then.

[workingjubilee]

That sounds like it should work, but yes, it probably is something that the cross maintainers can shed more light on.