musl-cross-make cross compiling and specifying "--with-libc=/custom/path/to/libc.so" is dismissed

[tlaurion]

Bug report

Trying to integrate CPython 3.9.2 under Heads firmware (coreboot+linux payload). This is followup of old integration issue at linuxboot/heads#689

CPython 3.9.2 (as all other modules under Heads but coreboot) is built by first building musl-cross-make and then having modules being build with musl-cross-make to have reproducible build environment, so that coreboot payload' linux kernel's initrd contains binaries needed in linux embedded environment (recovery shell and associated tools), are reproducible (this is the goal).

Your environment

• CPython versions tested on: 3.9.2 installed to build target 3.9.2 python
• Operating system and architecture: debian-11 host build environement

I'm experimenting with the following build recipe, and did a lot of variation without success https://github.com/osresearch/heads/blob/df50c5f7d242b502fc443157fd592b638da87af0/modules/python#L13-L33.

This results in configure script producing the following config.log output:
config.log

So above build recipe produces:

• a correct ./configure callm specifying ./configure --enable-optimizations --with-cxx-main=/home/user/heads/crossgcc/x86/bin/x86_64-linux-musl-gcc --with-libc=/home/user/heads/crossgcc/x86/x86_64-linux-musl/lib/libc.so --host x86_64-linux-musl --build x86_64 --disable-ipv6 --prefix= --exec-prefix=
• --with-libc="$(musl-cross_libraries)" translates into config.log, correctly, into configure:14435: result: set LIBC="/home/user/heads/crossgcc/x86/x86_64-linux-musl/lib/libc.so". This is the libc.so we need to rely on
• Unfortunately, the build fails (See in CircleCI logs here) with libc being completely dismissed. This results into error : ImportError: /usr/lib/x86_64-linux-gnu/libc.so: invalid ELF header. This is the wrong libc.so of the host, not of the cross built environment.

Expected behavior:

• Specifying --with-libc=/home/user/heads/crossgcc/x86/x86_64-linux-musl/lib/libc.so should result in libc.so used from /home/user/heads/crossgcc/x86/x86_64-linux-musl/lib/libc.so not /usr/lib/x86_64-linux-gnu/libc.so.

Why is the host libc used even when libc is specified and found at configure?

---

Was gonna investigate and adapt OpenWrt patches at https://github.com/openwrt/packages/tree/master/lang/python/python3/patches but reading about cross build issues it would be better to have this fixed upstream then downstream into all projects trying to cross build for their embedded systems.

Any insight?

[tlaurion]

Already tried to apply OpenWrt patches, but they won't work, since they do the builds under buildroot, but the resulting patch behavior might work in their use case since no other libc would be found under buildroot path discovered. Here, we want to specify CXX, CC LIBC and have those respected and used.

In the current issue, we try to force configure to point to the right libc, and the builds should not deviate from what is configured through ./configure --enable-optimizations --with-cxx-main=/home/user/heads/crossgcc/x86/bin/x86_64-linux-musl-gcc --with-libc=/home/user/heads/crossgcc/x86/x86_64-linux-musl/lib/libc.so --host x86_64-linux-musl --build x86_64 --disable-ipv6 --prefix= --exec-prefix=

Again, pointing to config.log directly if that helps for quicker response.

[tlaurion]

@tiran I have looked into other musl related issues like #95855 and #87278 which are similar and for which fixes should be included into 3.9.2 as I understood it. Yet, the issue is still present and configure scripts are doing the wrong thing.

[erlend-aasland]

@tlaurion, FYI: #107221 has now been merged into main; it will not be backported, though.