Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Undefined reference to 'gnu_get_libc_version' error when using alpine:3.16 as target image #1582

Open
4 of 11 tasks
FirelightFlagboy opened this issue Oct 30, 2024 · 6 comments

Comments

@FirelightFlagboy
Copy link

FirelightFlagboy commented Oct 30, 2024

Checklist

Describe your issue

I'm trying to compile a CLI that depends on openssl and fuse3.
I was able to statically build that CLI directly inside the docker container alpine:3.16.

Now I would like to use cross for that so I've configured target.x86_64-unknown-linux-musl.image to use alpine:3.16 but when build the project I'm getting errors about undefined reference to 'gnu_get_libc_version'.

Another thing that's odd for me is that I've never configured the target.x86_64-unknown-linux-musl.pre-build to install rust but cross seems to use one anyway

So how cross is able to use cargo if it's not provided by used the image?
Is it possible to force cross to use what provided by the image and error out if something is missing?

What target(s) are you cross-compiling for?

x86_64-unknown-linux-musl

Which operating system is the host (e.g computer cross is on) running?

  • macOS
  • Windows
  • Linux / BSD
  • other OS (specify in description)

What architecture is the host?

  • x86_64 / AMD64
  • arm32
  • arm64 (including Mac M1)

What container engine is cross using?

  • docker
  • podman
  • other container engine (specify in description)

cross version

cross 0.2.5

Example

  1. Execute the following commands:

    cargo new test-cross
    cd test-cross
    cargo add openssl@0.10.68 --features vendored
  2. Modify src/main.rs to be like the following:

    fn main() {
        use openssl::version;
        println!("Hello, world! {}", version::version());
    }
  3. Add the following cross configuration file in Cross.toml

    [target.x86_64-unknown-linux-musl]
    image = "alpine:3.16"
    pre-build = [
        "apk add --no-cache openssl-dev musl-dev build-base perl"
    ]
  4. Try to build with cross build --release --target=x86_64-unknown-linux-musl

Additional information / notes

Output of: cross b --release --target=x86_64-unknown-linux-musl
[+] Building 3.2s (6/6) FINISHED                                                                               docker:default
 => [internal] load build definition from Dockerfile.x86_64-unknown-linux-musl-custom                                    0.1s
 => => transferring dockerfile: 211B                                                                                     0.0s
 => [internal] load metadata for docker.io/library/alpine:3.16                                                           0.0s
 => [internal] load .dockerignore                                                                                        0.1s
 => => transferring context: 2B                                                                                          0.0s
 => CACHED [1/2] FROM docker.io/library/alpine:3.16                                                                      0.0s
 => [2/2] RUN eval "apk add --no-cache openssl-dev musl-dev build-base perl"                                             2.1s
 => exporting to image                                                                                                   0.5s 
 => => exporting layers                                                                                                  0.5s 
 => => writing image sha256:7e82daed6ca17160285f6dc4a18b6ef07d5439e9befa3d0b191b4634382d9ed8                             0.0s 
 => => naming to docker.io/library/cross-custom-test-cross:x86_64-unknown-linux-musl-b0299-pre-build                     0.0s 
   Compiling proc-macro2 v1.0.89                                                                                              
   Compiling openssl-src v300.4.0+3.4.0                                                                                       
   Compiling libc v0.2.161
   Compiling openssl v0.10.68
   Compiling openssl-sys v0.9.104
error: linking with `cc` failed: exit status: 1
  |
  = note: LC_ALL="C" PATH="/rust/lib/rustlib/x86_64-unknown-linux-gnu/bin:/rust/lib/rustlib/x86_64-unknown-linux-gnu/bin:/rust/lib/rustlib/x86_64-unknown-linux-gnu/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/rust/bin" VSLANG="1033" "cc" "-m64" "/tmp/rustc80xS4J/symbols.o" "/target/release/build/libc-118ed0ed7e3a5800/build_script_build-118ed0ed7e3a5800.build_script_build.1a8c240770426bfa-cgu.0.rcgu.o" "/target/release/build/libc-118ed0ed7e3a5800/build_script_build-118ed0ed7e3a5800.build_script_build.1a8c240770426bfa-cgu.1.rcgu.o" "/target/release/build/libc-118ed0ed7e3a5800/build_script_build-118ed0ed7e3a5800.build_script_build.1a8c240770426bfa-cgu.2.rcgu.o" "/target/release/build/libc-118ed0ed7e3a5800/build_script_build-118ed0ed7e3a5800.build_script_build.1a8c240770426bfa-cgu.3.rcgu.o" "/target/release/build/libc-118ed0ed7e3a5800/build_script_build-118ed0ed7e3a5800.7hnhmvsl3ug9mt9euyysn62d0.rcgu.o" "-Wl,--as-needed" "-Wl,-Bstatic" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-22be60875a4ac8d7.rlib" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/libpanic_unwind-4b832a03827ff95e.rlib" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/libobject-a25e7f31e2204719.rlib" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/libmemchr-1f1858edbb50ccb5.rlib" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/libaddr2line-1319b75889377807.rlib" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/libgimli-c9972d847170eb93.rlib" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_demangle-0868a75d0b80f801.rlib" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd_detect-f4254a923640cbea.rlib" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/libhashbrown-f21de9b2e2c92770.rlib" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_alloc-cf868f78468d45dd.rlib" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/libminiz_oxide-cce9a55233783113.rlib" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/libadler-6ef1f692f91f321e.rlib" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/libunwind-d2b4a3bc57672e2c.rlib" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcfg_if-9d8e4e6f4cb45592.rlib" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-858b7338441a56b6.rlib" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc-d182631d04e1eed2.rlib" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_core-7874c355ab5ed077.rlib" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-46b25da62cc69443.rlib" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-c71612932829263c.rlib" "-Wl,-Bdynamic" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-L" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-o" "/target/release/build/libc-118ed0ed7e3a5800/build_script_build-118ed0ed7e3a5800" "-Wl,--gc-sections" "-pie" "-Wl,-z,relro,-z,now" "-Wl,--strip-debug" "-nodefaultlibs"
  = note: /usr/lib/gcc/x86_64-alpine-linux-musl/11.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: /rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-22be60875a4ac8d7.rlib(std-22be60875a4ac8d7.std.5204e9590b4985ef-cgu.0.rcgu.o): in function `std::sys::pal::unix::os::glibc_version':
          /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/std/src/sys/pal/unix/os.rs:767: undefined reference to `gnu_get_libc_version'
          collect2: error: ld returned 1 exit status
          
  = note: some `extern` functions couldn't be found; some native libraries may need to be installed or have their path specified
  = note: use the `-l` flag to specify native libraries to link
  = note: use the `cargo:rustc-link-lib` directive to specify the native libraries to link with Cargo (see https://doc.rust-lang.org/cargo/reference/build-scripts.html#rustc-link-lib)

error: could not compile `libc` (build script) due to 1 previous error
warning: build failed, waiting for other jobs to finish...
error: linking with `cc` failed: exit status: 1
  |
  = note: LC_ALL="C" PATH="/rust/lib/rustlib/x86_64-unknown-linux-gnu/bin:/rust/lib/rustlib/x86_64-unknown-linux-gnu/bin:/rust/lib/rustlib/x86_64-unknown-linux-gnu/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/rust/bin" VSLANG="1033" "cc" "-m64" "/tmp/rustcEuKVr9/symbols.o" "/target/release/build/proc-macro2-0ac7f6da96b55015/build_script_build-0ac7f6da96b55015.build_script_build.2a63ada11ec2c93c-cgu.0.rcgu.o" "/target/release/build/proc-macro2-0ac7f6da96b55015/build_script_build-0ac7f6da96b55015.build_script_build.2a63ada11ec2c93c-cgu.1.rcgu.o" "/target/release/build/proc-macro2-0ac7f6da96b55015/build_script_build-0ac7f6da96b55015.5hzjfvlooogfnkn2jpw0xvl5l.rcgu.o" "-Wl,--as-needed" "-Wl,-Bstatic" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-22be60875a4ac8d7.rlib" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/libpanic_unwind-4b832a03827ff95e.rlib" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/libobject-a25e7f31e2204719.rlib" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/libmemchr-1f1858edbb50ccb5.rlib" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/libaddr2line-1319b75889377807.rlib" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/libgimli-c9972d847170eb93.rlib" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_demangle-0868a75d0b80f801.rlib" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd_detect-f4254a923640cbea.rlib" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/libhashbrown-f21de9b2e2c92770.rlib" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_alloc-cf868f78468d45dd.rlib" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/libminiz_oxide-cce9a55233783113.rlib" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/libadler-6ef1f692f91f321e.rlib" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/libunwind-d2b4a3bc57672e2c.rlib" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcfg_if-9d8e4e6f4cb45592.rlib" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-858b7338441a56b6.rlib" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc-d182631d04e1eed2.rlib" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_core-7874c355ab5ed077.rlib" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-46b25da62cc69443.rlib" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-c71612932829263c.rlib" "-Wl,-Bdynamic" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-L" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-o" "/target/release/build/proc-macro2-0ac7f6da96b55015/build_script_build-0ac7f6da96b55015" "-Wl,--gc-sections" "-pie" "-Wl,-z,relro,-z,now" "-Wl,--strip-debug" "-nodefaultlibs"
  = note: /usr/lib/gcc/x86_64-alpine-linux-musl/11.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: /rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-22be60875a4ac8d7.rlib(std-22be60875a4ac8d7.std.5204e9590b4985ef-cgu.0.rcgu.o): in function `std::sys::pal::unix::os::glibc_version':
          /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/std/src/sys/pal/unix/os.rs:767: undefined reference to `gnu_get_libc_version'
          collect2: error: ld returned 1 exit status
          
  = note: some `extern` functions couldn't be found; some native libraries may need to be installed or have their path specified
  = note: use the `-l` flag to specify native libraries to link
  = note: use the `cargo:rustc-link-lib` directive to specify the native libraries to link with Cargo (see https://doc.rust-lang.org/cargo/reference/build-scripts.html#rustc-link-lib)

error: could not compile `proc-macro2` (build script) due to 1 previous error
error: linking with `cc` failed: exit status: 1
  |
  = note: LC_ALL="C" PATH="/rust/lib/rustlib/x86_64-unknown-linux-gnu/bin:/rust/lib/rustlib/x86_64-unknown-linux-gnu/bin:/rust/lib/rustlib/x86_64-unknown-linux-gnu/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/rust/bin" VSLANG="1033" "cc" "-m64" "/tmp/rustcL8ItjU/symbols.o" "/target/release/build/openssl-sys-1c1c027f40443768/build_script_main-1c1c027f40443768.build_script_main.ed5b99cd75f09a29-cgu.0.rcgu.o" "/target/release/build/openssl-sys-1c1c027f40443768/build_script_main-1c1c027f40443768.build_script_main.ed5b99cd75f09a29-cgu.1.rcgu.o" "/target/release/build/openssl-sys-1c1c027f40443768/build_script_main-1c1c027f40443768.build_script_main.ed5b99cd75f09a29-cgu.2.rcgu.o" "/target/release/build/openssl-sys-1c1c027f40443768/build_script_main-1c1c027f40443768.evvfp87uswpd0rae3qahoih92.rcgu.o" "-Wl,--as-needed" "-Wl,-Bstatic" "/target/release/deps/libvcpkg-7ea8454f78fa1ff2.rlib" "/target/release/deps/libpkg_config-2f2c950cb259555d.rlib" "/target/release/deps/libopenssl_src-4f856c9b1e125d8d.rlib" "/target/release/deps/libcc-7bc7faf5d6bf3657.rlib" "/target/release/deps/libshlex-b41d57ce4f6400f0.rlib" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-22be60875a4ac8d7.rlib" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/libpanic_unwind-4b832a03827ff95e.rlib" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/libobject-a25e7f31e2204719.rlib" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/libmemchr-1f1858edbb50ccb5.rlib" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/libaddr2line-1319b75889377807.rlib" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/libgimli-c9972d847170eb93.rlib" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_demangle-0868a75d0b80f801.rlib" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd_detect-f4254a923640cbea.rlib" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/libhashbrown-f21de9b2e2c92770.rlib" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_alloc-cf868f78468d45dd.rlib" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/libminiz_oxide-cce9a55233783113.rlib" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/libadler-6ef1f692f91f321e.rlib" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/libunwind-d2b4a3bc57672e2c.rlib" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcfg_if-9d8e4e6f4cb45592.rlib" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/liblibc-858b7338441a56b6.rlib" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc-d182631d04e1eed2.rlib" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_std_workspace_core-7874c355ab5ed077.rlib" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-46b25da62cc69443.rlib" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-c71612932829263c.rlib" "-Wl,-Bdynamic" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-L" "/rust/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-o" "/target/release/build/openssl-sys-1c1c027f40443768/build_script_main-1c1c027f40443768" "-Wl,--gc-sections" "-pie" "-Wl,-z,relro,-z,now" "-Wl,--strip-debug" "-nodefaultlibs"
  = note: /usr/lib/gcc/x86_64-alpine-linux-musl/11.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: /rust/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-22be60875a4ac8d7.rlib(std-22be60875a4ac8d7.std.5204e9590b4985ef-cgu.0.rcgu.o): in function `std::sys::pal::unix::os::glibc_version':
          /rustc/f6e511eec7342f59a25f7c0534f1dbea00d01b14/library/std/src/sys/pal/unix/os.rs:767: undefined reference to `gnu_get_libc_version'
          collect2: error: ld returned 1 exit status
          
  = note: some `extern` functions couldn't be found; some native libraries may need to be installed or have their path specified
  = note: use the `-l` flag to specify native libraries to link
  = note: use the `cargo:rustc-link-lib` directive to specify the native libraries to link with Cargo (see https://doc.rust-lang.org/cargo/reference/build-scripts.html#rustc-link-lib)

error: could not compile `openssl-sys` (build script) due to 1 previous error
@Emilgardis
Copy link
Member

Cross works by mounting the host toolchain into the image, that means you don't need rust in the image itself, only cross-compilation tools for other languages.

I recognize this error, and had a way to solve it, but I can't remember where I documented it (I think the fix was installing some glibc compat package). Anyway, you should be able to get away from this by using a x86_64-unknown-linux-musl toolchain instead of the gnu one.

If you want to use a musl rust toolchain instead, use image.toolchain

[target.x86_64-unknown-linux-musl]
image.name = "alpine:3.16"
image.toolchain = ["x86_64-unknown-linux-musl"]

@FirelightFlagboy
Copy link
Author

If I use the following cross configuration:

[target.x86_64-unknown-linux-musl]
image.name = "alpine:3.16"
image.toolchain = ["x86_64-unknown-linux-musl"]
pre-build = [
    "apk add --no-cache openssl-dev musl-dev build-base perl"
]

I get the following error:

Error: 
   0: failed to parse file `"/tmp/test-cross/Cross.toml"` as TOML
   1: invalid type: map, expected a string for key `target.x86_64-unknown-linux-musl.image` at line 2 column 14

Is it something that was added since 0.2.5 ?

@Emilgardis
Copy link
Member

oh yeah, sorry thought you were using cross from the main branch. You can install it with cargo install cross --git https://github.com/cross-rs/cross

@FirelightFlagboy
Copy link
Author

I've installed cross from 4090bec, but I get another error

[+] Building 0.4s (6/6) FINISHED                                                                           docker:default
 => [internal] load build definition from Dockerfile.x86_64-unknown-linux-musl-custom                                0.1s
 => => transferring dockerfile: 211B                                                                                 0.0s
 => [internal] load metadata for docker.io/library/alpine:3.16                                                       0.0s
 => [internal] load .dockerignore                                                                                    0.1s
 => => transferring context: 2B                                                                                      0.0s
 => [1/2] FROM docker.io/library/alpine:3.16                                                                         0.0s
 => CACHED [2/2] RUN eval "apk add --no-cache openssl-dev musl-dev build-base perl"                                  0.0s
 => exporting to image                                                                                               0.0s
 => => exporting layers                                                                                              0.0s
 => => writing image sha256:f351ae4b0144db639081fb91c36df8ef1794e3f04cf55fee23e39948e4f1967a                         0.0s
 => => naming to localhost/cross-rs/cross-custom-test-cross:x86_64-unknown-linux-musl-b0299-pre-build                0.0s
cargo: error while loading shared libraries: /nix/store/sl141d1g77wvhr050ah87lcyz2czdxa3-glibc-2.40-36/lib/libc.so: invalid ELF header

It seems like cross still tries to use rust from my host system 😅

@Emilgardis
Copy link
Member

Emilgardis commented Oct 30, 2024

Yes, it will always do that, we dont support using rust installed in the image (although it is possible, but no reason to use the cross cli then)

I'm not sure how to fix a nix rustup/rustc setup. Try setting the NIX_STORE env to whereever it is located

edit: actually, not needed, the error message suggests it's been loaded, but you can make sure with -v

@FirelightFlagboy
Copy link
Author

In the end, I was able to configure cross to make it use the toolchain installed in the container with:

[target.x86_64-unknown-linux-musl]
image = "alpine:3.16"
pre-build = [
    "apk add --no-cache fuse3-dev fuse3-static openssl-dev musl-dev build-base perl rustup gcompat",
    # Install our own rustup toolchain since we prefer to not rely on the host provided one
    "rustup-init -y --default-toolchain stable --profile minimal",
    # Make sure the build user will be able to access the toolchain binaries
    "P=/root/.rustup/toolchains/stable-x86_64-unknown-linux-musl/bin; while [ \"$P\" != \"/\" ]; do chmod -v 755 $P; P=$(dirname $P); done"
]

[target.x86_64-unknown-linux-musl.env]
passthrough = [
    # Modify the PATH to prefer the rust toolchain of the container over the host one.
    "PATH=/root/.rustup/toolchains/stable-x86_64-unknown-linux-musl/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants