Unable to run tests in libc-test using musl (Alpine Linux 3.18.2)

[CuriouslyCurious]

I am unable to run the tests in libc-test using musl on Alpine Linux.

To reproduce:

1. Clone the libc repo.
2. cd libc-test && cargo test

This fails with:

  ...
  cargo:rerun-if-changed=../src/lib.rs
  cargo:rerun-if-changed=../src/macros.rs
  cargo:rerun-if-changed=../src/fixed_width_ints.rs
  cargo:rerun-if-changed=../src/unix/mod.rs
  cargo:rerun-if-changed=../src/unix/linux_like/mod.rs
  cargo:rerun-if-changed=../src/unix/linux_like/linux/mod.rs
  cargo:rerun-if-changed=../src/unix/linux_like/linux/musl/mod.rs
  cargo:rerun-if-changed=../src/unix/linux_like/linux/musl/lfs64.rs
  cargo:rerun-if-changed=../src/unix/linux_like/linux/musl/b64/mod.rs
  cargo:rerun-if-changed=../src/unix/linux_like/linux/musl/b64/x86_64/mod.rs
  cargo:rerun-if-changed=../src/unix/linux_like/linux/arch/mod.rs
  cargo:rerun-if-changed=../src/unix/linux_like/linux/arch/generic/mod.rs
  cargo:rerun-if-changed=../src/unix/linux_like/linux/align.rs
  OPT_LEVEL = Some("0")
  HOST = Some("x86_64-unknown-linux-musl")
  cargo:rerun-if-env-changed=CC_x86_64-unknown-linux-musl
  CC_x86_64-unknown-linux-musl = None
  cargo:rerun-if-env-changed=CC_x86_64_unknown_linux_musl
  CC_x86_64_unknown_linux_musl = None
  cargo:rerun-if-env-changed=HOST_CC
  HOST_CC = None
  cargo:rerun-if-env-changed=CC
  CC = None
  cargo:rerun-if-env-changed=CFLAGS_x86_64-unknown-linux-musl
  CFLAGS_x86_64-unknown-linux-musl = None
  cargo:rerun-if-env-changed=CFLAGS_x86_64_unknown_linux_musl
  CFLAGS_x86_64_unknown_linux_musl = None
  cargo:rerun-if-env-changed=HOST_CFLAGS
  HOST_CFLAGS = None
  cargo:rerun-if-env-changed=CFLAGS
  CFLAGS = None
  cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
  CRATE_CC_NO_DEFAULTS = None
  DEBUG = Some("true")
  CARGO_CFG_TARGET_FEATURE = Some("fxsr,sse,sse2")
  running: "cc" "-O0" "-ffunction-sections" "-fdata-sections" "-fPIC" "-gdwarf-4" "-fno-omit-frame-pointer" "-m64" "-Wall" "-Wextra" "-Wall" "-Wextra" "-Werror" "-Wno-unused-parameter" "-Wno-type-limits" "-Wno-address-of-packed-member" "-Wno-unknown-warning-option" "-Wno-deprecated-declarations" "-D_GNU_SOURCE" "-D__GLIBC_USE_DEPRECATED_SCANF" "-o" "/home/curious/dev/rust/forks/libc/target/debug/build/libc-test-37e96ea7152d2cd4/out/9e1282f7293605d7-main.o" "-c" "/home/curious/dev/rust/forks/libc/target/debug/build/libc-test-37e96ea7152d2cd4/out/main.c"
  cargo:warning=/home/curious/dev/rust/forks/libc/target/debug/build/libc-test-37e96ea7152d2cd4/out/main.c: In function '__test_fsize_af_alg_iv_iv':
  cargo:warning=/home/curious/dev/rust/forks/libc/target/debug/build/libc-test-37e96ea7152d2cd4/out/main.c:22283:34: error: invalid application of 'sizeof' to incomplete type '__u8[]' {aka 'unsigned char[]'}
  cargo:warning=22283 |                     return sizeof(foo->iv);
  cargo:warning=      |                                  ^
  cargo:warning=/home/curious/dev/rust/forks/libc/target/debug/build/libc-test-37e96ea7152d2cd4/out/main.c: In function '__test_roundtrip_af_alg_iv':
  cargo:warning=/home/curious/dev/rust/forks/libc/target/debug/build/libc-test-37e96ea7152d2cd4/out/main.c:22299:13: note: the ABI of passing struct with a flexible array member has changed in GCC 4.4
  cargo:warning=22299 |             ) {
  cargo:warning=      |             ^
  cargo:warning=/home/curious/dev/rust/forks/libc/target/debug/build/libc-test-37e96ea7152d2cd4/out/main.c: In function '__test_field_type_utmpx_ut_session':
  cargo:warning=/home/curious/dev/rust/forks/libc/target/debug/build/libc-test-37e96ea7152d2cd4/out/main.c:38937:28: error: returning 'int *' from a function with incompatible return type 'long int *' [-Werror=incompatible-pointer-types]
  cargo:warning=38937 |                     return &b->ut_session;
  cargo:warning=      |                            ^~~~~~~~~~~~~~
  cargo:warning=/home/curious/dev/rust/forks/libc/target/debug/build/libc-test-37e96ea7152d2cd4/out/main.c: In function '__test_fsize_af_alg_iv_iv':
  cargo:warning=/home/curious/dev/rust/forks/libc/target/debug/build/libc-test-37e96ea7152d2cd4/out/main.c:22284:17: error: control reaches end of non-void function [-Werror=return-type]
  cargo:warning=22284 |                 }
  cargo:warning=      |                 ^
  cargo:warning=At top level:
  cargo:warning=cc1: note: unrecognized command-line option '-Wno-unknown-warning-option' may have been intended to silence earlier diagnostics
  cargo:warning=cc1: all warnings being treated as errors
  exit status: 1
  ...

I've looked into the build scripts but have been unable to locate where this sizeof call gets made.

Extra info that may or may not help:

Kernel: 6.1.38-7-lts
rustc: 1.71.0

❯ rustc --version
rustc 1.71.0 (8ede3aae2 2023-07-12)

Architecture: Intel i7-3520M (4) @ 3.600GHz
OS: Alpine Linux 3.18.2

❯ bat -n /etc/os-release 
   1 NAME="Alpine Linux"
   2 ID=alpine
   3 VERSION_ID=3.18.2
   4 PRETTY_NAME="Alpine Linux v3.18"
   5 HOME_URL="https://alpinelinux.org/"
   6 BUG_REPORT_URL="https://gitlab.alpinelinux.org/alpine/aports/-/issues"

cc: 13.1.1

❯ cc --version
cc (Alpine 13.1.1_git20230715) 13.1.1 20230715
Copyright (C) 2023 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

clang: 16.0.6

❯ clang --version
Alpine clang version 16.0.6
Target: x86_64-alpine-linux-musl
Thread model: posix
InstalledDir: /usr/bin
Configuration file: /etc/clang16/x86_64-alpine-linux-musl.cfg

Full log:
log.txt

[devnexen]

Would this PR work for you ?

[CuriouslyCurious]

Would this PR work for you ?

Unfortunately it does not seem to work for me:

  ...
  DEBUG = Some("true")
  CARGO_CFG_TARGET_FEATURE = Some("fxsr,sse,sse2")
  running: "cc" "-O0" "-ffunction-sections" "-fdata-sections" "-fPIC" "-gdwarf-4" "-fno-omit-frame-pointer" "-m64" "-Wall" "-Wextra" "-Wall" "-Wextra" "-Werror" "-Wno-unused-parameter" "-Wno-type-limits" "-Wno-address-of-packed-member" "-Wno-unknown-warning-option" "-Wno-deprecated-declarations" "-D_GNU_SOURCE" "-D__GLIBC_USE_DEPRECATED_SCANF" "-o" "/home/curious/dev/rust/forks/utmpx_linux_musl/target/debug/build/libc-test-3c72df52c5a5615e/out/3a1bc9cc9021a524-main.o" "-c" "/home/curious/dev/rust/forks/utmpx_linux_musl/target/debug/build/libc-test-3c72df52c5a5615e/out/main.c"
  cargo:warning=/home/curious/dev/rust/forks/utmpx_linux_musl/target/debug/build/libc-test-3c72df52c5a5615e/out/main.c: In function '__test_fsize_af_alg_iv_iv':
  cargo:warning=/home/curious/dev/rust/forks/utmpx_linux_musl/target/debug/build/libc-test-3c72df52c5a5615e/out/main.c:22283:34: error: invalid application of 'sizeof' to incomplete type '__u8[]' {aka 'unsigned char[]'}
  cargo:warning=22283 |                     return sizeof(foo->iv);
  cargo:warning=      |                                  ^
  cargo:warning=/home/curious/dev/rust/forks/utmpx_linux_musl/target/debug/build/libc-test-3c72df52c5a5615e/out/main.c: In function '__test_roundtrip_af_alg_iv':
  cargo:warning=/home/curious/dev/rust/forks/utmpx_linux_musl/target/debug/build/libc-test-3c72df52c5a5615e/out/main.c:22299:13: note: the ABI of passing struct with a flexible array member has changed in GCC 4.4
  cargo:warning=22299 |             ) {
  cargo:warning=      |             ^
  cargo:warning=/home/curious/dev/rust/forks/utmpx_linux_musl/target/debug/build/libc-test-3c72df52c5a5615e/out/main.c: In function '__test_fsize_af_alg_iv_iv':
  cargo:warning=/home/curious/dev/rust/forks/utmpx_linux_musl/target/debug/build/libc-test-3c72df52c5a5615e/out/main.c:22284:17: error: control reaches end of non-void function [-Werror=return-type]
  cargo:warning=22284 |                 }
  cargo:warning=      |                 ^
  cargo:warning=At top level:
  cargo:warning=cc1: note: unrecognized command-line option '-Wno-unknown-warning-option' may have been intended to silence earlier diagnostics
  cargo:warning=cc1: all warnings being treated as errors
  exit status: 1

  --- stderr
  rust version: 1.71.0


  error occurred: Command "cc" "-O0" "-ffunction-sections" "-fdata-sections" "-fPIC" "-gdwarf-4" "-fno-omit-frame-pointer" "-m64" "-Wall" "-Wextra" "-Wall" "-Wextra" "-Werror" "-Wno-unused-parameter" "-Wno-type-limits" "-Wno-address-of-packed-member" "-Wno-unknown-warning-option" "-Wno-deprecated-declarations" "-D_GNU_SOURCE" "-D__GLIBC_USE_DEPRECATED_SCANF" "-o" "/home/curious/dev/rust/forks/utmpx_linux_musl/target/debug/build/libc-test-3c72df52c5a5615e/out/3a1bc9cc9021a524-main.o" "-c" "/home/curious/dev/rust/forks/utmpx_linux_musl/target/debug/build/libc-test-3c72df52c5a5615e/out/main.c" with args "cc" did not execute successfully (status code exit status: 1).
  ...

log.txt

[devnexen]

I wanted to see if the utmpx type was fixed, it seems the case. is it the same failure with clang ?

[CuriouslyCurious]

I wanted to see if the utmpx type was fixed, it seems the case. is it the same failure with clang ?

Looks like I'd need to rewrite the build.rs file to not use cc if I am understanding it correctly, which is a whole different kind of pain.

[jcajka]

For the record I have hit what seems as this issue in Fedora.

git clone https://github.com/rust-lang/libc
pushd libc/libc-test
cargo test

snip
.
.
.
 cargo:rerun-if-changed=../src/lib.rs
  cargo:rerun-if-changed=../src/macros.rs
  cargo:rerun-if-changed=../src/fixed_width_ints.rs
  cargo:rerun-if-changed=../src/unix/mod.rs
  cargo:rerun-if-changed=../src/unix/linux_like/mod.rs
  cargo:rerun-if-changed=../src/unix/linux_like/linux/mod.rs
  cargo:rerun-if-changed=../src/unix/linux_like/linux/gnu/mod.rs
  cargo:rerun-if-changed=../src/unix/linux_like/linux/gnu/b64/mod.rs
  cargo:rerun-if-changed=../src/unix/linux_like/linux/gnu/b64/x86_64/mod.rs
  cargo:rerun-if-changed=../src/unix/linux_like/linux/gnu/b64/x86_64/not_x32.rs
  cargo:rerun-if-changed=../src/unix/linux_like/linux/arch/mod.rs
  cargo:rerun-if-changed=../src/unix/linux_like/linux/arch/generic/mod.rs
  cargo:rerun-if-changed=../src/unix/linux_like/linux/align.rs
  OPT_LEVEL = Some("0")
  HOST = Some("x86_64-unknown-linux-gnu")
  cargo:rerun-if-env-changed=CC_x86_64-unknown-linux-gnu
  CC_x86_64-unknown-linux-gnu = None
  cargo:rerun-if-env-changed=CC_x86_64_unknown_linux_gnu
  CC_x86_64_unknown_linux_gnu = None
  cargo:rerun-if-env-changed=HOST_CC
  HOST_CC = None
  cargo:rerun-if-env-changed=CC
  CC = None
  cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
  CRATE_CC_NO_DEFAULTS = None
  DEBUG = Some("true")
  CARGO_CFG_TARGET_FEATURE = Some("fxsr,sse,sse2")
  cargo:rerun-if-env-changed=CFLAGS_x86_64-unknown-linux-gnu
  CFLAGS_x86_64-unknown-linux-gnu = None
  cargo:rerun-if-env-changed=CFLAGS_x86_64_unknown_linux_gnu
  CFLAGS_x86_64_unknown_linux_gnu = None
  cargo:rerun-if-env-changed=HOST_CFLAGS
  HOST_CFLAGS = None
  cargo:rerun-if-env-changed=CFLAGS
  CFLAGS = None
  running: "cc" "-O0" "-ffunction-sections" "-fdata-sections" "-fPIC" "-gdwarf-4" "-fno-omit-frame-pointer" "-m64" "-Wall" "-Wextra" "-Wall" "-Wextra" "-Werror" "-Wno-unused-parameter" "-Wno-type-limits" "-Wno-address-of-packed-member" "-Wno-unknown-warning-option" "-Wno-deprecated-declarations" "-D_GNU_SOURCE" "-D__GLIBC_USE_DEPRECATED_SCANF" "-o" "/root/libc/target/debug/build/libc-test-0e2fcb5e7316ed69/out/b0f3d9471b1ba493-main.o" "-c" "/root/libc/target/debug/build/libc-test-0e2fcb5e7316ed69/out/main.c"
  cargo:warning=/root/libc/target/debug/build/libc-test-0e2fcb5e7316ed69/out/main.c: In function ‘__test_fsize_af_alg_iv_iv’:

  cargo:warning=/root/libc/target/debug/build/libc-test-0e2fcb5e7316ed69/out/main.c:25438:34: error: invalid application of ‘sizeof’ to incomplete type ‘__u8[]’ {aka ‘unsigned char[]’}

  cargo:warning=25438 |                     return sizeof(foo->iv);

  cargo:warning=      |                                  ^

  cargo:warning=/root/libc/target/debug/build/libc-test-0e2fcb5e7316ed69/out/main.c: In function ‘__test_roundtrip_af_alg_iv’:

  cargo:warning=/root/libc/target/debug/build/libc-test-0e2fcb5e7316ed69/out/main.c:25454:13: note: the ABI of passing struct with a flexible array member has changed in GCC 4.4

  cargo:warning=25454 |             ) {

  cargo:warning=      |             ^

  cargo:warning=/root/libc/target/debug/build/libc-test-0e2fcb5e7316ed69/out/main.c: In function ‘__test_fsize_af_alg_iv_iv’:

  cargo:warning=/root/libc/target/debug/build/libc-test-0e2fcb5e7316ed69/out/main.c:25439:17: error: control reaches end of non-void function [-Werror=return-type]

  cargo:warning=25439 |                 }

  cargo:warning=      |                 ^

  cargo:warning=At top level:

  cargo:warning=cc1: note: unrecognized command-line option ‘-Wno-unknown-warning-option’ may have been intended to silence earlier diagnostics

  cargo:warning=cc1: all warnings being treated as errors

  exit status: 1

  --- stderr
  rust version: 1.71.0


  error occurred: Command "cc" "-O0" "-ffunction-sections" "-fdata-sections" "-fPIC" "-gdwarf-4" "-fno-omit-frame-pointer" "-m64" "-Wall" "-Wextra" "-Wall" "-Wextra" "-Werror" "-Wno-unused-parameter" "-Wno-type-limits" "-Wno-address-of-packed-member" "-Wno-unknown-warning-option" "-Wno-deprecated-declarations" "-D_GNU_SOURCE" "-D__GLIBC_USE_DEPRECATED_SCANF" "-o" "/root/libc/target/debug/build/libc-test-0e2fcb5e7316ed69/out/b0f3d9471b1ba493-main.o" "-c" "/root/libc/target/debug/build/libc-test-0e2fcb5e7316ed69/out/main.c" with args "cc" did not execute successfully (status code exit status: 1).

Same/similar issue is happening across all architectures supported by Fedora(x86_64 log^^,s390x, ppc64le and aarch64).
This is with rust-1.71, glibc-2.37, gcc-13.2.1 and binutils-2.40.

[JohnTitor]

I believe the cause is y'all use different versions for toolchains, e.g. musl, GCC, clang, etc. Since the current test cannot change type/value according to user system version, I'd recommend running the test via Docker, like CI.

[jcajka]

@JohnTitor Could you please list what versions of the respective, relevant tools that are supported by this rust crate?

[jcajka]

And for the record, if I run it in Fedora container I get the same results.

[xbjfk]

Sorry to necropost, but I believe this should be reopened, and the type changed or test excluded.
I was able to reproduce this in the latest Rust Alpine docker image:

docker run -it rust:1-alpine3.21 sh -c '\
apk add git musl-dev linux-headers && \
git clone https://github.com/rust-lang/libc.git && \
cd libc/libc-test \
&& cargo test'

I can see that 5-6 years ago, in Musl 1.2.0, the type changed to int with separate padding. Should libc be updated to reflect that change, or the test be excluded?