Rust programs require recent glibc due to `__cxa_thread_atexit_impl`

[Timmmm]

I think this is a bug but I'm not 100% sure. Anyway, I have cross-compiled a simple Rust program with the toolchain stable-x86_64-pc-windows-gnu and the target armv7-unknown-linux-gnueabihf, using this GCC cross compiler from Linaro.

However when I run it on a reasonably old system I get:

./foo: /lib/libc.so.6: version `GLIBC_2.18' not found (required by ./foo)

objdump -T foo reveals that this is the only symbol that needs glibc 2.18. The others all need 2.4.

00000000  w   DF *UND*  00000000  GLIBC_2.18  __cxa_thread_atexit_impl

The symbol is weakly defined, however the dependency on glibc 2.18 is still strong:

$ readelf -V foo
...
Version needs section '.gnu.version_r' contains 5 entries:
 Addr: 0x0000000000001e4c  Offset: 0x001e4c  Link: 6 (.dynstr)
  000000: Version: 1  File: ld-linux-armhf.so.3  Cnt: 1
  0x0010:   Name: GLIBC_2.4  Flags: none  Version: 9
  0x0020: Version: 1  File: librt.so.1  Cnt: 1
  0x0030:   Name: GLIBC_2.4  Flags: none  Version: 5
  0x0040: Version: 1  File: libgcc_s.so.1  Cnt: 4
  0x0050:   Name: GCC_4.3.0  Flags: none  Version: 10
  0x0060:   Name: GCC_3.0  Flags: none  Version: 7
  0x0070:   Name: GCC_3.5  Flags: none  Version: 6
  0x0080:   Name: GCC_3.3.1  Flags: none  Version: 4
  0x0090: Version: 1  File: libc.so.6  Cnt: 2
  0x00a0:   Name: GLIBC_2.18  Flags: none  Version: 8
  0x00b0:   Name: GLIBC_2.4  Flags: none  Version: 3
  0x00c0: Version: 1  File: libpthread.so.0  Cnt: 1
  0x00d0:   Name: GLIBC_2.4  Flags: none  Version: 2

I found a site where they should how to hex edit the binary so it shows this:

 0x00a0:   Name: GLIBC_2.18  Flags: WEAK   Version: 8

However it still doesn't work:

./foo: /lib/libc.so.6: weak version `GLIBC_2.18' not found (required by ./foo)
./foo: relocation error: ./foo: symbol __cxa_thread_atexit_impl, version GLIBC_2.18 not defined in file libc.so.6 with link time reference

Now, I wouldn't have thought this was a bug, except that the code that uses __cxa_thread_atexit_impl seems to have been written intending it to be a weak reference so it seems like a bug.

I'm not an expert on glibc's insanity though so I could be wrong. I posted a SO question but no answers yet. :-/

[nagisa]

Seems like an unintended behaviour indeed, though I feel like solving it is not something we can easily do! Probably the best way to crosscompile a rustc in a way which does not introduce a dependency on GLIBC_2.18 is to crosscompile from a system with glibc older than 2.18 as well.

http://stackoverflow.com/questions/6136423/how-can-i-avoid-dependency-on-glibc-x-y-when-using-weak-symbols#6139218 describes the same problem. I wonder what is meant by weak alias in the answer.

cc @alexcrichton

EDIT: less relevant: https://web.archive.org/web/20160107032111/http://www.trevorpounds.com/blog/?p=103 and http://stackoverflow.com/questions/2856438/how-can-i-link-to-a-specific-glibc-version.

[Timmmm]

This page describes weak aliases. I don't think the alias is important - probably it is the fact that he provided a default implementation, so GCC doesn't try very hard to find another. I wonder if the code were reworked somewhat like this it would work:

#[cfg(target_os = "linux")]
unsafe fn register_dtor(t: *mut u8, dtor: unsafe extern fn(*mut u8)) {

    extern {
        #[linkage = "extern_weak"]
        static __dso_handle: *mut u8;
    }

    __cxa_thread_atexit_impl(dtor, t, &__dso_handle as *const _ as *mut _);
}

#[cfg(target_os = "linux")]
#[linkage = "weak"]
extern "C" unsafe fn __cxa_thread_atexit_impl(dtor: unsafe extern fn(*mut u8),
                                  t: *mut u8,
                                  dso_handle: *mut u8) -> libc::c_int {

    use mem;
    use libc;
    use sys_common::thread_local as os;

    // The fallback implementation uses a vanilla OS-based TLS key to track
    // the list of destructors that need to be run for this thread. The key
    // then has its own destructor which runs all the other destructors.
    //
    // The destructor for DTORS is a little special in that it has a `while`
    // loop to continuously drain the list of registered destructors. It
    // *should* be the case that this loop always terminates because we
    // provide the guarantee that a TLS key cannot be set after it is
    // flagged for destruction.
    static DTORS: os::StaticKey = os::StaticKey::new(Some(run_dtors));
    type List = Vec<(*mut u8, unsafe extern fn(*mut u8))>;
    if DTORS.get().is_null() {
        let v: Box<List> = box Vec::new();
        DTORS.set(Box::into_raw(v) as *mut u8);
    }
    let list: &mut List = &mut *(DTORS.get() as *mut List);
    list.push((t, dtor));

    unsafe extern fn run_dtors(mut ptr: *mut u8) {
        while !ptr.is_null() {
            let list: Box<List> = Box::from_raw(ptr as *mut List);
            for &(ptr, dtor) in list.iter() {
                dtor(ptr);
            }
            ptr = DTORS.get();
            DTORS.set(ptr::null_mut());
        }
    }
}

I am a Rust beginner thought so kind of out of my depth with this code!

[alexcrichton]

I've always personally been under the impression that if you want compatibility with glibc version X+ then you must compile against version X. I think I've seen problems like where you compile against glibc Y and because you called memcpy you now can't work with glibc X where X < Y.

I'm not currently sure whether having a weak definition of __cxa_thread_atexit_impl would work or not, but in any case this is at least expected behavior to me in the sense that it should be fixed by compiling against an older glibc version.

[Timmmm]

Sorry I'm a bit confused - did you change your mind since writing "we run on lots older linuxes, as well as cross compiling from a newer linux to an older linux, so we also have a fallback implementation to use as well."? That suggests to me that you thought this should work at one time.

Anyway, compiling against an old version of glibc is a massive pain. The oldest version of GCC that Linaro provide still uses glibc 2.19.

I'm willing to try the above code but to be honest I have no idea how to cross-compile a new version of libstd.so from Windows. If you can give me a hint of the steps I will give it a go and see if it works!

[alexcrichton]

So that comment isn't really related to this scenario, compiling against a newer linux and then running on an older one. Once you've compiled for a particular glibc version you can never take that binary backwards (in theory).

Yeah it's true though that if you're compiling from Windows to Linux you'll basically just have to figure out how to get your hands on an older glibc. If you're on Linux you can typically use docker to achieve this easily, but that may not be too applicable to you unfortunately :(