link: failed to static link to c++ library when global variable is used

Hi, I am trying to build a binary written in rust static linked with a c++ library. I follow the instructions in official documentation, but end up with an error：

undefined reference to `__dso_handle'

Following is a simple example:

// test.cpp
#include <vector>

const std::vector<int> ids = {2, 3, 4};

extern "C" {
    extern int get_id(int idx);
}

int get_id(int idx) {
    return ids[idx];
}

// main.rs
#[link(name = "test")]
extern "C" {
    pub fn get_id(idx: usize) -> usize;
}

fn main() {
    unsafe {
        println!("id at {} is {}", 2, get_id(2));
    }
}

I build the static library using following command:

g++ -std=c++0x -c test.cpp -o test.o
ar rcs libtest.a test.o

And build the binary using following command:

LIBRARY_PATH=./ rustc main.rs -lstdc++ --target=x86_64-unknown-linux-musl

It failed with error:

note: ./libtest.a(test.o): In function __static_initialization_and_destruction_0(int, int)': test.cpp:(.text+0xa2): undefined reference to__dso_handle'
/usr/local/lib/rustlib/x86_64-unknown-linux-musl/lib/libstd-8102e29f.rlib(std-8102e29f.0.o):(.data._rust_extern_with_linkage___dso_handle+0x0): undefined reference to `__dso_handle'

I can build the binary successfully when linking dynamically. Am I missing any thing here? Any help is appreciated.

You are required to define two functions and one symbol in order to use global objects in your C++ files: __dso_handle symbol, __cxa_atexit and __cxa_finalize functions. These are part of Itanium C++ ABI and thus rustc has no responsibility of providing any of the symbols or calling any of the functions.

As an aside, that C++ snippet probably does not do what you expect it to do.

I don't get it, does it mean I can't static link to libraries that is written in C++ and uses global variables? Or is there an extra library that I should link to?

This is just a simple example to reproduce the failure. And it actually works.

does it mean I can't static link to libraries that is written in C++ and uses global variables

Not without extra effort on your part, no. Remember that you’re mixing the GNU C runtime (which defines the __dso_handle in an object file somewhere at /usr/lib/gcc/<target>/<version>/crtbegin.o) and MUSL runtime which probably has its own internal symbols and stuff to do that.

Or is there an extra library that I should link to?

Anything related to linking more libraries would likely not work as it would introduce symbol conflict between runtimes. Instead, either build your C++ object targetting MUSL, or use the -gnu target on the rust side.

Ah yes indeed, if g++ is a GNU gcc then linking those object files into a Rust musl program won't work (as you're seeing with weird symbol errors), unfortunately.

From what I can tell, this isn't a bug in rustc. Closing.

After keeping my CPU warm for a while by compiling:

musl
binutils
libunwind
g++ (targeting musl)

I now have in my hands a musl-compatible crtbegin(T/S).o and crtend(S?).o and, by compiling a modified rustc (only to stage2 thankfully 😥) and making the link command be:

cc .. crt1.o crti.o crtbeginT.o ... crtend.o crtn.o

I am able to create a static binary containing Rust and C++ code (in my case I was trying to compile the example code from https://github.com/tupshin/cassandra-rs which uses a C++ driver). So, this is possible 🎉 🎈!

Now, not everyone is going to go to the trouble I did above, but there are good use cases for static binaries (for one, they actually run everywhere, unlike "mostly static" binaries that are still glibc version dependent), and it would be nice to be able to modify the "early objects" and "late objects" lists that get passed to the linker.

#![link(name=crtbegin.o)] or cargo:rustc-link-lib=static=crtbeginT.o aren't sufficient since they don't control the order the extra libraries are added to the link command (also I'm not sure they work for .o files but I could easily make a libcrtbeginT.a if this was going to work).

@bossmc

@bossmc is it possible you could share this modified rustc? I am currently working on a cross compiling toolchain and I have the crt object files you mention... but of course rust fails on a C++ library.

@alexcrichton

Not from this PC, but the only change I made (I think 🤞) was to modify https://github.com/rust-lang/rust/blob/master/src/librustc_back/target/linux_musl_base.rs#L59-L61 to add the crtBeginT.o and crtEnd.o at the appropriate points (and to copy those files to the same path rustc installs it's copy of crti.o etc).

There are a few alternatives I've considered since:

Rust brings it's own C++ runtime objects (as it does with C runtime on musl)
- Seems dangerous, I doubt libc++ from GNU and from LLVM have compatible run-time objects
Rust allows the linker to add the run-time objects (as it does with the -gnu target)
- I'm not sure it's possible to ask the linker to add the run-time objects and not add the rest of the standard libraries
- It's not clear how to work out whether to include the C++ run-time, maybe include it unilaterally?
Add config flags to rustc (and Cargo's config - https://doc.rust-lang.org/cargo/reference/config.html#configuration-keys) to specify extra run-time objects to be included in the link commands
- Seems like a decent solution - given that I don't think rustc can assume the provenance of the libc++.a like it can with the musl libc.a (though even that is a little dubious in my mind)
Check if simply creating libcrt++.a from crtBeginT.o and crtEnd.o then adding cargo:rustc-link-lib=static=crt++ is sufficient
- If this works it's a pretty good workaround, but doesn't feel like a good long-term solution (what if you link in two crates that both link to C++ libraries... you only want the runtime objects once)
Use a custom linker wrapper script which adds these objects to the objects rustc asks for
- Sensible for a cross compiler, not so nice for a native compiler

@alexcrichton - I'd quite like to help resolve this, who should I be working with/talking to? Do we need an RFC?

@bossmc

@bossmc er sorry this is pretty out of cache for me, but if you've got some linking questions or whatnot feel free to hit me up on IRC!

Fair enough, you were the last name I recognised as a core dev to comment. I'll shout in IRC at some point when I've got some time.

@bossmc

For whom it may concern, I used the last suggestion @bossmc of creating a custom linker script. After some troubleshooting, this is what I came up with that works:

#!/bin/bash

args=()

for arg in "$@"; do
	if [[ $arg = *"Bdynamic"* ]]; then
		args+=() # we do not want this arg
	elif [[ $arg = *"crti.o"* ]]; then
		args+=("$arg" "/usr/arm-unknown-linux-musleabi/lib/gcc/arm-unknown-linux-musleabi/6.3.0/crtbeginT.o" "-Bstatic")
	elif [[ $arg = *"crtn.o"* ]]; then
		args+=("-lgcc" "-lgcc_eh" "-lc" "/usr/arm-unknown-linux-musleabi/lib/gcc/arm-unknown-linux-musleabi/6.3.0/crtend.o" "$arg")
	else
		args+=("$arg")
	fi
done
#echo "RUNNING WITH ARGS: ${args[@]}"
arm-unknown-linux-musleabi-real-g++ "${args[@]}"

The first if removes Bdynamic args, which is for some reason still included in static linking. Then, it adds crtbegin.o after it sees crti.o, and similar for crtn.o. For some reason a bunch of compiler symbols were still missing after I added those options, so I also added -lgcc -lgcc_eh -lc to ensure other libraries that were missing were included.

It might be a bit sloppy, but that is what worked for me.

@jonas-schievink

@jonas-schievink I think that means that a self-contained link will still not work for a C++ binary (even if the C++ bit is built with musl libc)? Is that just inevitable?

For now, yes, that still won't work.

Is that just inevitable?

CRT objects are not guaranteed to be compatible between different compilers, so the answer is yes.

Rollup merge of rust-lang#77901 - jonas-schievink:unignore-test-36710…

Auto merge of rust-lang#77901 - jonas-schievink:unignore-test-36710, …

Fix to build with C++ environment

Navigation Menu

Repository navigation

link: failed to static link to c++ library when global variable is used #36710

39 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

link: failed to static link to c++ library when global variable is used #36710

Description

Activity

nagisa commented on Sep 25, 2016

BusyJay commented on Sep 26, 2016

nagisa commented on Sep 26, 2016

alexcrichton commented on Sep 27, 2016

Mark-Simulacrum commented on May 14, 2017

bossmc commented on Jan 24, 2018

dl00 commented on Feb 6, 2018

bossmc commented on Feb 7, 2018

alexcrichton commented on Feb 7, 2018

bossmc commented on Feb 7, 2018

dl00 commented on Feb 10, 2018

39 remaining items

bossmc commented on Oct 15, 2020

jonas-schievink commented on Oct 15, 2020

mati865 commented on Oct 15, 2020

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions