MUSL dynamic linking support

[zhangyt26]

i noticed current musl target doesn't support dynamic linking:

rust/src/librustc_back/target/linux_musl_base.rs

Line 63 in 936b32a

// MUSL support doesn't currently include dynamic linking, so there's no

i was working to make an executable shared lib, but linked statically with musl, so that the library doesn't expect glibc from host system, and it can also be called from other languages dynamically. i was hoping dylib or cdylib for musl target is going to help me with that.

i also tried the following:
i follow the tutorial here: https://doc.rust-lang.org/book/advanced-linking.html#linux
and at the end of the compilation, instead of an executable, i chose staticlib as crate-type.
the above steps work fine.

after getting the staticlib .a file, i was trying to use:
musl-gcc -shared -o libexample.so -Wl,--whole-archive libexample.a
to convert the staticlib to dynamic lib, but no luck.
bunch of multiple definitions and missing definitions error.

is this even possible with musl? a shared lib with musl statically linked?

[alexcrichton]

Right now this is unfortunately not possible without modifying the compiler because, as you've noticed, the compiler hard-codes that it's linking statically to musl. We can probably change this, however, by adding a new target or adding some flags!

[zhangyt26]

@alexcrichton thx for the quick reply!
how about using --whole-archive to convert the musl staticlib to dynamic lib? do you have any knowledge to know if it actually makes sense?
i'm not sure how rust compiler currently packages the staticlib, but usually you can do that for c code....

[alexcrichton]

It may be possible yeah to take the same objects and use them, but the linkage of the C library and other related pieces are hardcoded in places that may be difficult to change, so this likely won't be a small patch one way or the other.

[sorear]

Are we sure musl actually supports that? Having two separate libcs in one process sounds like a recipe for problems (both due to symbol interposition, and unique kernel resources like SIGRT1).

[KennethAdamMiller]

I actually would like to just make my voice heard and let people know that I could really use this feature myself.

I've been in the process of following another route, which is adding another compile target for alternative libc implementations and editing the musl target to remove the static flag that's always getting passed in. Currently I'm stuck adding that target with a compile error where it's telling to add a file that already exists (you can see my changes on my fork). And I'm just starting down the road of adding a dynamic library target for musl, and I'm planning to both in parallel as best I can.

If anybody could tell me what I'm doing wrong with my fork changes, please let me know. I've done my best to follow the cross-compilation guides that are available for producing a rust compiler and std library that are targeted to uclibc

[KennethAdamMiller]

Oh, turns out that I just fixed my current fork issue, and am able to move past that compilation error for the new uclibc target. If I complete it, I will put in a merge request.

On another note, I haven't had a chance to start the dynamic musl target yet.

[japaric]

I haven't had a chance to start the dynamic musl target yet.

I got dynamic musl working. Check this thread; there's a gist to build a dynamic musl in a docker container there.

[KennethAdamMiller]

Wow! Awesome!

Ok, I can start down parallel paths then XD

[aidanhs]

We recently landed support for targets that use a dynamic musl (like alpine linux) in #40113, so that's all fine and lays the groundwork for someone adding a ulibc target (should be available in the next nightly).

However, the original issue as raised was

i was working to make an executable shared lib, but linked statically with musl, so that the library doesn't expect glibc from host system, and it can also be called from other languages dynamically.

As @sorear points out, this is impossible to do in the general case (maybe for very restricted individual use-cases) and you'll get no warnings at all if you get it wrong. I recommend you read http://www.openwall.com/lists/musl/2012/12/08/4 for all of the implications, but here's an extract to start you off:

now you're potentially using two
different versions of libc in the same program; if
implementation-internal data structures like FILE or the pthread
structure are not identical between the 2 versions, you'll hit an ABI
incompatibility, despite the fact that these data structures were
intended to be implementation-internal and never affect ABI. Even
without that issue, you have issues like potentially 2 copies of
malloc trying to manage the heap without being aware of one another,
and thus clobbering it

So if you have code that exposes no internal libc data structures, doesn't try and access thread locals or do anything with threads and doesn't call malloc, then it might be plausible, otherwise you're likely to struggle.

I'll close this issue now we have musl dynamic linking and given the above post about 'static shared libraries', but feel free to open issues if you encounter problems with our musl support!