As of 1.37.0, `dylib` shared libraries no longer support interpositioning of functions defined in C

[solb]

I encountered a stable-to-stable linking regression while working with a project that compiles to an x86-64 ELF shared library for Linux. The library exports a Rust interface, but also includes some wrappers of libc functions, written in C, that need to shadow the system implementations via interpositioning. I perform the final linking step using rustc, which works correctly under rustc 1.36.0 but subtly fails under 1.37.0: the libc wrapper functions are not exported as dynamic symbols, causing the program to behave differently at runtime. The offending patchset appears to be #59752, and I've managed to construct a minimal example to illustrate the problem...

Minimal example

The library consists of two files:

• interpose.rs defines a single-function Rust API:

#![crate_type = "dylib"]

pub fn relax() {
	println!("Relax said the night guard");
}

• exit.c defines an implementation of exit() that should shadow the one in libc:

#include <stdio.h>
#include <stdlib.h>

void exit(int ign) {
	(void) ign;
	puts("We are programmed to receive");
}

The following short program, hotel_california.rs, will be used to test its behavior:

extern crate interpose;

use interpose::relax;
use std::os::raw::c_int;

extern {
	// NB: Deliberately returns () instead of ! for the purpose of this example.
	fn exit(_: c_int);
}

fn main() {
	relax();
	unsafe { exit(1); }
	println!("You can check out any time you like but you can never leave");
}

Expected behavior (past stable releases)

This is how the program used to behave when built with a stable compiler:

$ rustc --version
rustc 1.36.0 (a53f9df32 2019-07-03)
$ rustc -Cprefer-dynamic -Clink-arg=exit.o interpose.rs
$ rustc -L. -Crpath hotel_california.rs
$ ./hotel_california
Relax said the night guard
We are programmed to receive
You can check out any time you like but you can never leave
$ echo $?
0

Notice that the call to exit() gets intercepted and does not, in fact, exit the program.

Broken behavior (as of 1.37.0 stable)

Newer versions of the compiler result in different program output:

$ rustc --version
rustc 1.37.0
$ rustc -Cprefer-dynamic -Clink-arg=exit.o interpose.rs
$ rustc -L. -Crpath hotel_california.rs
$ ./hotel_california
Relax said the night guard
$ echo $?
1

Discussion: symbol table entries

The problem is evident upon examining the static and dynamic symbol tables of the libinterpose.so file. When built with rustc 1.36.0, we see that exit is exported in the dynamic symbol table (indicated by the D):

$ objdump -tT libinterpose.so | grep exit$
000000000000118c g     F .text  000000000000001a exit
000000000000118c g    DF .text  000000000000001a exit

In contrast, the output from rustc 1.37.0 doesn't list exit in the dynamic symbol table because the static symbol table lists it as a local symbol (l) rather than a global one (g):

$ objdump -tT libinterpose.so | grep exit$
000000000000118c l     F .text  000000000000001a exit

Discussion: linker invocation

I was curious to see how rustc was invoking cc to link the program, so I traced the command-line arguments by substituting the fake linker false. Here's with rustc 1.36.0:

$ rustc -Clinker=false -Cprefer-dynamic -Clink-arg=exit.o interpose.rs
error: linking with `false` failed: exit code: 1
  |
  = note: "false" "-Wl,--as-needed" "-Wl,-z,noexecstack" "-m64" "-L" "/home/solb/Desktop/rust-1.36/lib/rustlib/x86_64-unknown-linux-gnu/lib" "interpose.interpose.3a1fbbbh-cgu.0.rcgu.o" "interpose.interpose.3a1fbbbh-cgu.1.rcgu.o" "-o" "libinterpose.so" "in
terpose.54bybojgvbim5uqh.rcgu.o" "-Wl,-zrelro" "-Wl,-znow" "-nodefaultlibs" "-L" "/home/solb/Desktop/rust-1.36/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,--start-group" "-L" "/home/solb/Desktop/rust-1.36/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-lst
d-9895e8982b0a79e7" "-Wl,--end-group" "-Wl,-Bstatic" "/tmp/user/1000/rustchJWjrY/libcompiler_builtins-38e90baf978bc428.rlib" "-Wl,-Bdynamic" "-ldl" "-lrt" "-lpthread" "-lgcc_s" "-lc" "-lm" "-lrt" "-lpthread" "-lutil" "-lutil" "-shared" "exit.o"
  = note: 

error: aborting due to previous error

And with rustc 1.37.0:

$ rustc -Clinker=false -Cprefer-dynamic -Clink-arg=exit.o interpose.rs
error: linking with `false` failed: exit code: 1
  |
  = note: "false" "-Wl,--as-needed" "-Wl,-z,noexecstack" "-m64" "-L" "/usr/lib/rustlib/x86_64-unknown-linux-gnu/lib" "interpose.interpose.3a1fbbbh-cgu.0.rcgu.o" "interpose.interpose.3a1fbbbh-cgu.1.rcgu.o" "-o" "libinterpose.so" "-Wl,--version-script=/tmp/
user/1000/rustc7Re7af/list" "interpose.54bybojgvbim5uqh.rcgu.o" "-Wl,-zrelro" "-Wl,-znow" "-nodefaultlibs" "-L" "/usr/lib/rustlib/x86_64-unknown-linux-gnu/lib" "-Wl,--start-group" "-L" "/usr/lib/x86_64-linux-gnu" "-lstd-6c8733432f42c6a2" "-Wl,--end-group"
 "-Wl,-Bstatic" "/tmp/user/1000/rustc7Re7af/libcompiler_builtins-67541964815c9eb5.rlib" "-Wl,-Bdynamic" "-ldl" "-lrt" "-lpthread" "-lgcc_s" "-lc" "-lm" "-lrt" "-lpthread" "-lutil" "-lutil" "-shared" "-Wl,-soname=libinterpose.so" "exit.o"
  = note: 

error: aborting due to previous error

Notice the newly-added -Wl,--version-script flag, which has no knowledge of the symbols from the exit.o object file.

Discussion: static library instead of bare object file

One might be tempted to work around the problem by telling rustc about the object file so it can keep the symbols it defines global. I tried this on rustc 1.36.0:

$ ar rs libexit.a exit.o
ar: creating libexit.a
$ rustc -Cprefer-dynamic -L. interpose.rs -lexit
$ rustc -L. -Crpath hotel_california.rs

This has a very surprising result: the exit symbol is not present at all in libinterpose.so, but it does exist somewhere (the LLVM bitcode for monomorphization, maybe?) that allows the compiler to statically link it into the executable:

$ objdump -tT libinterpose.so | grep exit$
$ objdump -tT hotel_california | grep exit$
0000000000001337 g     F .text  000000000000001a              exit
0000000000001337 g    DF .text  000000000000001a  Base        exit

This is no good either because it leads to subtly different interposition behavior. For example:

• Before, exit() could be further shadowed by libraries loaded via the LD_PRELOAD environment variable. Building it directly into the executable breaks this.
• The cc and ld apply very different optimizations to exit() because it is now part of a PIE instead of a PIC object; depending on how wrapping is implemented, this can break it and even result in infinite recursion.
• If a C program links against libinterpose.so, it will no longer get the interposed version of exit(). This is a very real situation for my project, because it also exports a C API via Rust's FFI.

Possible mitigation: expose a -Climit_rdylib_exports command-line switch

The simplest way to allow users to work around this would be to allow invokers of rustc to opt out of the change introduced by #59752. However, the change is likely to have broken other use cases as well, so perhaps it needs to be revisited in more detail.

See also

The same changeset seems to be causing problems with inline functions, as observed at #65610.

[nagisa]

Related #65610

[Zoxc]

This can be worked around with a #[link(name = "foo", type = "static")] attribute in interpose.rs on an public extern block declaring exit so rustc knows that needs to be exported.

[itamarst]

I hit this issue too, and the proposed workaround is not working for me, at least.

[solb]

I agree that the proposed workaround does not work. I'm still stuck on Rust 1.36 because of this change, but I've prototyped the following workaround to remove the --version-script linker flag:

1. Paste the following wrapper script into a file (named cc, for instance):

#!/bin/sh

args=""
for arg in "$@"
do
	case "$arg" in
	-Wl,--version-script*)
		;;
	*)
		args="$args '$arg'"
	esac
done

arg0="`basename "$0"`"
eval exec "'$arg0'"$args

2. Make the script executable: $ chmod +x cc

3. When compiling the library, tell rustc to use this wrapper script instead of the system linker directly, e.g.: $ rustc -Clinker=./cc -Cprefer-dynamic -Clink-arg=exit.o interpose.rs

(If using cargo, you can accomplish this as a one-off by doing $ cargo rustc -- -Clinker=./cc instead of $ cargo build, or permanently by modifying the build flags via a config file at .cargo/config.)

Sorry for the delay in noticing you needed this, @itamarst!

[o0Ignition0o]

@rustbot modify labels to +I-prioritize

[spastorino]

Assigning P-medium as discussed as part of the Prioritization Working Group process and removing I-prioritize.