Regression in File::read_to_end on 1.57.0 Nightly

[noproto]

PR #89582 (commit 910692d) created a regression in File::read_to_end on the Rust Nightly channel (1.57.0 Nightly). This is before main(), it impacts cdylib projects.

Code

lib.rs

use std::io::prelude::*;

// init_rtld_audit_interface
#[used]
#[allow(non_upper_case_globals)]
#[link_section = ".init_array"]
static init_rtld_audit_interface: unsafe extern "C" fn(i32, *const *const i8, *const *const i8) = {
    #[link_section = ".text.startup"]
    unsafe extern "C" fn init_rtld_audit_interface(_argc: i32, _argv: *const *const i8, _envp: *const *const i8) { read_proc_environ(); }
    init_rtld_audit_interface
};

// la_version
#[no_mangle]
unsafe extern "C" fn la_version(version: u32) -> u32 { version }

fn read_proc_environ() {
    let mut environ = std::fs::File::open("/proc/self/environ").expect("Lost track of environment");
    let mut environ_contents: Vec<u8> = vec![];
    environ.read_to_end(&mut environ_contents).expect("Unexpected null reference");
}

Cargo.toml

# General info
[package]
name = "readenv"
version = "0.1.0"
edition = "2021"

# Build targets
[lib]
name = "readenv"
path = "lib.rs"
crate-type = ["cdylib"]

Makefile

test:
	@echo "Rust nightly-10-09"
	cargo +nightly-2021-10-09 build --lib --release;LD_AUDIT=${PWD}/target/release/libreadenv.so /usr/bin/test || true
	@echo "Rust nightly-10-10"
	cargo +nightly-2021-10-10 build --lib --release;LD_AUDIT=${PWD}/target/release/libreadenv.so /usr/bin/test || true

I expected to see this happen: (no output)

Rust nightly-10-09
cargo +nightly-2021-10-09 build --lib --release;LD_AUDIT=/home/user/readenv/target/release/libreadenv.so /usr/bin/test || true
   Compiling readenv v0.1.0 (/home/user/readenv)
    Finished release [optimized] target(s) in 0.55s
Rust nightly-10-10
cargo +nightly-2021-10-10 build --lib --release;LD_AUDIT=/home/user/readenv/target/release/libreadenv.so /usr/bin/test || true
   Compiling readenv v0.1.0 (/home/user/readenv)
    Finished release [optimized] target(s) in 0.44s

Instead, this happened: Segmentation fault

Rust nightly-10-09
cargo +nightly-2021-10-09 build --lib --release;LD_AUDIT=/home/user/readenv/target/release/libreadenv.so /usr/bin/test || true
   Compiling readenv v0.1.0 (/home/user/readenv)
    Finished release [optimized] target(s) in 0.55s
Rust nightly-10-10
cargo +nightly-2021-10-10 build --lib --release;LD_AUDIT=/home/user/readenv/target/release/libreadenv.so /usr/bin/test || true
   Compiling readenv v0.1.0 (/home/user/readenv)
    Finished release [optimized] target(s) in 0.44s
Segmentation fault

Version it worked on

It most recently worked on: Rust 1.57.0-nightly, nightly-2021-10-09-aarch64-unknown-linux-gnu

Version with regression

Regression occurred in: Rust 1.57.0-nightly, nightly-2021-10-10-aarch64-unknown-linux-gnu

rustc --version --verbose:

rustc 1.57.0-nightly (a8f2463c6 2021-10-09)
binary: rustc
commit-hash: a8f2463c68a6532d74a13ec402ec5b513e4e2726
commit-date: 2021-10-09
host: aarch64-unknown-linux-gnu
release: 1.57.0-nightly
LLVM version: 13.0.0

@rustbot modify labels: +regression-from-stable-to-nightly -regression-untriaged

[jkugelman]

It does not crash on my PC, which is host: x86_64-unknown-linux-gnu.

What kind of ARM device are you using? I will try it on my Raspberry Pi 4 tonight.

[noproto]

It does not crash on my PC, which is host: x86_64-unknown-linux-gnu.

What kind of ARM device are you using? I will try it on my Raspberry Pi 4 tonight.

The crashes are occuring on two aarch64 virtual machines of Ubuntu 20.04 ARM LTS (Parallels running on Apple M1). It's interesting that the behavior isn't on x86_64, I'll check on my x86_64 device now.

[noproto]

Reproduced on Ubuntu 20.04 x86_64 LTS.

Rust nightly-10-09
cargo +nightly-2021-10-09 build --lib --release;LD_AUDIT=/home/user/readenv/target/release/libreadenv.so /usr/bin/test || true
   Compiling readenv v0.1.0 (/home/user/readenv)
    Finished release [optimized] target(s) in 1.09s
Rust nightly-10-10
cargo +nightly-2021-10-10 build --lib --release;LD_AUDIT=/home/user/readenv/target/release/libreadenv.so /usr/bin/test || true
   Compiling readenv v0.1.0 (/home/user/readenv)
    Finished release [optimized] target(s) in 0.88s
Segmentation fault

[jkugelman]

I can reproduce it on my WSL2 machine running Linux 5.10.16 x86_64.

[jkugelman]

The segfault is coming from the code I added to File::read_to_end. It queries the file size so it can reserve that much space in the output Vec<u8>.

Tracing through to the crash location, it is segfaulting in a function called try_statx on this statement:

rust/library/std/src/sys/unix/fs.rs

Lines 135 to 141 in 3c8f001

	// It is a trick to call `statx` with null pointers to check if the syscall
	// is available. According to the manual, it is expected to fail with EFAULT.
	// We do this mainly for performance, since it is nearly hundreds times
	// faster than a normal successful call.
	let err = cvt(statx(0, ptr::null(), 0, libc::STATX_ALL, ptr::null_mut()))
	.err()
	.and_then(|e| e.raw_os_error());

For context, this code tries to get file metadata on Linux by calling the statx() syscall. This system call was added in Linux 4.11 and isn't available in earlier kernel versions, so this line tries to detect whether the syscall is available by calling it with dummy null pointers and seeing if it returns EFAULT.

This code works perfectly fine in normal circumstances, however in the LD_AUDIT preload context it is crashing for some reason. I don't yet know why, though I think this is as deep as I can trace the Rust std code.

Other notes:

• I checked with strace and the statx syscall does not show up in the output. I take it to mean that the process is crashing before the syscall is able to be invoked.

• I was unable to reproduce the crash earlier on a CentOS 7 machine running Linux 3.10, where the statx syscall is not supported. I haven't tested it to confirm, but I expect that on that machine Rust will still end up at this line of code. I wonder why it doesn't segfault there when statx is missing?