Scoped thread implicit join doesn't wait for thread locals to be dropped

[pvillela]

See also #116179:

When a scoped thread is implicitly joined, the destructors of thread-local variables are not guaranteed to have completed when the scope is exited. When a scoped thread is explicitly joined, however, the destructors of thread-local variables do complete before the scope is exited.

[..]

---

Scoped thread implicit join fails 'happens before' guarantee

As documented in the Rust Atomics and Locks book by Mara Bos, "joining a thread creates a happens-before relationship between the joined thread and what happens after the join() call". This is not true for implicit joins on scoped threads.

I tried this code:

use std::{hint::black_box, mem::replace, thread};

static mut CONTROL: Option<String> = None;

fn main() {
    // Explicit join of scoped thread.
    {
        for _ in 0..5000 {
            thread::scope(|s| {
                let h = s.spawn(|| {
                    FOO.with(|v| {
                        black_box(v);
                    });
                });
                h.join().unwrap();
            });

            // SAFETY: this happens after the thread join, which provides a `happens before` guarantee
            let v = unsafe { &CONTROL };
            black_box(format!("{:?}", v));
            print!(".");
        }
        println!("Completed EXPLICIT join loop.");
    }

    println!();

    // Implicit join of scoped thread.
    {
        for _ in 0..5000 {
            thread::scope(|s| {
                let _h = s.spawn(|| {
                    FOO.with(|v| {
                        black_box(v);
                    });
                });
            });

            // SAFETY: this happens after the implicit thread join, which should provide a `happens before` guarantee
            let v = unsafe { &CONTROL };
            black_box(format!("{:?}", v));
            print!(".");
        }
        println!("Completed IMPLICIT join loop.");
    }
}

struct Foo(());

impl Drop for Foo {
    fn drop(&mut self) {
        // SAFETY: this happens before the thread join, which provides a `happens before` guarantee
        let _: Option<String> = unsafe { replace(&mut CONTROL, Some("abcd".to_owned())) };
    }
}

thread_local! {
    static FOO: Foo = Foo(());
}

I expected to see this happen: Both the explicit join and implicit join portions of the code should terminate normally.

Instead, this happened: The explicit join portion terminates normally as expected, but the implicit join portion panics (the number of iterations on the loop before it panics varies) .

This issue is related to issue #116179.

Meta

The same behaviour is observed on the nightly version nightly-x86_64-unknown-linux-gnu unchanged - rustc 1.74.0-nightly (0288f2e 2023-09-25).

rustc --version --verbose:

rustc 1.72.1 (d5c2e9c34 2023-09-13)
binary: rustc
commit-hash: d5c2e9c342b358556da91d61ed4133f6f50fc0c3
commit-date: 2023-09-13
host: x86_64-unknown-linux-gnu
release: 1.72.1
LLVM version: 16.0.5

Backtrace

thread 'main' panicked at 'byte index 5 is out of bounds of `hP�^`', library/core/src/fmt/mod.rs:2324:30
stack backtrace:
   0: rust_begin_unwind
             at /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/std/src/panicking.rs:593:5
   1: core::panicking::panic_fmt
             at /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/core/src/panicking.rs:67:14
   2: core::str::slice_error_fail_rt
   3: core::str::slice_error_fail
             at /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/core/src/str/mod.rs:87:9
   4: core::str::traits::<impl core::slice::index::SliceIndex<str> for core::ops::range::Range<usize>>::index
             at /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/core/src/str/traits.rs:235:21
   5: core::str::traits::<impl core::ops::index::Index<I> for str>::index
             at /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/core/src/str/traits.rs:61:15
   6: <str as core::fmt::Debug>::fmt
             at /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/core/src/fmt/mod.rs:2324:30
   7: <alloc::string::String as core::fmt::Debug>::fmt
             at /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/alloc/src/string.rs:2271:9
   8: <&T as core::fmt::Debug>::fmt
             at /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/core/src/fmt/mod.rs:2268:62
   9: core::fmt::builders::DebugTuple::field::{{closure}}
             at /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/core/src/fmt/builders.rs:322:17
  10: core::result::Result<T,E>::and_then
             at /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/core/src/result.rs:1319:22
  11: core::fmt::builders::DebugTuple::field
             at /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/core/src/fmt/builders.rs:309:35
  12: core::fmt::Formatter::debug_tuple_field1_finish
             at /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/core/src/fmt/mod.rs:2035:9
  13: <core::option::Option<T> as core::fmt::Debug>::fmt
             at /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/core/src/option.rs:559:37
  14: <&T as core::fmt::Debug>::fmt
             at /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/core/src/fmt/mod.rs:2268:62
  15: core::fmt::rt::Argument::fmt
             at /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/core/src/fmt/rt.rs:138:9
  16: core::fmt::write
             at /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/core/src/fmt/mod.rs:1094:21
  17: core::fmt::Write::write_fmt
             at /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/core/src/fmt/mod.rs:192:9
  18: alloc::fmt::format::format_inner
             at /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/alloc/src/fmt.rs:610:16
  19: alloc::fmt::format::{{closure}}
             at /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/alloc/src/fmt.rs:614:34
  20: core::option::Option<T>::map_or_else
             at /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/core/src/option.rs:1180:21
  21: alloc::fmt::format
             at /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/alloc/src/fmt.rs:614:5
  22: thread_local_destruction_in_scoped_thread2::main
             at ./general/src/bin/thread_local_destruction_in_scoped_thread2.rs:41:23
  23: core::ops::function::FnOnce::call_once
             at /rustc/d5c2e9c342b358556da91d61ed4133f6f50fc0c3/library/core/src/ops/function.rs:250:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

[workingjubilee]

You know, technically, it only says it has the "happens before" on the API docs of ScopedJoinHandle::join. I'm guessing there being a difference is not intended behavior, however. cc @m-ou-se

[workingjubilee]

This error is spotty, sometimes it happens, sometimes it doesn't. It happens since at least 1.63, when this function was stabilized.

[workingjubilee]

nope, seems this has been happening since day -128, checked with 7737e0b 2022-04-04 using

slightly modified code for version changes

#![feature(bench_black_box)]
#![feature(scoped_threads)]
use std::{hint::black_box, mem::replace, thread};

static mut CONTROL: Option<String> = None;

fn main() {
    // Explicit join of scoped thread.
    {
        for _ in 0..5000 {
            thread::scope(|s| {
                let h = s.spawn(|_| {
                    FOO.with(|v| {
                        black_box(v);
                    });
                });
                h.join().unwrap();
            });

            // SAFETY: this happens after the thread join, which provides a `happens before` guarantee
            let v = unsafe { &CONTROL };
            black_box(format!("{:?}", v));
            print!(".");
        }
        println!("Completed EXPLICIT join loop.");
    }

    println!();

    // Implicit join of scoped thread.
    {
        for _ in 0..5000 {
            thread::scope(|s| {
                let _h = s.spawn(|_| {
                    FOO.with(|v| {
                        black_box(v);
                    });
                });
            });

            // SAFETY: this happens after the implicit thread join, which should provide a `happens before` guarantee
            let v = unsafe { &CONTROL };
            black_box(format!("{:?}", v));
            print!(".");
        }
        println!("Completed IMPLICIT join loop.");
    }
}

struct Foo(());

impl Drop for Foo {
    fn drop(&mut self) {
        // SAFETY: this happens before the thread join, which provides a `happens before` guarantee
        let _: Option<String> = unsafe { replace(&mut CONTROL, Some("abcd".to_owned())) };
    }
}

thread_local! {
    static FOO: Foo = Foo(());
}