Miri hangs in spin loops forever

[RalfJung]

The following program loops forever in Miri, but terminates as expected with normal rustc:

use std::thread;
use std::sync::atomic::{AtomicUsize, Ordering};

static FLAG: AtomicUsize = AtomicUsize::new(0);

fn main() {
    let j = thread::spawn(|| {
        while FLAG.load(Ordering::Acquire) == 0 {
            // spin and wait
        }
    });
    thread::yield_now();
    FLAG.store(1, Ordering::Release);
    j.join().unwrap();
}

One could argue the spin loop should use hint::spin_loop();, but that is unstable -- and it seems reasonable to expect the scheduler to de-schedule a thread at some point even if it does not yield.

Cc @vakaras

[oli-obk]

We could do round robin, where every terminator causes the next thread to get activated.

[RalfJung]

Ah turns out there is a stable version of the same intrinsic, std::sync::atomic::spin_loop_hint().

We could do round robin, where every terminator causes the next thread to get activated.

I was more thinking "switch every N terminators", but N = 1 is certainly a possible choice. ;) We might make this configurable even.

[RalfJung]

There's an interesting testcase in once_cell where even adding a spin loop hint does not help. My guess is that the scheduler keeps swapping between two of the spinning threads so that the others are starved.

[RalfJung]

I think here is another example where yield_now is insufficient:
https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=f3ee4d476185291766d413ddeaa425ad
This test hangs in Miri; I assume what happens is that the two threads with their while !THREAD2_LAUNCHED.load(Ordering::SeqCst) loops are playing ping-pong and the thread that would set THREAD2_LAUNCHED never gets run.

I assume @JCTyblaidd's PR #1651 would help here.

[vakaras]

I think here is another example where yield_now is insufficient:
https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=f3ee4d476185291766d413ddeaa425ad
This test hangs in Miri; I assume what happens is that the two threads with their while !THREAD2_LAUNCHED.load(Ordering::SeqCst) loops are playing ping-pong and the thread that would set THREAD2_LAUNCHED never gets run.

This is the most likely scenario because the threads with the lower ID get priority (simply because we pick them by traversing the list).

[Kixunil]

Got bitten by this recently when I forgot to call thread::park() and attributed the spin loop to slowness/experimental nature of Miri. 😅 Tried to look into the docs and didn't find it. Found it when refactoring the code later.

I think it'd be useful to document this for now, I can send a PR if you agree.
Long-term, if some kind of hint is required, would it be feasible to have some heuristics where Miri could detect basic instances of this and warn?

Anyway, thanks for your great work on this tool! I feel much better now that I see my unsafe code is probably correct. :)

[RalfJung]

Long-term, if some kind of hint is required, would it be feasible to have some heuristics where Miri could detect basic instances of this and warn?

Seems better to use a different scheduling algorithm instead. One version of that started in #1651, but was not completed. A simpler alternative would be a round-robin scheduler.

Until that happens, I agree documenting this better would be good, and a PR would be definitely welcome. :)