Race in `Thread.join()`

[mpage]

Bug report

Bug description:

There is a race between when Thread._tstate_lock is released¹ in Thread._wait_for_tstate_lock() and when Thread._stop() asserts² that it is unlocked. Consider the following execution involving threads A, B, and C:

1. A starts.
2. B joins A, blocking on its _tstate_lock.
3. C joins A, blocking on its _tstate_lock.
4. A finishes and releases its _tstate_lock.
5. B acquires A's _tstate_lock in _wait_for_tstate_lock(), releases it, but is swapped out before calling _stop().
6. C is scheduled, acquires A's _tstate_lock in _wait_for_tstate_lock() but is swapped out before releasing it.
7. B is scheduled, calls _stop(), which asserts that A's _tstate_lock is not held. However, C holds it, so the assertion fails.

The race can be reproduced³ by inserting sleeps at the appropriate points in the threading code. To do so, run the repro_join_race.py from the linked repo.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Footnotes

1. https://github.com/python/cpython/blob/441affc9e7f419ef0b68f734505fa2f79fe653c7/Lib/threading.py#L1201 ↩

2. https://github.com/python/cpython/blob/441affc9e7f419ef0b68f734505fa2f79fe653c7/Lib/threading.py#L1115 ↩

3. https://github.com/mpage/cpython/commit/81946532792f938cd6f6ab4c4ff92a4edf61314f ↩