Use stop-the-world to make fork and interpreter shutdown thread-safe

[colesbury]

Feature or enhancement

In the free-threaded build, we should use a stop-the-world call to make fork() ¹ and shutdown (i.e, Py_FinalizeEx) thread-safe.

The two operations are similar in that they both involve deleting thread states for other threads.

Fork

Fork and in multithreaded environments is notorious for the problems it causes, but we can make the free-threaded build at least as safe as the default (GIL) build by pausing other threads with a stop-the-world call before forking. In the parent, we can use a normal start-the-world call to resume other threads after fork. In the child, we want to call start the world after other thread states are removed, but before we execute python code. This may require some refactoring.

Shutdown

The interpreter waits for threading module created non-daemon threads during shutdown, but this can be canceled by a ctrl-c and there may still be active thread states for daemon threads and threads created from C outside of the threading module. We should use a stop-the-world call to bring these threads to a consistent state before we try deleting their thread states.

Linked PRs

• gh-116522: Stop the world before fork() and during shutdown #116607
• gh-116522: Refactor _PyThreadState_DeleteExcept #117131

Footnotes

1. or at least less thread-unsafe... ↩

[Rajendra1308]

Hey @colesbury I would like to work on this, can I get more info. And what file to modify. I am a beginner to this repo, please help.

[colesbury]

Hi @Rajendra1308, thanks for your interest but I'm already working on this.

[Rajendra1308]

Hey @colesbury , alright. Good Luck!