Skip to content

Make multiprocessing SemLock thread-safe in the free-threaded build #117435

Closed
@colesbury

Description

@colesbury

Feature or enhancement

The multiprocessing SemLock code maintains an internal count field. When operating as a recursive mutex, count is the number of times the thread has acquired the mutex (i.e., 0, 1, .. N). When operating as a semaphore, count is often 0 or 1, but can be negative if the SemLock is initialized with maxvalue > 1

The modification of count is not thread-safe without the GIL within the process. Note that the count field is not shared across processes, unlike the underlying sem_t or HANDLE.

In particular, the modification of count after the semaphore is released is not thread-safe without the GIL:

if (sem_post(self->handle) < 0)
return PyErr_SetFromErrno(PyExc_OSError);
--self->count;
Py_RETURN_NONE;

I think this is the source of deadlocks when running test_multiprocessing_forkserver.test_processes with the GIL disabled.

Linked PRs

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions