[subinterpreters] Per-interpreter singletons (None, True, False, etc.)

[vstinner]

BPO	39511
Nosy	@rhettinger, @vstinner, @ericsnowcurrently, @mloskot, @pablogsal, @h-vetinari, @JunyiXie, @KubaO
PRs	bpo-39511: PyThreadState_Clear() calls on_delete #18296 bpo-39511: Fix multiprocessing semlock_acquire() #18298 [WIP] bpo-39511: Add Py_GetNone() and Py_GetNoneRef() functions #18301 bpo-43503: Make limited API objects effectively immutable. #24828

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2020-01-31.15:33:35.261>
labels = ['expert-subinterpreters', '3.9']
title = '[subinterpreters] Per-interpreter singletons (None, True, False, etc.)'
updated_at = <Date 2022-01-28.13:50:38.344>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2022-01-28.13:50:38.344>
actor = 'vstinner'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Subinterpreters']
creation = <Date 2020-01-31.15:33:35.261>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 39511
keywords = ['patch']
message_count = 32.0
messages = ['361121', '361122', '361137', '361138', '361145', '361146', '361161', '361163', '361164', '361256', '361257', '361258', '361355', '361356', '361357', '361380', '361382', '361543', '364384', '364429', '364446', '364469', '364471', '364485', '366342', '368937', '388409', '388417', '388894', '388921', '411512', '411996']
nosy_count = 8.0
nosy_names = ['rhettinger', 'vstinner', 'eric.snow', 'mloskot', 'pablogsal', 'h-vetinari', 'JunyiXie', 'KubaO']
pr_nums = ['18296', '18298', '18301', '24828']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue39511'
versions = ['Python 3.9']

[vstinner]

The long-term goal of the PEP-554 is to run two Python interpreters in parallel. To achieve this goal, no object must be shared between two interpreters. See for example my article "Pass the Python thread state explicitly" which gives a longer rationale:
https://vstinner.github.io/cpython-pass-tstate.html

In bpo-38858, I modified Objects/longobject.c to have per-interpreter small integer singletons: commit 630c8df.

This issue is about other singletons like None or Py_True which are currently shared between two interpreters.

I propose to add new functions. Example for None:

• Py_GetNone(): return a *borrowed* reference to the None singleton (similar to existing Py_None macro)
• Py_GetNoneRef(): return a *strong* reference to the None singleton (similar to "Py_INCREF(Py_None); return Py_None;" and Py_RETURN_NONE macro)

And add PyInterpreterState.none field: strong reference to the per-interpreter None object.

We should do that for each singletons:

• None (Py_None)
• True (Py_True)
• False (Py_False)
• Ellipsis (Py_Ellipsis)

GIL issue
=========

Py_GetNone() would look like:

PyObject* Py_GetNone(void)
{ return _PyThreadState_GET()->interp->none; }

Problem: _PyThreadState_GET() returns NULL if the caller function doesn't hold the GIL.

Using the Python C API when the GIL is not held is a violation of the API: it is not supported. But it worked previously.

One solution is to fail with an assertion error (abort the process) in debug mode, and let Python crash in release mode.

Another option is to only fail with an assertion error in debug mode in Python 3.9. In Python 3.9, Py_GetNone() would use PyGILState_GetThisThreadState() function which works even when the GIL is released. In Python 3.10, we would switch to _PyThreadState_GET() and so crash in release mode.

One concrete example of such issue can be found in the multiprocessing C code, in semlock_acquire():

            Py_BEGIN_ALLOW_THREADS
            if (timeout_obj == Py_None) {
                res = sem_wait(self->handle);
            }
            else {
                res = sem_timedwait(self->handle, &deadline);
            }
            Py_END_ALLOW_THREADS

Py_None is accessed when the GIL is released.

[jkloth]

Would it not suffice to just make the singletons "immortal"?

Without affecting the hotpaths that are Py_INCREF and Py_DECREF, changing _Py_Dealloc to test for objects with a "special" destructor could be used:

    destructor dealloc = Py_TYPE(op)->tp_dealloc;
    if (dealloc == _Py_SingletonSentinel) {
        /* reset refcnt so as to not return here too often */
        op->ob_refcnt = PY_SSIZE_T_MAX;
    }
    else {
        (*dealloc)(op);
    }

Even in the presence of multiple mutating threads, the object cannot be destroyed. Worst case, they all call _Py_Dealloc.

[vstinner]

New changeset 7dc1401 by Victor Stinner in branch 'master':
bpo-39511: Fix multiprocessing semlock_acquire() (GH-18298)
7dc1401

[vstinner]

Would it not suffice to just make the singletons "immortal"?

The problem is to make Py_INCREF/Py_DECREF efficient. Last time someone tried to use an atomic variable for ob_refcnt, it was 20% slower if I recall correctly. If many threads start to update such atomic variable, the CPU cacheline of common singletons like None, True and False can quickly become a performance bottleneck.

On the other side, if each interpreter has its own objects, there is no need to protect ob_refcnt, the interpreter lock protects it.

[vstinner]

New changeset 4d96b46 by Victor Stinner in branch 'master':
bpo-39511: PyThreadState_Clear() calls on_delete (GH-18296)
4d96b46

[vstinner]

PR 18301 is a WIP showing my intent. I'm not sure if it would be possible to land such change right now in Python. It has different drawbacks described in my previous messages. I don't know the impact on performance neither.

[jeremykloth]

The problem is to make Py_INCREF/Py_DECREF efficient.

That is exactly why I didn't propose a change to them. The singletons
still are refcounted as usual, just that their ob_refcnt is ignored.
If they somehow reach 0, they just "resurrect" themselves and ignore
the regular collection behavior. In the presence of multiple
DECREF'ers, the ob_refcnt field is garbage, but that is OK as it is
effectively ignored. Practicality vs Purity and all that.

Last time someone tried to use an atomic variable for ob_refcnt, it was 20% slower if I recall correctly. If many threads start to update such atomic variable, the CPU cacheline of common singletons like None, True and False can quickly become a performance bottleneck.

Exactly so, hence why I chose the simple solution of effectively
ignoring ob_refcnt.

On the other side, if each interpreter has its own objects, there is no need to protect ob_refcnt, the interpreter lock protects it.

My solution also does not need any protection around ob_refcnt.

[vstinner]

I vaguely recall discussions about immortal Python objects.

(*) Instagram gc.freeze()

• https://docs.python.org/dev/library/gc.html#gc.freeze
• https://instagram-engineering.com/dismissing-python-garbage-collection-at-instagram-4dca40b29172

(*) Python immortal strings

• PyUnicode_InternImmortal()
• SSTATE_INTERNED_IMMORTAL
• They are only destroyed if Python is compiled with Valgrind or Purify support: unicode_release_interned() function

(*) COUNT_ALLOCS

• When Python is built with COUNT_ALLOCS macro defined, types are immortal
• Many tests have to be skipped if COUNT_ALLOCS is used
• I plan to remove COUNT_ALLOCS feature in bpo-39489 :-)

(*) Static types

• Types which are not allocated on the heap but "static" are immortal
• These types cause various issues and should go away
• For example, they are incompatible with multi-phase initialization module (PEP-489) and stable ABI (PEP-384)

[vstinner]

Recently, Petr Viktorin proposed immortal singletons in my latest "Pass the Python thread state to internal C functions" thread on python-dev list:
https://mail.python.org/archives/list/python-dev@python.org/message/RAVSH7HYHTROXSTUR3677WGTCTEO6FYF/

In 2004, Jewett, Jim J proposed:

"What if a few common (constant, singleton) objects (such as None, -1, 0, 1) were declared immortal at compile-time?"

https://mail.python.org/archives/list/python-dev@python.org/thread/XWGRUATMRAVXXZKQ7QA2YX22KI55T7P5/#OQO7DWRVH7IIOE4RUJ2ZR7S5UT6WOCPS

[rhettinger]

Is the sub-interpreter PEP approved? If not, I had thought the plan was to only implement PRs that made clean-ups that would have been necessary anyway.

[rhettinger]

Random idea (not carefully thought-out): Would it be simpler to have these objects just ignore their refcount by having dealloc() be a null operation or having it set the refcount back to a positive number). That would let sub-interpreters share the objects without worrying about race-conditions on incref/decref operations. To make this work, the objects can register themselves as permanent, shared, objects; then, during shutdown, we could explicitly call a hard dealloc on those objects.