Running `atexit` from threads in free-threading build segfaults

[devdanzin]

Crash report

What happened?

It's possible to segfault or abort a no-gil interpreter running with PYTHON_GIL=0 by calling atexit functions from threads:

from threading import Thread
import atexit

def g():
    pass

def f():
    for x in range(100):
        atexit.register(g)
        atexit._clear()
        atexit.register(g)

for x in range(100):
    Thread(target=f, args=()).start()

Found using fusil by @vstinner.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

Python 3.14.0a1+ experimental free-threading build (heads/main-dirty:612ac283b81, Nov 16 2024, 01:37:56) [GCC 11.4.0]

Linked PRs

• gh-126907: Lock the atexit state on the free-threaded build #126908
• gh-126907: Use a list for atexit callbacks #127935

[kumaraditya303]

Please also post the stack trace preferably with a debug build for ease with crashers.

[ZeroIntensity]

At a glance, atexit just needs some locks on the interpreter state field. I can do it.

[devdanzin]

Please also post the stack trace preferably with a debug build for ease with crashers.

Sorry, will do from now on.

I get this backtrace for a slightly modified code (because original code wasn't triggering under gdb):

#0  atexit_delete_cb (state=state@entry=0x555555cfd8f8 <_PyRuntime+140088>, i=i@entry=0)
    at ./Modules/atexitmodule.c:58
#1  0x000055555598d962 in atexit_cleanup (state=0x555555cfd8f8 <_PyRuntime+140088>)
    at ./Modules/atexitmodule.c:75
#2  0x000055555598d9b6 in atexit_clear (module=<optimized out>, unused=<optimized out>)
    at ./Modules/atexitmodule.c:249
#3  0x0000555555702405 in cfunction_vectorcall_NOARGS (
    func=<built-in method _clear of module object at remote 0x200007752c0>, args=<optimized out>,
    nargsf=<optimized out>, kwnames=<optimized out>) at Objects/methodobject.c:495
#4  0x000055555567cc55 in _PyObject_VectorcallTstate (tstate=0x555555dc8fe0,
    callable=<built-in method _clear of module object at remote 0x200007752c0>, args=0x7ffff7c42b48,
    nargsf=9223372036854775808, kwnames=0x0) at ./Include/internal/pycore_call.h:167
#5  0x000055555567cd74 in PyObject_Vectorcall (
    callable=callable@entry=<built-in method _clear of module object at remote 0x200007752c0>,
    args=args@entry=0x7ffff7c42b48, nargsf=<optimized out>, kwnames=kwnames@entry=0x0)
    at Objects/call.c:327
#6  0x00005555558441f7 in _PyEval_EvalFrameDefault (tstate=tstate@entry=0x555555dc8fe0,
    frame=<optimized out>, throwflag=throwflag@entry=0) at Python/generated_cases.c.h:955
#7  0x0000555555872978 in _PyEval_EvalFrame (throwflag=0, frame=<optimized out>, tstate=0x555555dc8fe0)
    at ./Include/internal/pycore_ceval.h:116
#8  _PyEval_Vector (tstate=<optimized out>, func=0x20000a8b530, locals=locals@entry=0x0,
    args=0x7ffff7c42cd8, argcount=1, kwnames=<optimized out>) at Python/ceval.c:1901
#9  0x000055555567c622 in _PyFunction_Vectorcall (func=<optimized out>, stack=<optimized out>,
    nargsf=<optimized out>, kwnames=<optimized out>) at Objects/call.c:413
#10 0x00005555556816b4 in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=1, args=0x7ffff7c42cd8,
    callable=<function at remote 0x20000a8b530>, tstate=0x555555dc8fe0)
    at ./Include/internal/pycore_call.h:167

Code that caused it:

from threading import Thread
import atexit

def g():
    pass

def f():
    for x in range(100):
        atexit.register(g)
        atexit._clear()
        atexit.register(g)
        atexit.unregister(g)
        atexit._run_exitfuncs()

for x in range(100):
    Thread(target=f, args=()).start()

Also possible to get this error message:

Debug memory block at address p=0x20002110f90: API '�'
    9789596714021935601 bytes originally requested
    The 7 pad bytes at p-7 are not all FORBIDDENBYTE (0xfd):
        at p-7: 0xdd *** OUCH
        at p-6: 0xdd *** OUCH
        at p-5: 0xdd *** OUCH
        at p-4: 0xdd *** OUCH
        at p-3: 0xdd *** OUCH
        at p-2: 0xdd *** OUCH
        at p-1: 0xdd *** OUCH
    Because memory is corrupted at the start, the count of bytes requested
       may be bogus, and checking the trailing pad bytes may segfault.
    The 8 pad bytes at tail=0x87dba4580bace981 are 
Thread 84 "python" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7ffff6430640 (LWP 828561)]

In that case the backtrace is:

#0  0x000055555570fa78 in _PyObject_DebugDumpAddress (p=p@entry=0x20002110f90) at Objects/obmalloc.c:3016
#1  0x000055555570fde6 in _PyMem_DebugCheckAddress (
    func=func@entry=0x555555a6c860 <__func__.8> "_PyMem_DebugRawFree", api=<optimized out>,
    p=p@entry=0x20002110f90) at Objects/obmalloc.c:2956
#2  0x000055555571390a in _PyMem_DebugRawFree (ctx=ctx@entry=0x555555cdb928 <_PyRuntime+872>,
    p=p@entry=0x20002110f90) at Objects/obmalloc.c:2762
#3  0x0000555555713986 in _PyMem_DebugFree (ctx=0x555555cdb928 <_PyRuntime+872>, ptr=0x20002110f90)
    at Objects/obmalloc.c:2904
#4  0x000055555572c740 in PyMem_Free (ptr=ptr@entry=0x20002110f90) at Objects/obmalloc.c:1018
#5  0x000055555598d93c in atexit_delete_cb (state=state@entry=0x555555cfd8f8 <_PyRuntime+140088>,
    i=i@entry=1) at ./Modules/atexitmodule.c:61
#6  0x000055555598dc94 in atexit_unregister (module=<optimized out>,
    func=<function at remote 0x20000a8dbb0>) at ./Modules/atexitmodule.c:291
#7  0x00005555557021ef in cfunction_vectorcall_O (
    func=<built-in method unregister of module object at remote 0x200007752c0>, args=<optimized out>,
    nargsf=<optimized out>, kwnames=<optimized out>) at Objects/methodobject.c:523
#8  0x000055555567cc55 in _PyObject_VectorcallTstate (tstate=0x555555dc8fe0,
    callable=<built-in method unregister of module object at remote 0x200007752c0>, args=0x7ffff642fb48,
    nargsf=9223372036854775809, kwnames=0x0) at ./Include/internal/pycore_call.h:167
#9  0x000055555567cd74 in PyObject_Vectorcall (
    callable=callable@entry=<built-in method unregister of module object at remote 0x200007752c0>,
    args=args@entry=0x7ffff642fb48, nargsf=<optimized out>, kwnames=kwnames@entry=0x0)
    at Objects/call.c:327
#10 0x00005555558441f7 in _PyEval_EvalFrameDefault (tstate=tstate@entry=0x555555dc8fe0,
    frame=<optimized out>, throwflag=throwflag@entry=0) at Python/generated_cases.c.h:955
#11 0x0000555555872978 in _PyEval_EvalFrame (throwflag=0, frame=<optimized out>, tstate=0x555555dc8fe0)
    at ./Include/internal/pycore_ceval.h:116

[ZeroIntensity]

Eh, stack traces are generally not that helpful on the free-threaded build (mimalloc screws up most error messages), especially in a data race.