Crash when tracing specialized normal class call at deep Python recursion

[chgnrdv]

Crash report

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

Python 3.13.0a0 (heads/main:29bc6165ab, Aug 23 2023, 18:18:02) [GCC 10.2.1 20210110]

What happened?

Bisected to 04492cb.

import sys

def trace(frame, event, arg):
    return trace

class Foo:
    def __init__(self):
        pass

def f():
    Foo()
    f()

sys.settrace(trace)
f()

Error messages

Program received signal SIGSEGV, Segmentation fault.
0x00005555558c3fc0 in allocate_instrumentation_data (code=code@entry=0x555555bc97e0 <_Py_InitCleanup>) at Python/instrumentation.c:1458
1458	        code->_co_monitoring = PyMem_Malloc(sizeof(_PyCoMonitoringData));
(gdb) bt
#0  0x00005555558c3fc0 in allocate_instrumentation_data (code=code@entry=0x555555bc97e0 <_Py_InitCleanup>) at Python/instrumentation.c:1458
#1  0x00005555558c620b in update_instrumentation_data (code=code@entry=0x555555bc97e0 <_Py_InitCleanup>, interp=interp@entry=0x555555cdc7f8 <_PyRuntime+93016>)
    at Python/instrumentation.c:1478
#2  0x00005555558c7522 in _Py_Instrument (code=0x555555bc97e0 <_Py_InitCleanup>, interp=interp@entry=0x555555cdc7f8 <_PyRuntime+93016>) at Python/instrumentation.c:1549
#3  0x00005555558c7d63 in instrument_all_executing_code_objects (interp=interp@entry=0x555555cdc7f8 <_PyRuntime+93016>) at ./Include/internal/pycore_frame.h:84
#4  0x00005555558c7f12 in _PyMonitoring_SetEvents (tool_id=tool_id@entry=7, events=0) at Python/instrumentation.c:1735
#5  0x00005555558ca8a0 in _PyEval_SetTrace (tstate=tstate@entry=0x555555d42108 <_PyRuntime+509032>, func=func@entry=0x0, arg=arg@entry=0x0)
    at Python/legacy_tracing.c:511
#6  0x00005555558fe68c in trace_trampoline (self=<optimized out>, frame=0x7ffff77d5700, what=<optimized out>, arg=<optimized out>) at ./Python/sysmodule.c:1035
#7  0x00005555558c97ae in call_trace_func (self=0x7ffff777e880, arg=0x555555be1600 <_Py_NoneStruct>) at Python/legacy_tracing.c:123
#8  0x00005555558c990c in sys_trace_func2 (self=<optimized out>, args=<optimized out>, nargsf=<optimized out>, kwnames=<optimized out>) at Python/legacy_tracing.c:162
#9  0x00005555558c4fb1 in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=9223372036854775810, args=0x7fffffffdb08, callable=0x7ffff777e880, 
    tstate=0x555555d42108 <_PyRuntime+509032>) at ./Include/internal/pycore_call.h:186
#10 call_one_instrument (interp=interp@entry=0x555555cdc7f8 <_PyRuntime+93016>, tstate=tstate@entry=0x555555d42108 <_PyRuntime+509032>, args=args@entry=0x7fffffffdb08, 
    nargsf=nargsf@entry=-9223372036854775806, tool=tool@entry=7 '\a', event=event@entry=0) at Python/instrumentation.c:850
#11 0x00005555558c5890 in call_instrumentation_vector (tstate=tstate@entry=0x555555d42108 <_PyRuntime+509032>, event=event@entry=0, frame=frame@entry=0x7ffff7fae988, 
    instr=instr@entry=0x7ffff7796e08, nargs=<optimized out>, nargs@entry=2, args=args@entry=0x7fffffffdb00) at Python/instrumentation.c:980
#12 0x00005555558c653b in _Py_call_instrumentation (tstate=tstate@entry=0x555555d42108 <_PyRuntime+509032>, event=event@entry=0, frame=frame@entry=0x7ffff7fae988, 
    instr=instr@entry=0x7ffff7796e08) at Python/instrumentation.c:1015
#13 0x000055555584a828 in _PyEval_EvalFrameDefault (tstate=tstate@entry=0x555555d42108 <_PyRuntime+509032>, frame=0x7ffff7fae988, throwflag=throwflag@entry=0)
    at Python/generated_cases.c.h:43
#14 0x000055555586a69c in _PyEval_EvalFrame (throwflag=0, frame=<optimized out>, tstate=0x555555d42108 <_PyRuntime+509032>) at ./Include/internal/pycore_ceval.h:88
#15 _PyEval_Vector (tstate=tstate@entry=0x555555d42108 <_PyRuntime+509032>, func=func@entry=0x7ffff77407d0, locals=locals@entry=0x7ffff7755070, args=args@entry=0x0, 
    argcount=argcount@entry=0, kwnames=kwnames@entry=0x0) at Python/ceval.c:1622
#16 0x000055555586a73f in PyEval_EvalCode (co=co@entry=0x7ffff791d300, globals=globals@entry=0x7ffff7755070, locals=locals@entry=0x7ffff7755070) at Python/ceval.c:557
#17 0x00005555558eaf8e in run_eval_code_obj (tstate=tstate@entry=0x555555d42108 <_PyRuntime+509032>, co=co@entry=0x7ffff791d300, globals=globals@entry=0x7ffff7755070, 
    locals=locals@entry=0x7ffff7755070) at Python/pythonrun.c:1725
#18 0x00005555558eb994 in run_mod (mod=mod@entry=0x555555e01738, filename=filename@entry=0x7ffff7753140, globals=globals@entry=0x7ffff7755070, 
    locals=locals@entry=0x7ffff7755070, flags=flags@entry=0x7fffffffdf18, arena=arena@entry=0x7ffff7793940) at Python/pythonrun.c:1746
#19 0x00005555558ebaa4 in pyrun_file (fp=fp@entry=0x555555d85aa0, filename=filename@entry=0x7ffff7753140, start=start@entry=257, globals=globals@entry=0x7ffff7755070, 
    locals=locals@entry=0x7ffff7755070, closeit=closeit@entry=1, flags=0x7fffffffdf18) at Python/pythonrun.c:1646
#20 0x00005555558eeced in _PyRun_SimpleFileObject (fp=fp@entry=0x555555d85aa0, filename=filename@entry=0x7ffff7753140, closeit=closeit@entry=1, 
    flags=flags@entry=0x7fffffffdf18) at Python/pythonrun.c:463
#21 0x00005555558eef40 in _PyRun_AnyFileObject (fp=fp@entry=0x555555d85aa0, filename=filename@entry=0x7ffff7753140, closeit=closeit@entry=1, 
    flags=flags@entry=0x7fffffffdf18) at Python/pythonrun.c:79
#22 0x0000555555916e68 in pymain_run_file_obj (program_name=program_name@entry=0x7ffff77a1690, filename=filename@entry=0x7ffff7753140, skip_source_first_line=0)
    at Modules/main.c:360#23 0x0000555555917145 in pymain_run_file (config=config@entry=0x555555cdccf8 <_PyRuntime+94296>) at Modules/main.c:379
#24 0x000055555591820c in pymain_run_python (exitcode=exitcode@entry=0x7fffffffe08c) at Modules/main.c:610
#25 0x0000555555918264 in Py_RunMain () at Modules/main.c:688
#26 0x00005555559182b8 in pymain_main (args=args@entry=0x7fffffffe0d0) at Modules/main.c:718
#27 0x000055555591832d in Py_BytesMain (argc=<optimized out>, argv=<optimized out>) at Modules/main.c:742
#28 0x000055555565073e in main (argc=<optimized out>, argv=<optimized out>) at ./Programs/python.c:15

Linked PRs

• GH-108390: Prevent non-local events being set with sys.monitoring.set_local_events() #108420
• [3.12] GH-108390: Prevent non-local events being set with sys.monitoring.set_local_events() (GH-108420) #108899

[chgnrdv]

@markshannon

[gaogaotiantian]

It seems like the instrumentation system is trying to instrument _Py_InitCleanup which won't work.

The shim frame should be inserted without tracing, but because of the recursion limit is hit, it seems like the shim frame(I might be wrong here, but some frame) generated a max recursion error and triggered the instrumentation system to do an instrumentation on all code objects on the frame stack - which caused the error I mentioned above.

There might be a structural fix for this issue on the specialization part, but if more "read-only" code objects would be added in the future for optimization, maybe it's worth it to have some flag on it (CO_READONLY or CO_NOT_MONITORABLE) so the monitoring system (this might also be applicable to instruction specialization in such code objects in the future) won't try to instrument it.

[markshannon]

We can't mark code objects as read-only, because the PEP states that any code object can be instrumented.
In theory, that would be fine as long as immutable code objects do not produce local events.
Unfortunately, the implementation allows non-local events to be monitored per code-object, so that also needs to be fixed.

So, it looks like we need to do the following:

1. Disallow non local events in set_local_events
2. Make sure that _Py_InitCleanup doesn't generate local events
3. If a code object cannot generate locals events, skip instrumenting it.