Negative refcount crash in test_monitoring with -Xuops

[gvanrossum]

Bug report

Bug description:

./python.exe -Xuops -m test test_monitoring -v

Full output

== CPython 3.13.0a2+ (heads/main:4c4b08dd2bd, Jan 3 2024, 14:05:09) [Clang 15.0.0 (clang-1500.1.0.2.5)]
== macOS-14.1.2-arm64-arm-64bit little-endian
== Python build: debug pystats
== cwd: /Users/guido/cpython/build/test_python_worker_40548æ
== CPU count: 12
== encodings: locale=UTF-8 FS=utf-8
== resources: all test resources are disabled, use -u option to unskip tests

Using random seed: 414366294
0:00:00 load avg: 2.46 Run 1 test sequentially
0:00:00 load avg: 2.46 [1/1] test_monitoring
test_async_for (test.test_monitoring.ExceptionMonitoringTest.test_async_for) ... ok
test_explicit_reraise (test.test_monitoring.ExceptionMonitoringTest.test_explicit_reraise) ... ok
test_explicit_reraise_named (test.test_monitoring.ExceptionMonitoringTest.test_explicit_reraise_named) ... ok
test_implicit_reraise (test.test_monitoring.ExceptionMonitoringTest.test_implicit_reraise) ... ok
test_implicit_reraise_named (test.test_monitoring.ExceptionMonitoringTest.test_implicit_reraise_named) ... ok
test_implicit_stop_iteration (test.test_monitoring.ExceptionMonitoringTest.test_implicit_stop_iteration) ... ok
test_simple_try_except (test.test_monitoring.ExceptionMonitoringTest.test_simple_try_except) ... ok
test_throw (test.test_monitoring.ExceptionMonitoringTest.test_throw) ... ok
test_try_finally (test.test_monitoring.ExceptionMonitoringTest.test_try_finally) ... ok
test_branch (test.test_monitoring.LineMonitoringTest.test_branch) ... ok
test_linear (test.test_monitoring.LineMonitoringTest.test_linear) ... ok
test_lines_loop (test.test_monitoring.LineMonitoringTest.test_lines_loop) ... ok
test_lines_single (test.test_monitoring.LineMonitoringTest.test_lines_single) ... ok
test_lines_two (test.test_monitoring.LineMonitoringTest.test_lines_two) ... ok
test_try_except (test.test_monitoring.LineMonitoringTest.test_try_except) ... ok
test_has_objects (test.test_monitoring.MonitoringBasicTest.test_has_objects) ... ok
test test_monitoring crashed -- Traceback (most recent call last):
  File "/Users/guido/cpython/Lib/test/libregrtest/single.py", line 178, in _runtest_env_changed_exc
    _load_run_test(result, runtests)
    ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "/Users/guido/cpython/Lib/test/libregrtest/single.py", line 135, in _load_run_test
    regrtest_runner(result, test_func, runtests)
    ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/guido/cpython/Lib/test/libregrtest/single.py", line 88, in regrtest_runner
    test_result = test_func()
                  ~~~~~~~~~^^
  File "/Users/guido/cpython/Lib/test/libregrtest/single.py", line 132, in test_func
    return run_unittest(test_mod)
           ~~~~~~~~~~~~^^^^^^^^^^
  File "/Users/guido/cpython/Lib/test/libregrtest/single.py", line 37, in run_unittest
    return _run_suite(tests)
           ~~~~~~~~~~^^^^^^^
  File "/Users/guido/cpython/Lib/test/libregrtest/single.py", line 57, in _run_suite
    result = runner.run(suite)
             ~~~~~~~~~~^^^^^^^
  File "/Users/guido/cpython/Lib/unittest/runner.py", line 240, in run
    test(result)
    ~~~~^^^^^^^^
  File "/Users/guido/cpython/Lib/unittest/suite.py", line 84, in __call__
    return self.run(*args, **kwds)
           ~~~~~~~~^^^^^^^^^^^^^^^
  File "/Users/guido/cpython/Lib/unittest/suite.py", line 122, in run
    test(result)
    ~~~~^^^^^^^^
  File "/Users/guido/cpython/Lib/unittest/suite.py", line 84, in __call__
    return self.run(*args, **kwds)
           ~~~~~~~~^^^^^^^^^^^^^^^
  File "/Users/guido/cpython/Lib/unittest/suite.py", line 107, in run
    for index, test in enumerate(self):
    ...<19 lines>...
            self._removeTestAtIndex(index)
TypeError: 'enumerate' object does not support the asynchronous context manager protocol

test_monitoring failed (uncaught exception)

== Tests result: FAILURE ==

1 test failed:
    test_monitoring

Total duration: 56 ms
Total tests: run=0
Total test files: run=1/1 failed=1
Result: FAILURE
Objects/codeobject.c:1493: _Py_NegativeRefcount: Assertion failed: object has negative ref count
<object at 0x13a2c8820 is freed>
Fatal Python error: _PyObject_AssertFailed: _PyObject_AssertFailed
Python runtime state: finalizing (tstate=0x0000000100d27968)

Current thread 0x00000001da335ec0 (most recent call first):
  Garbage-collecting
  <no Python frame>
Abort trap: 6

CPython versions tested on:

CPython main branch

Operating systems tested on:

macOS

Linked PRs

• GH-113689: Fix broken handling of invalid executors #113694

[gvanrossum]

@markshannon Any idea? If I add -i test_has_objects the tests pass, so it may be something that test does. But running just that test passes -- I think it leaves something bad behind.

[brandtbucher]

I think in the past this particular error message has been caused by jumping into a cache. The error message comes from BEFORE_ASYNC_WITH, which has the same value as a fresh warmup counter (1).

I think we have an off-by-one error somewhere.

[brandtbucher]

I think I might have found it: dc8df6e doesn't rewind the current instruction before DISPATCH_GOTO() when an executor is invalidated.

I'll try out a fix.

[brandtbucher]

(Also, I think this further illustrates the need for a debug CI job with PYTHON_UOPS=1. Tier two is being broken several times a week at this point, which is painful for JIT development.)