Tracking test_gil_scoped.py `DEADLOCK`s

[rwgk]

Using this Issue to track test_gil_scoped.py failures like this one:

E               RuntimeError: DEADLOCK, most likely, exactly what this test is meant to detect.

Background: PRs #4105, #4306

[rwgk]

PR #4371

https://github.com/pybind/pybind11/actions/runs/3590629225/jobs/6044221872

CI — 🐍 3.6 • macos-latest • x64

platform darwin -- Python 3.6.15, pytest-7.0.0, pluggy-1.0.0
C++ Info: Apple LLVM 14.0.0 (clang-1400.0.29.102) C++11 __pybind11_internals_v4_clang_libcpp_cxxabi1002_sh_def__ PYBIND11_SIMPLE_GIL_MANAGEMENT=True
...
=================================== FAILURES ===================================
_______________ test_run_in_process_direct[test_nested_acquire] ________________

test_fn = <function test_nested_acquire at 0x104218e18>

    @pytest.mark.parametrize("test_fn", ALL_BASIC_TESTS_PLUS_INTENTIONAL_DEADLOCK)
    def test_run_in_process_direct(test_fn):
        """Makes sure there is no GIL deadlock when using processes.
    
        This test is for completion, but it was never an issue.
        """
>       assert _run_in_process(test_fn) == 0

test_fn    = <function test_nested_acquire at 0x104218e18>

test_gil_scoped.py:244: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

target = <function test_nested_acquire at 0x104218e18>, args = (), kwargs = {}
test_fn = <function test_nested_acquire at 0x104218e18>, timeout = 10
process = <Process(Process-70, stopped[255] daemon)>
t_start = 1669882422.6540549, t_delta = 10.150235176086426, @py_assert1 = None

    def _run_in_process(target, *args, **kwargs):
        if len(args) == 0:
            test_fn = target
        else:
            test_fn = args[0]
        # Do not need to wait much, 10s should be more than enough.
        timeout = 0.1 if test_fn is _intentional_deadlock else 10
        process = multiprocessing.Process(target=target, args=args, kwargs=kwargs)
        process.daemon = True
        try:
            t_start = time.time()
            process.start()
            if timeout >= [100](https://github.com/pybind/pybind11/actions/runs/3590629225/jobs/6044221872#step:12:101):  # For debugging.
                print(
                    "\nprocess.pid STARTED", process.pid, (sys.argv, target, args, kwargs)
                )
                print(f"COPY-PASTE-THIS: gdb {sys.argv[0]} -p {process.pid}", flush=True)
            process.join(timeout=timeout)
            if timeout >= 100:
                print("\nprocess.pid JOINED", process.pid, flush=True)
            t_delta = time.time() - t_start
            if process.exitcode == 66 and m.defined_THREAD_SANITIZER:  # Issue #2754
                # WOULD-BE-NICE-TO-HAVE: Check that the message below is actually in the output.
                # Maybe this could work:
                # https://gist.github.com/alexeygrigorev/01ce847f2e721b513b42ea4a6c96905e
                pytest.skip(
                    "ThreadSanitizer: starting new threads after multi-threaded fork is not supported."
                )
            elif test_fn is _intentional_deadlock:
                assert process.exitcode is None
                return 0
            elif process.exitcode is None:
                assert t_delta > 0.9 * timeout
                msg = "DEADLOCK, most likely, exactly what this test is meant to detect."
                if env.PYPY and env.WIN:
                    pytest.skip(msg)
>               raise RuntimeError(msg)
E               RuntimeError: DEADLOCK, most likely, exactly what this test is meant to detect.

args       = ()
kwargs     = {}
msg        = 'DEADLOCK, most likely, exactly what this test is meant to detect.'
process    = <Process(Process-70, stopped[255] daemon)>
t_delta    = 10.150235176086426
t_start    = 1669882422.6540549
target     = <function test_nested_acquire at 0x[104](https://github.com/pybind/pybind11/actions/runs/3590629225/jobs/6044221872#step:12:105)218e18>
test_fn    = <function test_nested_acquire at 0x104218e18>
timeout    = 10

test_gil_scoped.py:186: RuntimeError

[rwgk]

PR #4382

https://github.com/pybind/pybind11/actions/runs/3615276319/jobs/6092282270

🐍 pypy-3.9 • macos-latest • x64

platform darwin -- Python 3.9.12[pypy-7.3.9-final], pytest-7.0.0, pluggy-1.0.0
C++ Info: Apple LLVM 14.0.0 (clang-1400.0.29.102) C++11 __pybind11_internals_v4_clang_libcpp_cxxabi1002__ PYBIND11_SIMPLE_GIL_MANAGEMENT=True
...
test_gil_scoped.py ..................................................... [ 46%]
.......F.............................                                    [ 53%]
test_iostream.py ......................                                  [ 57%]
test_kwargs_and_defaults.py ........                                     [ 58%]
test_local_bindings.py ..........                                        [ 60%]
test_methods_and_attributes.py ............xx........                    [ 64%]
        process.daemon = True
        try:
            t_start = time.time()
            process.start()
            if timeout >= 100:  # For debugging.
                print(
                    "\nprocess.pid STARTED", process.pid, (sys.argv, target, args, kwargs)
                )
                print(f"COPY-PASTE-THIS: gdb {sys.argv[0]} -p {process.pid}", flush=True)
            process.join(timeout=timeout)
            if timeout >= 100:
                print("\nprocess.pid JOINED", process.pid, flush=True)
            t_delta = time.time() - t_start
            if process.exitcode == 66 and m.defined_THREAD_SANITIZER:  # Issue #2754
                # WOULD-BE-NICE-TO-HAVE: Check that the message below is actually in the output.
                # Maybe this could work:
                # https://gist.github.com/alexeygrigorev/01ce847f2e721b513b42ea4a6c96905e
                pytest.skip(
                    "ThreadSanitizer: starting new threads after multi-threaded fork is not supported."
                )
            elif test_fn is _intentional_deadlock:
                assert process.exitcode is None
                return 0
            elif process.exitcode is None:
                assert t_delta > 0.9 * timeout
                msg = "DEADLOCK, most likely, exactly what this test is meant to detect."
                if env.PYPY and env.WIN:
                    pytest.skip(msg)
>               raise RuntimeError(msg)
E               RuntimeError: DEADLOCK, most likely, exactly what this test is meant to detect.

args       = (<function test_cross_module_gil_inner_custom_released at 0x00007fe8111f8ac0>,)
kwargs     = {'num_threads': 8, 'parallel': False}
msg        = 'DEADLOCK, most likely, exactly what this test is meant to detect.'
process    = <Process name='Process-43' pid=9763 parent=8413 stopped exitcode=-SIGTERM daemon>
t_delta    = 10.249207973480225
t_start    = 1670[19](https://github.com/pybind/pybind11/actions/runs/3615276319/jobs/6092282270#step:12:20)1997.6[23](https://github.com/pybind/pybind11/actions/runs/3615276319/jobs/6092282270#step:12:24)723
target     = <function _run_in_threads at 0x00007fe8109294c0>
test_fn    = <function test_cross_module_gil_inner_custom_released at 0x00007fe8111f8ac0>
timeout    = 10

test_gil_scoped.py:186: RuntimeError

[rwgk]

PR #4384

https://github.com/pybind/pybind11/actions/runs/3618069828/jobs/6097514565

🐍 3.6 • macos-latest • x64

platform darwin -- Python 3.6.15, pytest-7.0.0, pluggy-1.0.0
C++ Info: Apple LLVM 14.0.0 (clang-1400.0.29.102) C++17 __pybind11_internals_v4_clang_libcpp_cxxabi1002__ PYBIND11_SIMPLE_GIL_MANAGEMENT=False
rootdir: /Users/runner/work/pybind11/pybind11/tests, configfile: pytest.ini
plugins: timeout-2.1.0
timeout: 300.0s
timeout method: signal
timeout func_only: False
collected 790 items

../../tests/test_async.py ..                                             [  0%]
../../tests/test_buffers.py .........                                    [  1%]
../../tests/test_builtin_casters.py ....................                 [  3%]
../../tests/test_call_policies.py ........                               [  4%]
../../tests/test_callbacks.py .............s                             [  6%]
../../tests/test_chrono.py ...........................................   [ 12%]
../../tests/test_class.py ...............................                [ 16%]
../../tests/test_const_name.py ......................                    [ 18%]
../../tests/test_constants_and_functions.py .....                        [ [19](https://github.com/pybind/pybind11/actions/runs/3618069828/jobs/6097514565#step:18:20)%]
../../tests/test_copy_move.py ........                                   [ [20](https://github.com/pybind/pybind11/actions/runs/3618069828/jobs/6097514565#step:18:21)%]
../../tests/test_custom_type_casters.py ...                              [ 20%]
../../tests/test_custom_type_setup.py ..                                 [ [21](https://github.com/pybind/pybind11/actions/runs/3618069828/jobs/6097514565#step:18:22)%]
../../tests/test_docstring_options.py .                                  [ 21%]
../../tests/test_eigen_matrix.py ..............................          [ 25%]
../../tests/test_eigen_tensor.py ....................................... [ 30%]
......................................................................   [ 38%]
../../tests/test_enum.py .........                                       [ 40%]
../../tests/test_eval.py ....                                            [ 40%]
../../tests/test_exceptions.py ........................                  [ 43%]
../../tests/test_factory_constructors.py ...............                 [ 45%]
../../tests/test_gil_scoped.py ........................................F [ 50%]
.................................................                        [ 56%]
../../tests/test_iostream.py ......................                      [ 59%]
            test_fn = target
        else:
            test_fn = args[0]
        # Do not need to wait much, 10s should be more than enough.
        timeout = 0.1 if test_fn is _intentional_deadlock else 10
        process = multiprocessing.Process(target=target, args=args, kwargs=kwargs)
        process.daemon = True
        try:
            t_start = time.time()
            process.start()
            if timeout >= 100:  # For debugging.
                print(
                    "\nprocess.pid STARTED", process.pid, (sys.argv, target, args, kwargs)
                )
                print(f"COPY-PASTE-THIS: gdb {sys.argv[0]} -p {process.pid}", flush=True)
            process.join(timeout=timeout)
            if timeout >= 100:
                print("\nprocess.pid JOINED", process.pid, flush=True)
            t_delta = time.time() - t_start
            if process.exitcode == 66 and m.defined_THREAD_SANITIZER:  # Issue #2754
                # WOULD-BE-NICE-TO-HAVE: Check that the message below is actually in the output.
                # Maybe this could work:
                # https://gist.github.com/alexeygrigorev/01ce847f2e721b513b42ea4a6c96905e
                pytest.skip(
                    "ThreadSanitizer: starting new threads after multi-threaded fork is not supported."
                )
            elif test_fn is _intentional_deadlock:
                assert process.exitcode is None
                return 0
            elif process.exitcode is None:
                assert t_delta > 0.9 * timeout
                msg = "DEADLOCK, most likely, exactly what this test is meant to detect."
                if env.PYPY and env.WIN:
                    pytest.skip(msg)
>               raise RuntimeError(msg)
E               RuntimeError: DEADLOCK, most likely, exactly what this test is meant to detect.

args       = (<function test_cross_module_gil_released at 0x103a96e18>,)
kwargs     = {'num_threads': 8, 'parallel': True}
msg        = 'DEADLOCK, most likely, exactly what this test is meant to detect.'
process    = <Process(Process-23, stopped[255] daemon)>
t_delta    = 10.005295753479004
t_start    = 1670[22](https://github.com/pybind/pybind11/actions/runs/3618069828/jobs/6097514565#step:18:23)8304.4042292
target     = <function _run_in_threads at 0x103a3c730>
test_fn    = <function test_cross_module_gil_released at 0x103a96e18>
timeout    = 10

../../tests/test_gil_scoped.py:186: RuntimeError
=========================== short test summary info ============================
SKIPPED [1] ../../tests/test_callbacks.py:203: Current PYBIND11_INTERNALS_VERSION too low
SKIPPED [1] ../../tests/test_pytypes.py:397: Not defined: PYBIND11_HANDLE_REF_DEBUG
SKIPPED [1] ../../tests/test_stl.py:143: no <experimental/optional>
FAILED ../../tests/test_gil_scoped.py::test_run_in_process_multiple_threads_parallel[test_cross_module_gil_released]
================== 1 failed, 786 passed, 3 skipped in 59.09s ===================

[rwgk]

PR #2768

https://github.com/pybind/pybind11/actions/runs/3655052690/jobs/6176034826

🐍 pypy-3.7 • macos-latest • x64

============================= test session starts ==============================
platform darwin -- Python 3.7.13[pypy-7.3.9-final], pytest-7.0.0, pluggy-1.0.0
C++ Info: Apple LLVM 14.0.0 (clang-1400.0.29.102) C++17 __pybind11_internals_v10000000_clang_libcpp_cxxabi1002__ PYBIND11_SIMPLE_GIL_MANAGEMENT=True
rootdir: /Users/runner/work/pybind11/pybind11/tests, configfile: pytest.ini
plugins: timeout-2.1.0
timeout: 300.0s
timeout method: signal
timeout func_only: False
collected 100 items

../../tests/test_call_policies.py X.x.....                               [  8%]
../../tests/test_gil_scoped.py ......................................... [ 49%]
...............................................F.                        [ 98%]
../../tests/test_thread.py ..                                            [100%]

=================================== FAILURES ===================================
_____ test_run_in_process_direct[test_multi_acquire_release_cross_module] ______

test_fn = <function test_multi_acquire_release_cross_module at 0x00007fac9fcc9380>

    @pytest.mark.parametrize("test_fn", ALL_BASIC_TESTS_PLUS_INTENTIONAL_DEADLOCK)
    def test_run_in_process_direct(test_fn):
        """Makes sure there is no GIL deadlock when using processes.
    
        This test is for completion, but it was never an issue.
        """
>       assert _run_in_process(test_fn) == 0

test_fn    = <function test_multi_acquire_release_cross_module at 0x00007fac9fcc9380>

../../tests/test_gil_scoped.py:244: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

target = <function test_multi_acquire_release_cross_module at 0x00007fac9fcc9380>
args = (), kwargs = {}
test_fn = <function test_multi_acquire_release_cross_module at 0x00007fac9fcc9380>
timeout = 10, process = <Process(Process-71, stopped[SIGTERM] daemon)>
t_start = [16](https://github.com/pybind/pybind11/actions/runs/3655052690/jobs/6176034826#step:22:17)70568874.939685, t_delta = 10.020744800567627, @py_assert1 = None

    def _run_in_process(target, *args, **kwargs):
        if len(args) == 0:
            test_fn = target
        else:
            test_fn = args[0]
        # Do not need to wait much, 10s should be more than enough.
        timeout = 0.1 if test_fn is _intentional_deadlock else 10
        process = multiprocessing.Process(target=target, args=args, kwargs=kwargs)
        process.daemon = True
        try:
            t_start = time.time()
            process.start()
            if timeout >= 100:  # For debugging.
                print(
                    "\nprocess.pid STARTED", process.pid, (sys.argv, target, args, kwargs)
                )
                print(f"COPY-PASTE-THIS: gdb {sys.argv[0]} -p {process.pid}", flush=True)
            process.join(timeout=timeout)
            if timeout >= 100:
                print("\nprocess.pid JOINED", process.pid, flush=True)
            t_delta = time.time() - t_start
            if process.exitcode == 66 and m.defined_THREAD_SANITIZER:  # Issue #2754
                # WOULD-BE-NICE-TO-HAVE: Check that the message below is actually in the output.
                # Maybe this could work:
                # https://gist.github.com/alexeygrigorev/01ce847f2e721b513b42ea4a6c96905e
                pytest.skip(
                    "ThreadSanitizer: starting new threads after multi-threaded fork is not supported."
                )
            elif test_fn is _intentional_deadlock:
                assert process.exitcode is None
                return 0
            elif process.exitcode is None:
                assert t_delta > 0.9 * timeout
                msg = "DEADLOCK, most likely, exactly what this test is meant to detect."
                if env.PYPY and env.WIN:
                    pytest.skip(msg)
>               raise RuntimeError(msg)
E               RuntimeError: DEADLOCK, most likely, exactly what this test is meant to detect.

args       = ()
kwargs     = {}
msg        = 'DEADLOCK, most likely, exactly what this test is meant to detect.'
process    = <Process(Process-71, stopped[SIGTERM] daemon)>
t_delta    = 10.020744800567627
t_start    = 1670568874.939685
target     = <function test_multi_acquire_release_cross_module at 0x00007fac9fcc9380>
test_fn    = <function test_multi_acquire_release_cross_module at 0x00007fac9fcc9380>
timeout    = 10

../../tests/test_gil_scoped.py:[18](https://github.com/pybind/pybind11/actions/runs/3655052690/jobs/6176034826#step:22:19)6: RuntimeError

[rwgk]

PR #4463

https://github.com/pybind/pybind11/actions/runs/3993013804/jobs/6849373017

🐍 3.10 • macos-latest • x64

============================= test session starts ==============================
platform darwin -- Python 3.10.9, pytest-7.0.0, pluggy-1.0.0
C++ Info: Apple LLVM 14.0.0 (clang-1400.0.29.202) C++11 __pybind11_internals_v4_clang_libcpp_cxxabi1002__ PYBIND11_SIMPLE_GIL_MANAGEMENT=True
...
=================================== FAILURES ===================================
_____ test_run_in_process_multiple_threads_sequential[test_nested_acquire] _____

test_fn = <function test_nested_acquire at 0x10ea3d480>

    @pytest.mark.parametrize("test_fn", ALL_BASIC_TESTS_PLUS_INTENTIONAL_DEADLOCK)
    def test_run_in_process_multiple_threads_sequential(test_fn):
        """Makes sure there is no GIL deadlock when running in a thread multiple times sequentially.
    
        It runs in a separate process to be able to stop and assert if it deadlocks.
        """
>       assert _run_in_process(_run_in_threads, test_fn, num_threads=8, parallel=False) == 0

test_fn    = <function test_nested_acquire at 0x10ea3d480>

test_gil_scoped.py:234: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

target = <function _run_in_threads at 0x10ea3d750>
args = (<function test_nested_acquire at 0x10ea3d480>,)
kwargs = {'num_threads': 8, 'parallel': False}
test_fn = <function test_nested_acquire at 0x10ea3d480>, timeout = 10
process = <Process name='Process-52' pid=17490 parent=16338 stopped exitcode=0 daemon>
t_start = 1674534973.682475, t_delta = 10.0825610[160](https://github.com/pybind/pybind11/actions/runs/3993013804/jobs/6849373017#step:12:161)82764, @py_assert1 = None

    def _run_in_process(target, *args, **kwargs):
        if len(args) == 0:
            test_fn = target
        else:
            test_fn = args[0]
        # Do not need to wait much, 10s should be more than enough.
        timeout = 0.1 if test_fn is _intentional_deadlock else 10
        process = multiprocessing.Process(target=target, args=args, kwargs=kwargs)
        process.daemon = True
        try:
            t_start = time.time()
            process.start()
            if timeout >= 100:  # For debugging.
                print(
                    "\nprocess.pid STARTED", process.pid, (sys.argv, target, args, kwargs)
                )
                print(f"COPY-PASTE-THIS: gdb {sys.argv[0]} -p {process.pid}", flush=True)
            process.join(timeout=timeout)
            if timeout >= 100:
                print("\nprocess.pid JOINED", process.pid, flush=True)
            t_delta = time.time() - t_start
            if process.exitcode == 66 and m.defined_THREAD_SANITIZER:  # Issue #2754
                # WOULD-BE-NICE-TO-HAVE: Check that the message below is actually in the output.
                # Maybe this could work:
                # https://gist.github.com/alexeygrigorev/01ce847f2e721b513b42ea4a6c96905e
                pytest.skip(
                    "ThreadSanitizer: starting new threads after multi-threaded fork is not supported."
                )
            elif test_fn is _intentional_deadlock:
                assert process.exitcode is None
                return 0
            elif process.exitcode is None:
                assert t_delta > 0.9 * timeout
                msg = "DEADLOCK, most likely, exactly what this test is meant to detect."
                if env.PYPY and env.WIN:
                    pytest.skip(msg)
>               raise RuntimeError(msg)
E               RuntimeError: DEADLOCK, most likely, exactly what this test is meant to detect.

args       = (<function test_nested_acquire at 0x10ea3d480>,)
kwargs     = {'num_threads': 8, 'parallel': False}
msg        = 'DEADLOCK, most likely, exactly what this test is meant to detect.'
process    = <Process name='Process-52' pid=17490 parent=[163](https://github.com/pybind/pybind11/actions/runs/3993013804/jobs/6849373017#step:12:164)38 stopped exitcode=0 daemon>
t_delta    = 10.082561016082764
t_start    = [167](https://github.com/pybind/pybind11/actions/runs/3993013804/jobs/6849373017#step:12:168)4534973.682475
target     = <function _run_in_threads at 0x10ea3d750>
test_fn    = <function test_nested_acquire at 0x10ea3d480>
timeout    = 10

test_gil_scoped.py:[186](https://github.com/pybind/pybind11/actions/runs/3993013804/jobs/6849373017#step:12:187): RuntimeError
=========================== short test summary info ============================
SKIPPED [1] test_builtin_casters.py:145: no <string_view>
SKIPPED [1] test_callbacks.py:[213](https://github.com/pybind/pybind11/actions/runs/3993013804/jobs/6849373017#step:12:214): Current PYBIND11_INTERNALS_VERSION too low
SKIPPED [1] test_copy_move.py:77: no <optional>
SKIPPED [1] test_pytypes.py:403: Not defined: PYBIND11_HANDLE_REF_DEBUG
SKIPPED [1] test_sequences_and_iterators.py:13: no <optional>
SKIPPED [1] test_stl.py:111: no <optional>
SKIPPED [1] test_stl.py:143: no <experimental/optional>
SKIPPED [1] test_stl.py:234: no <filesystem>
SKIPPED [1] test_stl.py:270: no std::monostate
FAILED test_gil_scoped.py::test_run_in_process_multiple_threads_sequential[test_nested_acquire]
============= 1 failed, 784 passed, 9 skipped in 71.28s (0:01:11) ==============

[rwgk]

PR #4504

https://github.com/pybind/pybind11/actions/runs/4121981206/jobs/7119639395

🐍 pypy-3.8 • macos-latest • x64

============================= test session starts ==============================
platform darwin -- Python 3.8.16[pypy-7.3.11-final], pytest-7.0.0, pluggy-1.0.0
C++ Info: Apple LLVM 14.0.0 (clang-1400.0.29.202) C++11 __pybind11_internals_v4_clang_libcpp_cxxabi1002_sh_def__ PYBIND11_SIMPLE_GIL_MANAGEMENT=True
...
=================================== FAILURES ===================================
_ test_run_in_process_multiple_threads_sequential[test_cross_module_gil_inner_pybind11_released] _

test_fn = <function test_cross_module_gil_inner_pybind11_released at 0x00007f967d9694c0>

    @pytest.mark.parametrize("test_fn", ALL_BASIC_TESTS_PLUS_INTENTIONAL_DEADLOCK)
    def test_run_in_process_multiple_threads_sequential(test_fn):
        """Makes sure there is no GIL deadlock when running in a thread multiple times sequentially.
    
        It runs in a separate process to be able to stop and assert if it deadlocks.
        """
>       assert _run_in_process(_run_in_threads, test_fn, num_threads=8, parallel=False) == 0

test_fn    = <function test_cross_module_gil_inner_pybind11_released at 0x00007f967d9694c0>

test_gil_scoped.py:234: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

target = <function _run_in_threads at 0x00007f967d969f60>
args = (<function test_cross_module_gil_inner_pybind11_released at 0x00007f967d9694c0>,)
kwargs = {'num_threads': 8, 'parallel': False}
test_fn = <function test_cross_module_gil_inner_pybind11_released at 0x00007f967d9694c0>
timeout = 10
process = <Process name='Process-45' pid=18326 parent=15388 stopped exitcode=-SIGTERM daemon>
t_start = [167](https://github.com/pybind/pybind11/actions/runs/4121981206/jobs/7119639395#step:12:168)5848907.445822, t_delta = 10.322304964065552, @py_assert1 = None

    def _run_in_process(target, *args, **kwargs):
        if len(args) == 0:
            test_fn = target
        else:
            test_fn = args[0]
        # Do not need to wait much, 10s should be more than enough.
        timeout = 0.1 if test_fn is _intentional_deadlock else 10
        process = multiprocessing.Process(target=target, args=args, kwargs=kwargs)
        process.daemon = True
        try:
            t_start = time.time()
            process.start()
            if timeout >= 100:  # For debugging.
                print(
                    "\nprocess.pid STARTED", process.pid, (sys.argv, target, args, kwargs)
                )
                print(f"COPY-PASTE-THIS: gdb {sys.argv[0]} -p {process.pid}", flush=True)
            process.join(timeout=timeout)
            if timeout >= 100:
                print("\nprocess.pid JOINED", process.pid, flush=True)
            t_delta = time.time() - t_start
            if process.exitcode == 66 and m.defined_THREAD_SANITIZER:  # Issue #2754
                # WOULD-BE-NICE-TO-HAVE: Check that the message below is actually in the output.
                # Maybe this could work:
                # https://gist.github.com/alexeygrigorev/01ce847f2e721b513b42ea4a6c96905e
                pytest.skip(
                    "ThreadSanitizer: starting new threads after multi-threaded fork is not supported."
                )
            elif test_fn is _intentional_deadlock:
                assert process.exitcode is None
                return 0
            elif process.exitcode is None:
                assert t_delta > 0.9 * timeout
                msg = "DEADLOCK, most likely, exactly what this test is meant to detect."
                if env.PYPY and env.WIN:
                    pytest.skip(msg)
>               raise RuntimeError(msg)
E               RuntimeError: DEADLOCK, most likely, exactly what this test is meant to detect.

args       = (<function test_cross_module_gil_inner_pybind11_released at 0x00007f967d9694c0>,)
kwargs     = {'num_threads': 8, 'parallel': False}
msg        = 'DEADLOCK, most likely, exactly what this test is meant to detect.'
process    = <Process name='Process-45' pid=18326 parent=15388 stopped exitcode=-SIGTERM daemon>
t_delta    = 10.322304964065552
t_start    = 1675848907.445822
target     = <function _run_in_threads at 0x00007f967d969f60>
test_fn    = <function test_cross_module_gil_inner_pybind11_released at 0x00007f967d9694c0>
timeout    = 10

test_gil_scoped.py:186: RuntimeError
=========================== short test summary info ============================
SKIPPED [1] test_buffers.py:11: could not import 'numpy': No module named 'numpy'
SKIPPED [1] test_eigen_matrix.py:5: could not import 'numpy': No module named 'numpy'
SKIPPED [1] test_eigen_tensor.py:5: could not import 'numpy': No module named 'numpy'
SKIPPED [1] test_numpy_array.py:6: could not import 'numpy': No module named 'numpy'
SKIPPED [1] test_numpy_dtypes.py:8: could not import 'numpy': No module named 'numpy'
SKIPPED [1] test_numpy_vectorize.py:5: could not import 'numpy': No module named 'numpy'
SKIPPED [1] test_builtin_casters.py:145: no <string_view>
SKIPPED [1] test_builtin_casters.py:320: could not import 'numpy': No module named 'numpy'
SKIPPED [1] test_builtin_casters.py:486: could not import 'numpy': No module named 'numpy'
SKIPPED [1] test_callbacks.py:134: PyPy segfaults on here. See discussion on #1413.
SKIPPED [1] test_callbacks.py:203: Current PYBIND11_INTERNALS_VERSION too low
SKIPPED [1] test_class_sh_basic.py:123: unconditional skip
SKIPPED [1] test_copy_move.py:77: no <optional>
SKIPPED [1] test_custom_type_setup.py:35: condition: env.PYPY
SKIPPED [1] test_custom_type_setup.py:43: condition: env.PYPY
SKIPPED [1] test_exceptions.py:321: PyErr_NormalizeException Segmentation fault
SKIPPED [1] test_modules.py:104: Sufficiently exercised on platforms other than PyPy.
SKIPPED [1] test_pytypes.py:403: Not defined: PYBIND11_HANDLE_REF_DEBUG
SKIPPED [1] test_sequences_and_iterators.py:13: no <optional>
SKIPPED [1] test_stl.py:111: no <optional>
SKIPPED [1] test_stl.py:143: no <experimental/optional>
SKIPPED [1] test_stl.py:234: no <filesystem>
SKIPPED [1] test_stl.py:270: no std::monostate
SKIPPED [1] test_stl_binders.py:89: could not import 'numpy': No module named 'numpy'
SKIPPED [1] test_type_caster_odr_guard_1.py:30: type_caster_odr_guard_registry_values() is None
SKIPPED [1] test_type_caster_odr_guard_1.py:38: type_caster_odr_violation_detected_count() is None
SKIPPED [1] test_type_caster_odr_guard_2.py:9: UNEXPECTED: test_type_caster_odr_guard_2.cpp prevailed (to_python).
XFAIL test_call_policies.py::test_alive_gc
  _PyObject_GetDictPtr is unimplemented
XFAIL test_class.py::test_class_refcount
  condition: env.PYPY
XFAIL test_class.py::test_final
  condition: env.PYPY
XFAIL test_class.py::test_non_final_final
  condition: env.PYPY
XFAIL test_class_sh_disowning_mi.py::test_multiple_inheritance_python
  condition: env.PYPY
XFAIL test_class_sh_disowning_mi.py::test_disown_base1_first[MI2-3-4-1]
  condition: env.PYPY
XFAIL test_class_sh_disowning_mi.py::test_disown_base1_first[MI3-5-6-1]
  condition: env.PYPY
XFAIL test_class_sh_disowning_mi.py::test_disown_base1_first[MI4-7-8-1]
  condition: env.PYPY
XFAIL test_class_sh_disowning_mi.py::test_disown_base2_first[MI2-3-4-1]
  condition: env.PYPY
XFAIL test_class_sh_disowning_mi.py::test_disown_base2_first[MI3-5-6-1]
  condition: env.PYPY
XFAIL test_class_sh_disowning_mi.py::test_disown_base2_first[MI4-7-8-1]
  condition: env.PYPY
XFAIL test_class_sh_disowning_mi.py::test_disown_base2[MI7-13-4]
  condition: env.PYPY
XFAIL test_class_sh_disowning_mi.py::test_disown_base2[MI8b-15-3]
  condition: env.PYPY
XFAIL test_class_sh_property.py::test_valu_getter[m_valu_readonly]
  gc after `del field` is apparently deferred
XFAIL test_class_sh_property.py::test_valu_getter[m_valu_readwrite]
  gc after `del field` is apparently deferred
XFAIL test_eval.py::test_eval_file
  condition: env.PYPY
XFAIL test_exceptions.py::test_cross_module_exception_translator
  See Issue #2847, PR #2999, PR #4324
XFAIL test_exceptions.py::test_python_alreadyset_in_destructor
  Failure on PyPy 3.8 (7.3.7)
XFAIL test_methods_and_attributes.py::test_dynamic_attributes
  condition: env.PYPY
XFAIL test_methods_and_attributes.py::test_cyclic_gc
  condition: env.PYPY
XFAIL test_multiple_inheritance.py::test_multiple_inheritance_mix1
  condition: env.PYPY
XFAIL test_multiple_inheritance.py::test_multiple_inheritance_python
  condition: env.PYPY
XFAIL test_pickling.py::test_roundtrip_with_dict[PickleableWithDict]
  condition: env.PYPY
XFAIL test_pickling.py::test_roundtrip_with_dict[PickleableWithDictNew]
  condition: env.PYPY
XFAIL test_pytypes.py::test_memoryview_refcount[test_memoryview_object]
  getrefcount is not available
XFAIL test_pytypes.py::test_memoryview_refcount[test_memoryview_buffer_info]
  getrefcount is not available
XFAIL test_virtual_functions.py::test_move_support
  condition: env.PYPY
XPASS test_call_policies.py::test_keep_alive_argument sometimes comes out 1 off on PyPy
XPASS test_class_sh_disowning_mi.py::test_disown_base1_first[MI1-1-2-None] condition: env.PYPY
XPASS test_class_sh_disowning_mi.py::test_disown_base1_first[MI5-10-11-1] condition: env.PYPY
XPASS test_class_sh_disowning_mi.py::test_disown_base2_first[MI1-1-2-None] condition: env.PYPY
XPASS test_class_sh_disowning_mi.py::test_disown_base2_first[MI5-10-11-1] condition: env.PYPY
XPASS test_class_sh_disowning_mi.py::test_disown_base2[MI6-12-1] condition: env.PYPY
XPASS test_class_sh_disowning_mi.py::test_disown_base2[MI8-14-1] condition: env.PYPY
FAILED test_gil_scoped.py::test_run_in_process_multiple_threads_sequential[test_cross_module_gil_inner_pybind11_released]
= 1 failed, 717 passed, 27 skipped, 27 xfailed, 7 xpassed in 182.55s (0:03:02) =

[rwgk]

PR #4524

https://github.com/pybind/pybind11/actions/runs/4246322249/jobs/7382986308

🐍 3.10 • macos-latest • x64

============================= test session starts ==============================
platform darwin -- Python 3.10.10, pytest-7.0.0, pluggy-1.0.0
C++ Info: Apple LLVM 14.0.0 (clang-1400.0.29.202) C++11 __pybind11_internals_v4_clang_libcpp_cxxabi1002__ PYBIND11_SIMPLE_GIL_MANAGEMENT=True
rootdir: /Users/runner/work/pybind11/pybind11/tests, configfile: pytest.ini
plugins: timeout-2.1.0
timeout: 300.0s
timeout method: signal
timeout func_only: False
collected 1002 items

test_async.py ..                                                         [  0%]
test_buffers.py .........                                                [  1%]
test_builtin_casters.py .....s...............                            [  3%]
test_call_policies.py ........                                           [  3%]
test_callbacks.py .............s                                         [  5%]
test_chrono.py ...........................................               [  9%]
test_class.py ................................                           [ 12%]
test_class_sh_basic.py .................................s......          [ 16%]
test_class_sh_disowning.py ...                                           [ 17%]
test_class_sh_disowning_mi.py .........................                  [ 19%]
test_class_sh_factory_constructors.py ......                             [ 20%]
test_class_sh_inheritance.py .......                                     [ 20%]
test_class_sh_mi_thunks.py ..........                                    [ 21%]
test_class_sh_module_local.py ..                                         [ 22%]
test_class_sh_property.py .................                              [ 23%]
test_class_sh_shared_ptr_copy_move.py .....                              [ 24%]
test_class_sh_trampoline_basic.py .....                                  [ 24%]
test_class_sh_trampoline_self_life_support.py .....                      [ 25%]
test_class_sh_trampoline_shared_from_this.py ..............              [ 26%]
test_class_sh_trampoline_shared_ptr_cpp_arg.py ........                  [ 27%]
test_class_sh_trampoline_unique_ptr.py ..                                [ 27%]
test_class_sh_unique_ptr_member.py ...                                   [ 28%]
test_class_sh_virtual_py_cpp_mix.py ...............                      [ 29%]
test_class_sh_void_ptr_capsule.py ........                               [ 30%]
test_classh_mock.py .                                                    [ 30%]
test_const_name.py ......................                                [ 32%]
test_constants_and_functions.py .....                                    [ 33%]
test_copy_move.py ....s...                                               [ 33%]
test_custom_type_casters.py ...                                          [ 34%]
test_custom_type_setup.py ..                                             [ 34%]
test_docstring_options.py .                                              [ 34%]
test_eigen_matrix.py ..............................                      [ 37%]
test_eigen_tensor.py ................................................... [ 42%]
..........................................................               [ 48%]
test_enum.py .........                                                   [ 49%]
test_eval.py ....                                                        [ 49%]
test_exc_namespace_visibility.py .                                       [ 49%]
test_exceptions.py .........................                             [ 52%]
test_factory_constructors.py ...............                             [ 53%]
test_gil_scoped.py ..................................................... [ 59%]
..........F..........................                                    [ 62%]
test_iostream.py ......................                                  [ 64%]
test_kwargs_and_defaults.py ........                                     [ 65%]
test_local_bindings.py ..........                                        [ 66%]
test_methods_and_attributes.py ......................                    [ 68%]
test_modules.py .......                                                  [ 69%]
test_multiple_inheritance.py ..................                          [ 71%]
test_numpy_array.py .................................................... [ 76%]
.....                                                                    [ 77%]
test_numpy_dtypes.py ...............                                     [ 78%]
test_numpy_vectorize.py ........                                         [ 79%]
test_opaque_types.py ...                                                 [ 79%]
test_operator_overloading.py .....                                       [ 80%]
test_pickling.py ........                                                [ 81%]
test_pytypes.py .............................s.......................... [ 86%]
..............................                                           [ 89%]
test_return_value_policy_override.py .......................             [ 91%]
test_sequences_and_iterators.py .s............                           [ 93%]
test_smart_ptr.py .............                                          [ 94%]
test_stl.py ........ss..s.s.........                                     [ 97%]
test_stl_binders.py ..........                                           [ 98%]
test_tagbased_polymorphic.py .                                           [ 98%]
test_thread.py ..                                                        [ 98%]
test_type_caster_odr_guard_1.py ..ss                                     [ 98%]
test_type_caster_odr_guard_2.py s.                                       [ 98%]
test_union.py .                                                          [ 99%]
test_virtual_functions.py ..........                                     [100%]

=================================== FAILURES ===================================
_ test_run_in_process_multiple_threads_sequential[test_cross_module_gil_inner_pybind11_acquired] _

test_fn = <function test_cross_module_gil_inner_pybind11_acquired at 0x10e222320>

    @pytest.mark.parametrize("test_fn", ALL_BASIC_TESTS_PLUS_INTENTIONAL_DEADLOCK)
    def test_run_in_process_multiple_threads_sequential(test_fn):
        """Makes sure there is no GIL deadlock when running in a thread multiple times sequentially.
    
        It runs in a separate process to be able to stop and assert if it deadlocks.
        """
>       assert _run_in_process(_run_in_threads, test_fn, num_threads=8, parallel=False) == 0

test_fn    = <function test_cross_module_gil_inner_pybind11_acquired at 0x10e222320>

test_gil_scoped.py:234: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

target = <function _run_in_threads at 0x10e222950>
args = (<function test_cross_module_gil_inner_pybind11_acquired at 0x10e222320>,)
kwargs = {'num_threads': 8, 'parallel': False}
test_fn = <function test_cross_module_gil_inner_pybind11_acquired at 0x10e222320>
timeout = 10
process = <Process name='Process-46' pid=26332 parent=25287 stopped exitcode=-SIGTERM daemon>
t_start = 1677095556.202225, t_delta = 10.501205921173096, @py_assert1 = None

    def _run_in_process(target, *args, **kwargs):
        if len(args) == 0:
            test_fn = target
        else:
            test_fn = args[0]
        # Do not need to wait much, 10s should be more than enough.
        timeout = 0.1 if test_fn is _intentional_deadlock else 10
        process = multiprocessing.Process(target=target, args=args, kwargs=kwargs)
        process.daemon = True
        try:
            t_start = time.time()
            process.start()
            if timeout >= 100:  # For debugging.
                print(
                    "\nprocess.pid STARTED", process.pid, (sys.argv, target, args, kwargs)
                )
                print(f"COPY-PASTE-THIS: gdb {sys.argv[0]} -p {process.pid}", flush=True)
            process.join(timeout=timeout)
            if timeout >= 100:
                print("\nprocess.pid JOINED", process.pid, flush=True)
            t_delta = time.time() - t_start
            if process.exitcode == 66 and m.defined_THREAD_SANITIZER:  # Issue #2754
                # WOULD-BE-NICE-TO-HAVE: Check that the message below is actually in the output.
                # Maybe this could work:
                # https://gist.github.com/alexeygrigorev/01ce847f2e721b513b42ea4a6c96905e
                pytest.skip(
                    "ThreadSanitizer: starting new threads after multi-threaded fork is not supported."
                )
            elif test_fn is _intentional_deadlock:
                assert process.exitcode is None
                return 0
            elif process.exitcode is None:
                assert t_delta > 0.9 * timeout
                msg = "DEADLOCK, most likely, exactly what this test is meant to detect."
                if env.PYPY and env.WIN:
                    pytest.skip(msg)
>               raise RuntimeError(msg)
E               RuntimeError: DEADLOCK, most likely, exactly what this test is meant to detect.

args       = (<function test_cross_module_gil_inner_pybind11_acquired at 0x10e222320>,)
kwargs     = {'num_threads': 8, 'parallel': False}
msg        = 'DEADLOCK, most likely, exactly what this test is meant to detect.'
process    = <Process name='Process-46' pid=26332 parent=25287 stopped exitcode=-SIGTERM daemon>
t_delta    = 10.501205921173096
t_start    = 1677095556.202225
target     = <function _run_in_threads at 0x10e222950>
test_fn    = <function test_cross_module_gil_inner_pybind11_acquired at 0x10e222320>
timeout    = 10

test_gil_scoped.py:186: RuntimeError
=========================== short test summary info ============================
SKIPPED [1] test_builtin_casters.py:145: no <string_view>
SKIPPED [1] test_callbacks.py:213: Current PYBIND11_INTERNALS_VERSION too low
SKIPPED [1] test_class_sh_basic.py:123: unconditional skip
SKIPPED [1] test_copy_move.py:77: no <optional>
SKIPPED [1] test_pytypes.py:403: Not defined: PYBIND11_HANDLE_REF_DEBUG
SKIPPED [1] test_sequences_and_iterators.py:13: no <optional>
SKIPPED [1] test_stl.py:111: no <optional>
SKIPPED [1] test_stl.py:143: no <experimental/optional>
SKIPPED [1] test_stl.py:234: no <filesystem>
SKIPPED [1] test_stl.py:270: no std::monostate
SKIPPED [1] test_type_caster_odr_guard_1.py:30: type_caster_odr_guard_registry_values() is None
SKIPPED [1] test_type_caster_odr_guard_1.py:38: type_caster_odr_violation_detected_count() is None
SKIPPED [1] test_type_caster_odr_guard_2.py:9: UNEXPECTED: test_type_caster_odr_guard_2.cpp prevailed (to_python).
FAILED test_gil_scoped.py::test_run_in_process_multiple_threads_sequential[test_cross_module_gil_inner_pybind11_acquired]
============= 1 failed, 988 passed, 13 skipped in 63.10s (0:01:03) =============

[rwgk]

PR #4570

macos-latest • brew install llvm

============================= test session starts ==============================
platform darwin -- Python 3.11.2, pytest-7.0.0, pluggy-1.0.0
C++ Info: Homebrew Clang 15.0.7 C++17 pybind11_internals_v4_clang_libcpp_cxxabi1002 PYBIND11_SIMPLE_GIL_MANAGEMENT=False