test_signal: test_interprocess_signal() failed on GHA macOS

[vstinner]

The problem is that the first subprocess.Popen may not be removed immediately after with self.subprocess_send_signal(pid, "SIGHUP") as child: block, it can survive a little bit. But while it is being deleted automatically, oooops, sigusr1_handler() triggers and raises an SIGUSR1Exception exception which is logged as an "ignored exception", but it is ignored! The test fails.

GHA macOS:

======================================================================
FAIL: test_interprocess_signal (test.test_signal.PosixTests.test_interprocess_signal)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/runner/work/cpython/cpython/Lib/test/test_signal.py", line 108, in test_interprocess_signal
    assert_python_ok(script)
  File "/Users/runner/work/cpython/cpython/Lib/test/support/script_helper.py", line 166, in assert_python_ok
    return _assert_python(True, *args, **env_vars)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/runner/work/cpython/cpython/Lib/test/support/script_helper.py", line 151, in _assert_python
    res.fail(cmd_line)
  File "/Users/runner/work/cpython/cpython/Lib/test/support/script_helper.py", line 76, in fail
    raise AssertionError("Process return code is %d\n"
AssertionError: Process return code is 1
command line: ['/Users/runner/work/cpython/cpython/python.exe', '-X', 'faulthandler', '-I', '/Users/runner/work/cpython/cpython/Lib/test/signalinterproctester.py']

stdout:
---

---

stderr:
---
Exception ignored in: <function Popen.__del__ at 0x109080dd0>
Traceback (most recent call last):
  File "/Users/runner/work/cpython/cpython/Lib/subprocess.py", line 1120, in __del__
    def __del__(self, _maxsize=sys.maxsize, _warn=warnings.warn):

  File "/Users/runner/work/cpython/cpython/Lib/test/signalinterproctester.py", line 23, in sigusr1_handler
    raise SIGUSR1Exception
SIGUSR1Exception: 
F
======================================================================
FAIL: test_interprocess_signal (__main__.InterProcessSignalTests.test_interprocess_signal)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/runner/work/cpython/cpython/Lib/test/signalinterproctester.py", line 62, in test_interprocess_signal
    with self.assertRaises(SIGUSR1Exception):
AssertionError: SIGUSR1Exception not raised

----------------------------------------------------------------------
Ran 1 test in 0.252s

FAILED (failures=1)
---

----------------------------------------------------------------------
Ran 46 tests in 132.286s

FAILED (failures=1, skipped=10)

build: https://github.com/python/cpython/actions/runs/6340922718/job/17223429951?pr=110026

Linked PRs

• gh-110033: Fix signal test_interprocess_signal() #110035
• [3.12] gh-110033: Fix signal test_interprocess_signal() (GH-110035) #110040
• [3.11] gh-110033: Fix signal test_interprocess_signal() (GH-110035) #110041

[vstinner]

I modified the test locally to reproduce the issue on Linux:

    def test_interprocess_signal(self):
        # Install handlers. This function runs in a sub-process, so we
        # don't worry about re-setting the default handlers.
        signal.signal(signal.SIGHUP, self.sighup_handler)
        signal.signal(signal.SIGUSR1, self.sigusr1_handler)
        signal.signal(signal.SIGUSR2, signal.SIG_IGN)
        signal.signal(signal.SIGALRM, signal.default_int_handler)

        # Let the sub-processes know who to send signals to.
        pid = str(os.getpid())

        class RefCycle:
            pass

        print("SIGHUP")
        with self.subprocess_send_signal(pid, "SIGHUP") as child:
            self.wait_signal(child, 'SIGHUP')

            cycle = RefCycle()
            cycle.cycle = cycle
            cycle.child = child

        self.assertEqual(self.got_signals, {'SIGHUP': 1, 'SIGUSR1': 0,
                                            'SIGALRM': 0})
        print("SIGHUP--")

        print("SIGUSR1")
        mask = [signal.SIGUSR1]
        with self.assertRaises(SIGUSR1Exception):
            print("block")
            signal.pthread_sigmask(signal.SIG_BLOCK, mask)
            with self.subprocess_send_signal(pid, "SIGUSR1") as child:
                print("> unblock")
                time.sleep(0.05)
                cycle = None
                signal.pthread_sigmask(signal.SIG_UNBLOCK, mask)
                import gc; gc.collect()
                print(">> wait signal")
                self.wait_signal(child, 'SIGUSR1')
                print(">>> after wait signal?")
        self.assertEqual(self.got_signals, {'SIGHUP': 1, 'SIGUSR1': 1,
                                            'SIGALRM': 0})
        print("SIGUSR1--")

Output:

$ ./python Lib/test/signalinterproctester.py  -v
test_interprocess_signal (__main__.InterProcessSignalTests.test_interprocess_signal) ... SIGHUP
SIGHUP--
SIGUSR1
block
> unblock
~~~~~~ got SIGUSR1
Exception ignored in: <function Popen.__del__ at 0x7f64c886cc50>
Traceback (most recent call last):
  File "/home/vstinner/python/main/Lib/subprocess.py", line 1120, in __del__
    def __del__(self, _maxsize=sys.maxsize, _warn=warnings.warn):

  File "/home/vstinner/python/main/Lib/test/signalinterproctester.py", line 24, in sigusr1_handler
    raise SIGUSR1Exception
SIGUSR1Exception: 
>> wait signal
>>> after wait signal?
FAIL

======================================================================
FAIL: test_interprocess_signal (__main__.InterProcessSignalTests.test_interprocess_signal)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/vstinner/python/main/Lib/test/signalinterproctester.py", line 75, in test_interprocess_signal
    with self.assertRaises(SIGUSR1Exception):
AssertionError: SIGUSR1Exception not raised

----------------------------------------------------------------------
Ran 1 test in 0.135s

FAILED (failures=1)
__del__