Problem with faulthandler when used with Twisted Logger and "pytest --capture=no"

[amotl]

Dear pytest developers,

thanks a stack for conceiving and maintaining this excellent package. I never expected to file an issue or submit a patch here, but here we go.

Introduction

On behalf of daq-tools/kotori#38, we are currently in the progress of finally upgrading Kotori to Python 3. Kotori is based on Twisted and uses pytest for testing. Within kotori.util.logger, we tried hard to apply some magic to make everything work together on all occasions with respect to appropriately configuring twisted.logger to our needs.

Environment

We are on macOS 10.13.6.

$ pip list
pytest                        6.2.1
pytest-twisted                1.13.2
Twisted                       20.3.0

Details

The background on this is that the Twisted Logger intercepts the logging by providing a file-like wrapper around the STDERR stream, which is obviously not an actual stream. In this case, when running with pytest capture=no, pytest's faulthandler fails when trying to restore the stderr stream through sys.stderr.fileno() at [1] as this will actually return -1, in turn signaling an invalid file descriptor [2].

It will a) raise the exception outlined below and b) won't stop the process on teardown.

[1] https://github.com/pytest-dev/pytest/blob/6.2.1/src/_pytest/faulthandler.py#L69-L77
[2] https://github.com/twisted/twisted/blob/twisted-20.3.0/src/twisted/logger/_io.py#L132-L139

Traceback

Traceback (most recent call last):
  File "/path/to/.venv/bin/pytest", line 8, in <module>
    sys.exit(console_main())
  File "/path/to/.venv/lib/python3.8/site-packages/_pytest/config/__init__.py", line 185, in console_main
    code = main()
  File "/path/to/.venv/lib/python3.8/site-packages/_pytest/config/__init__.py", line 162, in main
    ret: Union[ExitCode, int] = config.hook.pytest_cmdline_main(
  File "/path/to/.venv/lib/python3.8/site-packages/pluggy/hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/path/to/.venv/lib/python3.8/site-packages/pluggy/manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/path/to/.venv/lib/python3.8/site-packages/pluggy/manager.py", line 84, in <lambda>
    self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
  File "/path/to/.venv/lib/python3.8/site-packages/pluggy/callers.py", line 208, in _multicall
    return outcome.get_result()
  File "/path/to/.venv/lib/python3.8/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/path/to/.venv/lib/python3.8/site-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/path/to/.venv/lib/python3.8/site-packages/_pytest/main.py", line 316, in pytest_cmdline_main
    return wrap_session(config, _main)
  File "/path/to/.venv/lib/python3.8/site-packages/_pytest/main.py", line 311, in wrap_session
    config._ensure_unconfigure()
  File "/path/to/.venv/lib/python3.8/site-packages/_pytest/config/__init__.py", line 987, in _ensure_unconfigure
    self.hook.pytest_unconfigure(config=self)
  File "/path/to/.venv/lib/python3.8/site-packages/pluggy/hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/path/to/.venv/lib/python3.8/site-packages/pluggy/manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/path/to/.venv/lib/python3.8/site-packages/pluggy/manager.py", line 84, in <lambda>
    self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
  File "/path/to/.venv/lib/python3.8/site-packages/pluggy/callers.py", line 208, in _multicall
    return outcome.get_result()
  File "/path/to/.venv/lib/python3.8/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/path/to/.venv/lib/python3.8/site-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/path/to/.venv/lib/python3.8/site-packages/_pytest/faulthandler.py", line 69, in pytest_unconfigure
    faulthandler.enable(file=self._get_stderr_fileno())
ValueError: file is not a valid file descripter

Patch

This problem is mitigated by #8250.

With kind regards,
Andreas.