bpo-25588: Fix regrtest when run inside IDLE #3962

vstinner · 2017-10-12T09:42:35Z

When regrtest in run inside IDLE, sys.stdout and sys.stderr are not
TextIOWrapper objects and have no file descriptor associated:
sys.stderr.fileno() raises io.UnsupportedOperation.

Disable faulthandler and don't replace sys.stdout in that case.

https://bugs.python.org/issue25588

When regrtest in run inside IDLE, sys.stdout and sys.stderr are not TextIOWrapper objects and have no file descriptor associated: sys.stderr.fileno() raises io.UnsupportedOperation. Disable faulthandler and don't replace sys.stdout in that case.

vstinner · 2017-10-12T09:43:28Z

I chose to not add a NEWS entry since running the Python test suite in IDLE causes many tests to fail. I don't think that we should promote running tests in IDLE.

serhiy-storchaka

LGTM even if it doesn't fix all tests.

serhiy-storchaka · 2017-10-12T10:32:06Z

Lib/test/libregrtest/setup.py

+    except io.UnsupportedOperation:
+        # On IDLE, sys.stdout has no file descriptor and is not a TextIOWrapper
+        # object. Leaving sys.stdout unchanged.
+        return


I'm wondering if we could use codecs.StreamRecoder here. But this looks like a more complex issue.

I expect that the IDLE shell handles correctly Unicode. replace_stdout() was written for terminals using ASCII/strict for stdout.

serhiy-storchaka · 2017-10-12T10:32:13Z

Lib/test/libregrtest/setup.py

-        faulthandler.register(signum, chain=True)
+    try:
+        stderr_fd = sys.stderr.fileno()
+    except io.UnsupportedOperation:


Maybe catch ValueError? This would cover the case of closed files too. And if support the case sys.stderr is None we should catch AttributeError.

ValueError: ok, I made the change. It avoids "import io", so I like it :-) Even if I'm not sure that we are supposed to support running tests with closed stderr...

AtttributeError: I'm not sure that it makes sense to run the test suite if there is no sys.stderr. Where are you supposed to write the output if there is not stdout nor stderr?

terryjreedy · 2017-10-13T01:20:20Z

Lib/test/libregrtest/setup.py

+        if hasattr(signal, 'SIGUSR1'):
+            signals.append(signal.SIGUSR1)
+        for signum in signals:
+            faulthandler.register(signum, chain=True, file=stderr_fd)


Do the faulthandler functions absolutely need an fd int rather than the existing output object?

faulthandler really needs a FD internally. The API accepts a file and calls .fileno() for you. Since we already know the FD, I prefer to directly pass the FD.

bedevere-bot · 2017-10-13T01:22:47Z

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

terryjreedy · 2017-10-13T01:33:42Z

Lib/test/libregrtest/setup.py

+        #
+        # Catch ValueError to catch io.UnsupportedOperation on TextIOBase
+        # and ValueError on a closed stream.
+        stderr_fd = None


Replace line with

try: stderr_fd = sys.__stderr__.fileno() except ValueError: stderr_fd = None

If IDLE is started from a command line or python (by import idlelib.idle), the backup call returns 2.

sys.stderr
<_io.TextIOWrapper name='' mode='w' encoding='utf-8'>
sys.stderr.fileno()
2

I meant to request changes

terryjreedy

see previous comment

bedevere-bot · 2017-10-13T01:38:45Z

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

vstinner · 2017-10-13T13:46:04Z

I have made the requested changes; please review again

bedevere-bot · 2017-10-13T13:46:07Z

Thanks for making the requested changes!

@serhiy-storchaka, @terryjreedy: please review the changes made to this pull request.

serhiy-storchaka · 2017-10-13T14:27:32Z

Lib/test/libregrtest/setup.py

+    try:
+        stderr_fd = sys.__stderr__.fileno()
+    except (ValueError, AttributeError):
+        # In IDLE, sys.stderr.fileno() raises io.UnsupportedOperation: the


If you use sys.__stderr__.fileno() this comment looks outdated.

Right, removed.

terryjreedy · 2017-10-13T17:29:56Z

Apveyor failure looks random and unrelated to this patch. Tests pass on my machine with a fresh build.

ERROR: setUpClass (test.test_nntplib.NetworkedNNTP_SSLTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\projects\cpython\lib\test\test_nntplib.py", line 295, in setUpClass
    usenetrc=False)
  File "C:\projects\cpython\lib\nntplib.py", line 1077, in __init__
    self.sock = _encrypt_on(self.sock, ssl_context, host)
  File "C:\projects\cpython\lib\nntplib.py", line 292, in _encrypt_on
    return context.wrap_socket(sock, server_hostname=hostname)
  File "C:\projects\cpython\lib\ssl.py", line 410, in wrap_socket
    _session=session
  File "C:\projects\cpython\lib\ssl.py", line 821, in __init__
    self.do_handshake()
  File "C:\projects\cpython\lib\ssl.py", line 1075, in do_handshake
    self._sslobj.do_handshake()
  File "C:\projects\cpython\lib\ssl.py", line 696, in do_handshake
    self._sslobj.do_handshake()
OSError: [Errno 0] Error

miss-islington · 2017-10-13T19:59:15Z

Thanks @Haypo for the PR 🌮🎉.. I'm working now to backport this PR to: 2.7, 3.6.
🐍🍒⛏🤖

bedevere-bot · 2017-10-13T19:59:26Z

GH-3987 is a backport of this pull request to the 3.6 branch.

When regrtest in run inside IDLE, sys.stdout and sys.stderr are not TextIOWrapper objects and have no file descriptor associated: sys.stderr.fileno() raises io.UnsupportedOperation. Disable faulthandler and don't replace sys.stdout in that case. (cherry picked from commit ccef823)

vstinner · 2017-10-13T20:00:14Z

Apveyor failure looks random and unrelated to this patch. Tests pass on my machine with a fresh build.

"ERROR: setUpClass (test.test_nntplib.NetworkedNNTP_SSLTests)"

That's an old but unfixed bug: https://bugs.python.org/issue30188

miss-islington · 2017-10-13T20:00:18Z

Sorry, @Haypo, I could not cleanly backport this to 2.7 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker ccef823939d4ef602f2d8d13d0bfec29eda597a5 2.7

vstinner · 2017-10-13T20:03:20Z

@serhiy-storchaka serhiy-storchaka added needs backport to 2.7 needs backport to 3.6 type-tests labels a day ago

faulthandler doesn't exist in Python 2.7, not replace_stdout(). I tested: "import test.autotest" works in Python 2.7.

When regrtest in run inside IDLE, sys.stdout and sys.stderr are not TextIOWrapper objects and have no file descriptor associated: sys.stderr.fileno() raises io.UnsupportedOperation. Disable faulthandler and don't replace sys.stdout in that case. (cherry picked from commit ccef823)

serhiy-storchaka · 2017-10-14T08:28:55Z

Thanks @Haypo!

the-knights-who-say-ni added the CLA signed label Oct 12, 2017

bedevere-bot added the awaiting merge label Oct 12, 2017

vstinner added the skip news label Oct 12, 2017

vstinner requested review from serhiy-storchaka and terryjreedy October 12, 2017 09:43

serhiy-storchaka approved these changes Oct 12, 2017

View reviewed changes

serhiy-storchaka added needs backport to 2.7 tests Tests in the Lib/test dir labels Oct 12, 2017

Catch ValueError instead of io.UnsupportedOperation

c1b6e13

terryjreedy requested changes Oct 13, 2017

View reviewed changes

bedevere-bot added awaiting changes and removed awaiting merge labels Oct 13, 2017

terryjreedy previously approved these changes Oct 13, 2017

View reviewed changes

bedevere-bot added awaiting merge and removed awaiting changes labels Oct 13, 2017

terryjreedy requested changes Oct 13, 2017

View reviewed changes

bedevere-bot added awaiting changes and removed awaiting merge labels Oct 13, 2017

Use sys.__stderr__ and catch AttributeError

4901d69

bedevere-bot added awaiting change review and removed awaiting changes labels Oct 13, 2017

serhiy-storchaka reviewed Oct 13, 2017

View reviewed changes

Remove outdated commment

72884c6

terryjreedy approved these changes Oct 13, 2017

View reviewed changes

bedevere-bot added awaiting merge and removed awaiting change review labels Oct 13, 2017

vstinner merged commit ccef823 into python:master Oct 13, 2017

bedevere-bot removed the awaiting merge label Oct 13, 2017

vstinner deleted the regrtest_idle branch October 13, 2017 19:59

bedevere-bot removed the needs backport to 3.6 label Oct 13, 2017

vstinner removed the needs backport to 2.7 label Oct 13, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

bpo-25588: Fix regrtest when run inside IDLE #3962

bpo-25588: Fix regrtest when run inside IDLE #3962

vstinner commented Oct 12, 2017 •

edited by bedevere-bot

Loading

vstinner commented Oct 12, 2017

serhiy-storchaka left a comment

serhiy-storchaka Oct 12, 2017

vstinner Oct 12, 2017

serhiy-storchaka Oct 12, 2017

vstinner Oct 12, 2017

terryjreedy Oct 13, 2017

vstinner Oct 13, 2017

bedevere-bot commented Oct 13, 2017

terryjreedy Oct 13, 2017

vstinner Oct 13, 2017

terryjreedy left a comment

bedevere-bot commented Oct 13, 2017

vstinner commented Oct 13, 2017

bedevere-bot commented Oct 13, 2017

serhiy-storchaka Oct 13, 2017

vstinner Oct 13, 2017

terryjreedy commented Oct 13, 2017

miss-islington commented Oct 13, 2017

bedevere-bot commented Oct 13, 2017

vstinner commented Oct 13, 2017

miss-islington commented Oct 13, 2017

vstinner commented Oct 13, 2017

serhiy-storchaka commented Oct 14, 2017

bpo-25588: Fix regrtest when run inside IDLE #3962

bpo-25588: Fix regrtest when run inside IDLE #3962

Conversation

vstinner commented Oct 12, 2017 • edited by bedevere-bot Loading

vstinner commented Oct 12, 2017

serhiy-storchaka left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bedevere-bot commented Oct 13, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

terryjreedy left a comment

Choose a reason for hiding this comment

bedevere-bot commented Oct 13, 2017

vstinner commented Oct 13, 2017

bedevere-bot commented Oct 13, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

terryjreedy commented Oct 13, 2017

miss-islington commented Oct 13, 2017

bedevere-bot commented Oct 13, 2017

vstinner commented Oct 13, 2017

miss-islington commented Oct 13, 2017

vstinner commented Oct 13, 2017

serhiy-storchaka commented Oct 14, 2017

vstinner commented Oct 12, 2017 •

edited by bedevere-bot

Loading