Support Python 3.11

[The-Compiler]

Python 3.11 is planned for October 3rd, and the first beta will be released soon (May 6th). Thus, I made a first attempt at running the tests on 3.11.0a7:

• de0aa69 ci: Update Python versions
• 9fbaba2 Ignore certifi deprecation warning
• fc57a80 Update YAML C extension check
• 323b64c ignore hypothesis warnings
• 6fc30a1 ci: Mark Python 3.11 as experimental

Unfortunately, I couldn't really get things to work fully. Some of the tests in tests/end2end/features/test_hints_bdd.py and tests/end2end/features/test_tabs_bdd.py seem to be flaky in strange ways. We should probably have a closer look, but for now, I should probably focus on Qt 6 first.

[The-Compiler]

Gave this another try with the 3.11 branch (i.e. after Python 3.11.0b1) on qt6-v2. It looks better, though I needed some adjustments:

• 9124c22 mkvenv: Remove sip packages from binary args
• 543da7a mypy requirements: Skip lxml on Python 3.11
• 4960c18 Patch PyQt6 QUrl flag behavior for Python 3.11
• 64e8c71 tests: Ignore Flask/Werkzeug ResourceWarnings

And reported/reproduced some upstream bugs:

• Python/pystate.c:2218: _PyThreadState_PopFrame: Assertion `tstate->datastack_top >= base' failed. python/cpython#93252
• Objects/codeobject.c:814: retreat: Assertion `bounds->ar_start > 0' failed python/cpython#93249
• Default enum.FlagBoundary should be KEEP? python/cpython#93250
• Crash on AST with misordered linenos python/cpython#92597
• ResourceWarning for unclosed sockets with development server pallets/werkzeug#2421

CPython patches:

From ea116aaa1c9904149befdd8a45a985d550cad2aa Mon Sep 17 00:00:00 2001
From: Florian Bruhin <me@the-compiler.org>
Date: Thu, 26 May 2022 14:40:23 +0200
Subject: [PATCH] Disable broken asserts

---
 Python/compile.c | 2 +-
 Python/pystate.c | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/Python/compile.c b/Python/compile.c
index cc0d76e038..42507532a5 100644
--- a/Python/compile.c
+++ b/Python/compile.c
@@ -7531,7 +7531,7 @@ write_location_info_long_form(struct assembler* a, struct instr* i, int length)
     assert(length > 0 &&  length <= 8);
     write_location_first_byte(a, PY_CODE_LOCATION_INFO_LONG, length);
     write_location_signed_varint(a, i->i_lineno - a->a_lineno);
-    assert(i->i_end_lineno >= i->i_lineno);
+    //assert(i->i_end_lineno >= i->i_lineno);
     write_location_varint(a, i->i_end_lineno - i->i_lineno);
     write_location_varint(a, i->i_col_offset+1);
     write_location_varint(a, i->i_end_col_offset+1);
diff --git a/Python/pystate.c b/Python/pystate.c
index 3e28a6ab69..c801cdcd8b 100644
--- a/Python/pystate.c
+++ b/Python/pystate.c
@@ -2215,7 +2215,7 @@ _PyThreadState_PopFrame(PyThreadState *tstate, _PyInterpreterFrame * frame)
     }
     else {
         assert(tstate->datastack_top);
-        assert(tstate->datastack_top >= base);
+        //assert(tstate->datastack_top >= base);
         tstate->datastack_top = base;
     }
 }
-- 
2.36.1

This all results in:

FAILED tests/end2end/features/test_downloads_bdd.py::test_answering_a_question_for_a_cancelled_download_415 - end2end.fixtures.testprocess.WaitForTimeout: Timed out after 5000ms waiting for {'category': 'commands', 'module': 'command',...
FAILED tests/end2end/features/test_downloads_bdd.py::test_answering_a_question_for_a_cancelled_download_415 - AssertionError: assert False
FAILED tests/end2end/features/test_tabs_bdd.py::test_undo_with_window_and_count - end2end.fixtures.testprocess.WaitForTimeout: Timed out while waiting for about:blank to be loaded
FAILED tests/unit/utils/test_debug.py::TestQEnumKey::test_qenum_key[Qt-1024-None-0x0400] - TypeError: Can't guess enum class of an int!
FAILED tests/unit/utils/test_debug.py::TestQFlagsKey::test_qflags_key[Qt-4352-AlignmentFlag-AlignBaseline|0x1000] - AttributeError: 'int' object has no attribute 'name'
ERROR tests/end2end/features/test_downloads_bdd.py::test_downloading_with_infinite_redirect - end2end.fixtures.testprocess.InvalidLine: 
ERROR tests/end2end/features/test_downloads_bdd.py::test_quitting_with_finished_downloads_and_confirm_quitdownloads_issue_846 - end2end.fixtures.testprocess.InvalidLine: 
ERROR tests/end2end/features/test_downloads_bdd.py::test_retrying_a_failed_download - end2end.fixtures.testprocess.InvalidLine: 
ERROR tests/end2end/features/test_hints_bdd.py::test_using_rapid_number_hinting_twice - end2end.fixtures.testprocess.InvalidLine: 
ERROR tests/end2end/features/test_hints_bdd.py::test_using_hintsauto_follow__uniquematch_in_letter_mode - end2end.fixtures.testprocess.WaitForTimeout: Timed out after 5000ms waiting for {'category': 'ipc', 'module': 'ipc', 'function': ...
ERROR tests/end2end/features/test_tabs_bdd.py::test_undo_with_window_and_count - end2end.fixtures.testprocess.ProcessExited

when running:

.venv-py311/bin/pytest  -- tests/end2end/features/test_downloads_bdd.py tests/end2end/features/test_hints_bdd.py tests/end2end/features/test_tabs_bdd.py tests/unit/utils/test_debug.py     

It seems like mostly timeout issues (perhaps due to the thread issue above?) and this:

E           Exception ignored in: <function IOBase.__del__ at 0x7f7f228c4f70>
E           Traceback (most recent call last):
E             File "/home/florian/proj/cpython/Lib/_pyio.py", line 436, in __del__
E               self.close()
E               ^^^^^^^^^^^^
E             File "/home/florian/proj/cpython/Lib/_pyio.py", line 1338, in close
E               self.flush()
E               ^^^^^^^^^^^^
E             File "/home/florian/proj/cpython/Lib/_pyio.py", line 1299, in flush
E               self._flush_unlocked()
E               ^^^^^^^^^^^^^^^^^^^^^^
E             File "/home/florian/proj/qutebrowser/git/.venv-py311/lib/python3.11/site-packages/cheroot/makefile.py", line 45, in _flush_unlocked
E               n = self.raw.write(bytes(self._write_buf))
E                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
E             File "/home/florian/proj/cpython/Lib/socket.py", line 723, in write
E               return self._sock.send(b)
E                      ^^^^^^^^^^^^^^^^^^
E           OSError: [Errno 9] Bad file descriptor
E           Exception ignored in: <function IOBase.__del__ at 0x7f7f228c4f70>
E           Traceback (most recent call last):
E             File "/home/florian/proj/cpython/Lib/_pyio.py", line 436, in __del__
E               self.close()
E               ^^^^^^^^^^^^
E             File "/home/florian/proj/cpython/Lib/_pyio.py", line 1338, in close
E               self.flush()
E               ^^^^^^^^^^^^
E             File "/home/florian/proj/cpython/Lib/_pyio.py", line 1299, in flush
E               self._flush_unlocked()
E               ^^^^^^^^^^^^^^^^^^^^^^
E             File "/home/florian/proj/qutebrowser/git/.venv-py311/lib/python3.11/site-packages/cheroot/makefile.py", line 45, in _flush_unlocked
E               n = self.raw.write(bytes(self._write_buf))
E                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
E             File "/home/florian/proj/cpython/Lib/socket.py", line 723, in write
E               return self._sock.send(b)
E                      ^^^^^^^^^^^^^^^^^^
E           OSError: [Errno 9] Bad file descriptor

Should probably retry with a non-debug Python and the master branch too.

[The-Compiler]

Tried with a release build of Python 3.11b2 with python/cpython#93397 on top. The only remaining blocker with that is python/cpython#93252 I think.

[The-Compiler]

With python/cpython#94708 and python/cpython#93895 I believe we can finally get back to this once 3.11.0b4 is released. Not sure what the exact plan is for that, see Mailman 3 [Release] Status of Python 3.11 release - Python-Dev - python.org.

It looks like there are some new failing tests though, due to some kind of coverage/tracing changes:

>       assert covtest.check() == [expected]
E       AssertionError: assert [Message(typ=...h coverage!')] == [Message(typ=...h coverage!')]
E         At index 0 diff: Message(typ=<MsgType.insufficient_coverage: 1>, filename='module.py', text='module.py has 71.43% line and 50.00% branch coverage!') != Message(typ=<MsgType.insufficient_coverage: 1>, filename='module.py', text='module.py has 83.33% line and 50.00% branch coverage!')

and others:

FAILED tests/unit/scripts/test_check_coverage.py::test_tested_no_branches - AssertionError: assert [Message(typ=...h coverage!')] == []
FAILED tests/unit/scripts/test_check_coverage.py::test_tested_with_branches - AssertionError: assert [Message(typ=...h coverage!')] == []
FAILED tests/unit/scripts/test_check_coverage.py::test_untested - AssertionError: assert [Message(typ=...h coverage!')] == [Message(typ=...h coverage!')]
FAILED tests/unit/scripts/test_check_coverage.py::test_untested_floats - AssertionError: assert [Message(typ=...h coverage!')] == [Message(typ=...h coverage!')]
FAILED tests/unit/scripts/test_check_coverage.py::test_untested_branches - AssertionError: assert [Message(typ=...h coverage!')] == [Message(typ=...h coverage!')]
FAILED tests/unit/scripts/test_check_coverage.py::test_tested_unlisted - AssertionError: assert [] == [Message(typ=...fect_files!')]

Also, the "Bad file descriptor" issue in cheroot still seems to be around:

E           end2end.fixtures.testprocess.InvalidLine: 
E           Exception ignored in: <function IOBase.__del__ at 0x7ff4c226cec0>
E           Traceback (most recent call last):
E             File "/home/florian/proj/cpython/Lib/_pyio.py", line 436, in __del__
E               self.close()
E               ^^^^^^^^^^^^
E             File "/home/florian/proj/cpython/Lib/_pyio.py", line 1338, in close
E               self.flush()
E               ^^^^^^^^^^^^
E             File "/home/florian/proj/cpython/Lib/_pyio.py", line 1299, in flush
E               self._flush_unlocked()
E               ^^^^^^^^^^^^^^^^^^^^^^
E             File "/home/florian/proj/qutebrowser/git/.venv-py311/lib/python3.11/site-packages/cheroot/makefile.py", line 45, in _flush_unlocked
E               n = self.raw.write(bytes(self._write_buf))
E                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
E             File "/home/florian/proj/cpython/Lib/socket.py", line 723, in write
E               return self._sock.send(b)
E                      ^^^^^^^^^^^^^^^^^^
E           OSError: [Errno 9] Bad file descriptor

in various somewhat random tests:

ERROR tests/end2end/features/test_downloads_bdd.py::test_not_remembering_the_last_download_directory - end2end.fixtures.testprocess.InvalidLine: 
ERROR tests/end2end/features/test_hints_bdd.py::test_yanking_javascript_link_to_clipboard - end2end.fixtures.testprocess.InvalidLine: 
ERROR tests/end2end/features/test_private_bdd.py::test_adblocking_after_reiniting_private_profile - end2end.fixtures.testprocess.InvalidLine: 

[The-Compiler]

3.11.0b4 is out now. The coverage issue is nedbat/coveragepy#1419 which has been fixed in nedbat/coveragepy@0d64498 - and indeed our tests work fine with that.

That means only the OSError: [Errno 9] Bad file descriptor is remaining now. Only happens with --with-pydebug builds, though. Maybe it's not even specific to Python 3.11 either.

edit: Indeed, also happens with a Python 3.10 built with --with-pydebug.

Probably time to re-enable Python 3.11 on CI and close this! 🎉 (at least when there has been a coverage.py release)

[The-Compiler]

Oh, looks like GitHub Actions is not quite ready yet, probably waiting for actions/python-versions#177.

[The-Compiler]

Works fine now: https://github.com/qutebrowser/qutebrowser/runs/7319313602?check_suite_focus=true ✨