Repository navigation
Exception raised in traceback.StackSummary._should_show_carets exits interpreter: IndexError on tree.body[0] #122145
Description
Activity
- changed the title
[-]Exception raised in traceback.StackSummary._should_show_carets exits interpreter on Windows[/-][+]Exception raised in traceback.StackSummary._should_show_carets exits interpreter[/+]on Jul 23, 2024 In my opinion, we should figure out that issue first. This is probably something that should not happen - it's kind of equivalent to a comment only file generates an exception, that's just impossible.
In my opinion, we should figure out that issue first. This is probably something that should not happen - it's kind of equivalent to a comment only file generates an exception, that's just impossible.
Indeed, ISTM that due to the linecache bug we try to parse "lines" consisting of only a comment then access the tree's body first node. So fixing that issue will fix this specific failure.
Maybe it would still be interesting to make the code more robust against exceptions in
_should_show_carets?Looks like there are two bugs:
- With incorrect module detection, exception is not in
Lib/_pyrepl/__main__.py(this file is used for sources) - But,
tree.bodycan be empty in other cases as well, so why not guard it?
diff --git Lib/traceback.py Lib/traceback.py index 6ee1a50ca68..eba6f03a3c6 100644 --- Lib/traceback.py +++ Lib/traceback.py @@ -698,6 +698,8 @@ def _should_show_carets(self, start_offset, end_offset, all_lines, anchors): with suppress(SyntaxError, ImportError): import ast tree = ast.parse('\n'.join(all_lines)) + if not tree.body: + return False statement = tree.body[0] value = None def _spawns_full_line(value):
After:
» ./python.exe Python 3.14.0a0 (heads/main-dirty:3de092b82f5, Jul 20 2024, 09:23:52) [Clang 15.0.0 (clang-1500.3.9.4)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> exec(compile("tuple()[0]", "s", "exec")) Traceback (most recent call last): File "<python-input-0>", line 1, in <module> exec(compile("tuple()[0]", "s", "exec")) ~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "s", line 1, in <module> # Important: don't add things to this module, as they will end up in the REPL's IndexError: tuple index out of range- With incorrect module detection, exception is not in
I've fixed the second issue and added a test for that one. I can move to fixing the first issue as well.
2. But, `tree.body` can be empty in other cases as well, so why not guard it?Exactly. Another way to get an empty
tree.bodyis to run:exec(compile("tuple()[0]", "x.py", "exec"))
Where you have a x.py file that consists of:
# Just a comment.Yet another way to get an emptytree.body:exec(compile("print(2**100000)", "s", "exec"))
EDIT: never mind, I had reverted the fix.
Errr... ok. I'll amend the NEWS entry in the PR since it only talks about comments.
EDIT: Actually, I think you can only have an empty AST body if you are either an empty string or if you are a comment.
In
StackSummary.format_frame_summary,StackSummary._should_show_caretsis called without error suppression (like is done for_extract_caret_anchors_from_line_segment). This causes any errors in that method to exit the interpreter. I suggest we also ignore errors from_should_show_caretshere:Lines 581 to 583 in d27a53f
This time, an error in parsing a too deeply nested structure exits the interpreter:
>>> with open("lambda.txt", "w") as n: ... n.write("lambda: lambda: " * 1000 + "None") ... 16004 >>> exec(compile("str(2**100000)", "lambda.txt", "exec")) Traceback (most recent call last): Exception ignored in the internal traceback machinery: Traceback (most recent call last): File "/home/danzin/projects/cpython/Lib/traceback.py", line 139, in _print_exception_bltin return print_exception(exc, limit=BUILTIN_EXCEPTION_LIMIT, file=file, colorize=colorize) File "/home/danzin/projects/cpython/Lib/traceback.py", line 130, in print_exception te.print(file=file, chain=chain, colorize=colorize) File "/home/danzin/projects/cpython/Lib/traceback.py", line 1449, in print for line in self.format(chain=chain, colorize=colorize): File "/home/danzin/projects/cpython/Lib/traceback.py", line 1385, in format yield from _ctx.emit(exc.stack.format(colorize=colorize)) File "/home/danzin/projects/cpython/Lib/traceback.py", line 748, in format formatted_frame = self.format_frame_summary(frame_summary, colorize=colorize) File "/home/danzin/projects/cpython/Lib/traceback.py", line 583, in format_frame_summary show_carets = self._should_show_carets(start_offset, end_offset, all_lines, anchors) File "/home/danzin/projects/cpython/Lib/traceback.py", line 701, in _should_show_carets tree = ast.parse('\n'.join(all_lines)) File "/home/danzin/projects/cpython/Lib/ast.py", line 53, in parse return compile(source, filename, mode, flags, RecursionError: maximum recursion depth exceeded during compilation Traceback (most recent call last): File "/home/danzin/projects/cpython/Lib/code.py", line 91, in runcode exec(code, self.locals) File "<python-input-1>", line 1, in <module> File "lambda.txt", line 1, in <module> lambda: lambda: lambda: lambda: lambda: [...] lambda: None ValueError: Exceeds the limit (4300 digits) for integer string conversion; use sys.set_int_max_str_digits() to increase the limit During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/danzin/projects/cpython/Lib/runpy.py", line 198, in _run_module_as_main return _run_code(code, main_globals, None, File "/home/danzin/projects/cpython/Lib/runpy.py", line 88, in _run_code exec(code, run_globals) File "/home/danzin/projects/cpython/Lib/_pyrepl/__main__.py", line 6, in <module> __pyrepl_interactive_console() File "/home/danzin/projects/cpython/Lib/_pyrepl/main.py", line 59, in interactive_console run_multiline_interactive_console(console) File "/home/danzin/projects/cpython/Lib/_pyrepl/simple_interact.py", line 156, in run_multiline_interactive_console more = console.push(_strip_final_indent(statement), filename=input_name, _symbol="single") # type: ignore[call-arg] File "/home/danzin/projects/cpython/Lib/code.py", line 303, in push more = self.runsource(source, filename, symbol=_symbol) File "/home/danzin/projects/cpython/Lib/_pyrepl/console.py", line 200, in runsource self.runcode(code) File "/home/danzin/projects/cpython/Lib/code.py", line 95, in runcode self.showtraceback() File "/home/danzin/projects/cpython/Lib/_pyrepl/console.py", line 168, in showtraceback super().showtraceback(colorize=self.can_colorize) File "/home/danzin/projects/cpython/Lib/code.py", line 147, in showtraceback lines = traceback.format_exception(ei[0], ei[1], last_tb.tb_next, colorize=colorize) File "/home/danzin/projects/cpython/Lib/traceback.py", line 155, in format_exception return list(te.format(chain=chain, colorize=colorize)) File "/home/danzin/projects/cpython/Lib/traceback.py", line 1385, in format yield from _ctx.emit(exc.stack.format(colorize=colorize)) File "/home/danzin/projects/cpython/Lib/traceback.py", line 748, in format formatted_frame = self.format_frame_summary(frame_summary, colorize=colorize) File "/home/danzin/projects/cpython/Lib/traceback.py", line 583, in format_frame_summary show_carets = self._should_show_carets(start_offset, end_offset, all_lines, anchors) File "/home/danzin/projects/cpython/Lib/traceback.py", line 701, in _should_show_carets tree = ast.parse('\n'.join(all_lines)) File "/home/danzin/projects/cpython/Lib/ast.py", line 53, in parse return compile(source, filename, mode, flags, RecursionError: maximum recursion depth exceeded during compilation
If an even deeper nested structure is used,
MemoryErroris raised, but somehow doesn't exit the interpreter.Doesn't seem to affect 3.13. Should a new issue be created for this one?
IMO #122071 is a duplicate of this issue.
I think #122071 affects all the Python versions:
# Hi, I'm in the error message import traceback try: exec(compile("tuple()[0]", "s", "exec")) except: traceback.print_exc()
$ python3.12 repro.py Traceback (most recent call last): File "/home/churchyard/tmp/repro.py", line 4, in <module> exec(compile("tuple()[0]", "s", "exec")) File "s", line 1, in <module> # Hi, I'm in the error message ^^^^^^^^^^ IndexError: tuple index out of rangeWhile this one only affects Python 3.13, e.g.:
>>> code = "#boom\nraise RuntimeError('see the line above')" >>> eval(compile(code, "str_eval", "exec")) Traceback (most recent call last): Traceback (most recent call last): File "<python-input-1>", line 1, in <module> File "str_eval", line 2, in <module> RuntimeError: see the line above During handling of the above exception, another exception occurred: Traceback (most recent call last): File "<frozen runpy>", line 198, in _run_module_as_main File "<frozen runpy>", line 88, in _run_code File "/usr/lib64/python3.13/_pyrepl/__main__.py", line 6, in <module> __pyrepl_interactive_console() File "/usr/lib64/python3.13/_pyrepl/main.py", line 59, in interactive_console run_multiline_interactive_console(console) File "/usr/lib64/python3.13/_pyrepl/simple_interact.py", line 157, in run_multiline_interactive_console more = console.push(_strip_final_indent(statement), filename=input_name, _symbol="single") # type: ignore[call-arg] File "/usr/lib64/python3.13/code.py", line 313, in push more = self.runsource(source, filename, symbol=_symbol) File "/usr/lib64/python3.13/_pyrepl/console.py", line 205, in runsource self.runcode(code) File "/usr/lib64/python3.13/code.py", line 96, in runcode self.showtraceback() File "/usr/lib64/python3.13/code.py", line 129, in showtraceback self._showtraceback(typ, value, tb.tb_next, '') File "/usr/lib64/python3.13/code.py", line 144, in _showtraceback self._excepthook(typ, value, tb) File "/usr/lib64/python3.13/_pyrepl/console.py", line 169, in _excepthook lines = traceback.format_exception( File "/usr/lib64/python3.13/traceback.py", line 155, in format_exception return list(te.format(chain=chain, colorize=colorize)) File "/usr/lib64/python3.13/traceback.py", line 1388, in format yield from _ctx.emit(exc.stack.format(colorize=colorize)) File "/usr/lib64/python3.13/traceback.py", line 747, in format formatted_frame = self.format_frame_summary(frame_summary, colorize=colorize) File "/usr/lib64/python3.13/traceback.py", line 583, in format_frame_summary show_carets = self._should_show_carets(start_offset, end_offset, all_lines, anchors) File "/usr/lib64/python3.13/traceback.py", line 701, in _should_show_carets statement = tree.body[0] IndexError: list index out of range
I think they build on top of each other -- when #122071 is fixed, this one will likely also get fixed, but I am not quite sure.
- changed the title
[-]Exception raised in traceback.StackSummary._should_show_carets exits interpreter[/-][+]Exception raised in traceback.StackSummary._should_show_carets exits interpreter: IndexError on tree.body[0][/+]on Sep 17, 2024 This bug is impacting Fedora Installer (Anaconda) and was marked as a Fedora 41 blocker issue: https://bugzilla.redhat.com/show_bug.cgi?id=2311907 So I also mark this bug as a blocker issue.
@Yhg1s explained me that if we add a bugfix after 3.13rc2, another rc3 is needed. I don't think that this bug should require a whole new rc3. So I removed the release-blocker label. The fix can wait for 3.13.1 even if it's bad.
For Fedora, we can have a downstream patch to unblock Fedora 41, until the fix is merged into the 3.13 branch.
- moved this from Todo to Done in Release and Deferred blockers 🚫
on Sep 18, 2024 - added a commit that references this issue
on Sep 30, 2024
Bug report
Bug description:
Running the code sample from #122071 in 3.13.0b4 or main exits the interpreter due to
traceback.StackSummary._should_show_caretsraising an exception:We can either protect the
_should_show_caretscall withsuppress(Exception)here:cpython/Lib/traceback.py
Lines 581 to 583 in 2762c6c
Or guard against
tree.bodybeing empty here:cpython/Lib/traceback.py
Lines 700 to 701 in 2762c6c
(or both?)
Should I submit a PR with one of the fixes above? Any preferences?
CPython versions tested on:
3.13, CPython main branch
Operating systems tested on:
Windows, Linux
Linked PRs