Implement PEP 654: Exception Groups

[iritkatriel]

BPO	45292
Nosy	@gvanrossum, @markshannon, @tirkarthi, @erlend-aasland, @iritkatriel
PRs	bpo-45292: [PEP 654] add the ExceptionGroup and BaseExceptionGroup classes #28569 bpo-45292: [PEP 654] Update traceback display code to work with exception groups #29207 bpo-45292: Use raw strings for regex in tests #29545 bpo-45292: [PEP-654] add except* #29581 bpo-45292: [PEP-654] exception groups and except* documentation #30158

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/iritkatriel'
closed_at = <Date 2022-01-06.19:06:15.397>
created_at = <Date 2021-09-26.12:07:11.663>
labels = ['interpreter-core', '3.11', 'type-feature', 'library', 'docs']
title = 'Implement PEP 654: Exception Groups'
updated_at = <Date 2022-01-06.19:06:15.396>
user = 'https://github.com/iritkatriel'

bugs.python.org fields:

activity = <Date 2022-01-06.19:06:15.396>
actor = 'iritkatriel'
assignee = 'iritkatriel'
closed = True
closed_date = <Date 2022-01-06.19:06:15.397>
closer = 'iritkatriel'
components = ['Documentation', 'Interpreter Core', 'Library (Lib)']
creation = <Date 2021-09-26.12:07:11.663>
creator = 'iritkatriel'
dependencies = []
files = []
hgrepos = []
issue_num = 45292
keywords = ['patch']
message_count = 12.0
messages = ['402661', '404837', '405773', '406266', '406400', '408549', '408557', '408559', '408560', '408589', '408590', '409898']
nosy_count = 5.0
nosy_names = ['gvanrossum', 'Mark.Shannon', 'xtreak', 'erlendaasland', 'iritkatriel']
pr_nums = ['28569', '29207', '29545', '29581', '30158']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue45292'
versions = ['Python 3.11']

Linked PRs

• GH-89455: Add missing attributes (added in 3.11) to traceback module docs #105044
• [3.12] GH-89455: Add missing attributes (added in 3.11) to traceback module docs (GH-105044) #105066
• [3.11] GH-89455: Add missing attributes (added in 3.11) to traceback module docs (GH-105044) #105067

[iritkatriel]

We will implement Exception Groups and except* in a series of PRs:

1. Add the ExceptionGroup and BaseExceptionGroup classes
2. Update traceback rendering code (python and C) for exception groups.
3. Implement except*
4. Write documentation sections.

[iritkatriel]

New changeset f30ad65 by Irit Katriel in branch 'main':
bpo-45292: [PEP-654] add the ExceptionGroup and BaseExceptionGroup classes (GH-28569)
f30ad65

[iritkatriel]

New changeset 3509b26 by Irit Katriel in branch 'main':
bpo-45292: [PEP-654] Update traceback display code to work with exception groups (GH-29207)
3509b26

[tirkarthi]

The tests emit some deprecation warnings :

PYTHONWARNINGS=always ./python -Wall -m test test_exception_group
0:00:00 load avg: 0.39 Run tests sequentially
0:00:00 load avg: 0.39 [1/1] test_exception_group
/home/karthikeyan/stuff/python/cpython/Lib/test/test_exception_group.py:41: DeprecationWarning: invalid escape sequence '\('
  MSG = 'second argument \(exceptions\) must be a sequence'
/home/karthikeyan/stuff/python/cpython/Lib/test/test_exception_group.py:47: DeprecationWarning: invalid escape sequence '\('
  MSG = 'second argument \(exceptions\) must be a non-empty sequence'
/home/karthikeyan/stuff/python/cpython/Lib/test/test_exception_group.py:52: DeprecationWarning: invalid escape sequence '\('
  MSG = ('Item [0-9]+ of second argument \(exceptions\)'

== Tests result: SUCCESS ==

1 test OK.

Total duration: 51 ms
Tests result: SUCCESS

[tirkarthi]

New changeset 8b06d01 by Irit Katriel in branch 'main':
bpo-45292: Use raw strings for regex in tests (GH-29545)
8b06d01

[iritkatriel]

New changeset d60457a by Irit Katriel in branch 'main':
bpo-45292: [PEP-654] add except* (GH-29581)
d60457a

[gvanrossum]

We should have a discussion here about improvements that Mark Shannon would like to see, in particular to do more work in the compiler instead of the interpreter.

[markshannon]

PR 29581 resulted in a 1% slowdown, which is not terrible, but code not using except* should not be slowed down at all.

IMO, the way to avoid the slowdown is to implement except* using the existing instruction set (perhaps with a few minor additions)

We already implement try-finally, named except blocks and with statements without any complex bytecodes (except perhaps WITH_EXCEPT_START).

These used to involve a lot of state and more complex bytecodes. So it is possible to make these simplifications,
but it does take work.

There are a number of techniques we can use:

If any state is needed, push it to the stack as we do with ctx.__exit__ in the with statement, and when pushing f_lasti in exception handlers.
Duplicate code paths when the semantics differ in different cases, as we do for finally blocks.
If anything is too complex to handle on the stack, put it in a temporary variable.
Be liberal in your use of virtual try-excepts (SETUP_FINALLY, POP_FINALLY pairs), as zero-cost exception handling should keep the cost down.

It may be too late for this advice, but if I were writing the except* implementation from scratch, I would:

1. Sketch out the pseudo Python that a try-except* would map to. This is a good opportunity to discover any design bugs that might result in undesirable behavior for corner cases.

2. Implement the translation in the compiler, not worrying about any redundancy or inefficiency, just correctness.

3. Look to improve the above, either in the compiler front-end, or by replacing inefficient code patterns in the back-end. Handling the optimization in the backend has the advantage that other code might benefit as well.