[REQUEST] Support for PEP 654 tracebacks (ExceptionGroups)

[Tinche]

Hello,

let me provide some context first. PEP 654 introduces a new type of exception - an ExceptionGroup. (Also a BaseExceptionGroup, which is like the BaseException version of it). An ExceptionGroup is essentially a container for a list of inner exceptions. (It also introduces other things, like the except* clause, but I don't think that's relevant to Rich tracebacks.)

The hope is this ExceptionGroup will become a standard way for libraries to raise groups of exceptions. For example, this is a common use case for the Hypothesis library. There will be a backport package on PyPI to provide ExceptionGroups to older Python versions, so third party libraries will be able to use them without issue.

There is another PEP, PEP 678, that builds on ExceptionGroups to reify the concept of an exception note. This PEP has not yet been accepted, and is currently being discussed. (Due to a fluke, the CPython 3.11a4 release contains an implementation of it.) The idea of this PEP is for libraries to be able to attach a string to any exception under the __note__ attribute, and for traceback machinery to simply print it out if it is present. The notes can and will be multiline in practice, I think.

So ExceptionGroups are an official thing, notes will maybe become a thing.

I'm working on a couple of libraries that can benefit from ExceptionGroups so I can give an example. Imagine you're structuring a JSON payload into a class, and there are errors. Setup code:

from attrs import define
from cattrs import structure

@define
class Test:
    a: int
    b: int
    c: float


@define
class Outer:
    inner: Test
    c: int

structure({"inner": {"a": 'not_an_int', "b": 1}}, Outer)

An ExceptionGroup flies out, and the default traceback module in 3.11 renders it like this:

  + Exception Group Traceback (most recent call last):
  |   File "/Users/tintvrtkovic/pg/cattrs/a01.py", line 33, in <module>
  |     structure({"inner": {"a": 'not_an_int', "b": 1}}, Outer)
  |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  |   File "/Users/tintvrtkovic/pg/cattrs/src/cattr/converters.py", line 308, in structure
  |     return self._structure_func.dispatch(cl)(obj, cl)
  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  |   File "<cattrs generated structure __main__.Outer>", line 14, in structure_Outer
  | ExceptionGroup
  +-+---------------- 1 ----------------
    | Exception Group Traceback (most recent call last):
    |   File "<cattrs generated structure __main__.Outer>", line 5, in structure_Outer
    |   File "<cattrs generated structure __main__.Test>", line 19, in structure_Test
    | ExceptionGroup
    | Structuring class attribute Outer.inner
    +-+---------------- 1 ----------------
      | Traceback (most recent call last):
      |   File "<cattrs generated structure __main__.Test>", line 5, in structure_Test
      | ValueError: invalid literal for int() with base 10: 'not_an_int'
      | Structuring class attribute Test.a
      +---------------- 2 ----------------
      | Traceback (most recent call last):
      |   File "<cattrs generated structure __main__.Test>", line 15, in structure_Test
      | KeyError: 'c'
      | Structuring class attribute Test.c
      +------------------------------------
    +---------------- 2 ----------------
    | Traceback (most recent call last):
    |   File "<cattrs generated structure __main__.Outer>", line 10, in structure_Outer
    | KeyError: 'c'
    | Structuring class attribute Outer.c
    +------------------------------------

The Structuring class attribute Test.a strings are notes, added by cattrs.

It's a little messy, but there's a lot of information in there. The Outer class has errors for attributes inner (an ExceptionGroup) and c (a KeyError), and diving in, the Inner class (under inner) has errors for a (ValueError) and c (KeyError).

Anyway, this is how the traceback module prints this. I'm sure a Rich version would be much easier on the eyes ;)

[willmcgugan]

I would like to add support for ExceptionGroups. I'd probably wait until it is official and I've wrapped my head around it a bit more.

[Tinche]

Both of these PEPs (ExceptionGroups and __note__) have now been accepted! 🎉

[AndreasBackx]

I've put up a draft PR just to primarily get some feedback (see PR). Let me know what everyone thinks.

[Zac-HD]

This looks great! I'd love to see support for PEP-678 notes too (as the author I may be biased!), but that would make sense as a separate PR.

[AndreasBackx]

Yes, I was hoping to do that as well. Though it might depend on the feedback I get on that so I decided to get some initial feedback first.

[BabakAmini]

I'm also looking for this feature. Does anybody happen to have any update on when it will be available?

[AndreasBackx]

@BabakAmini #3033 is a proof of concept. I am waiting to hear back from one of the project members to see if this PoC is the right way forward. Then I can polish it for a proper PR and implement __notes__ as well possibly.

[flying-sheep]

I think this is getting more urgent. By now frameworks like Typer turn on Rich by default while exception groups are being used in the wild. When Rich’s exception handler is activated, users now sometimes see less information than without it, so I’d say this deserves some prioritization.

I’d say you should either add a hotfix that disables Rich’s rendering for exception groups while real support is in the pipes, or get that PR in quickly.

[mawildoer]

It's a bit of a bonkers work around, but if you install this or something like it as your program's exception hook, it'll print all the exception group exceptions

import logging
import sys
from rich.logging import RichHandler

logging.basicConfig(
    level="NOTSET",
    format="%(message)s",
    datefmt="[%X]",
    handlers=[RichHandler(rich_tracebacks=True)]
)

log = logging.getLogger("rich")
try:
    raise ExceptionGroup("test", [Exception("test1"), Exception("test2")])
except *Exception as ex:
    ex_group = ex
else:
    sys.exit(0)

for ex in ex_group.exceptions:
    try:
        raise ex
    except Exception:
        log.exception("Error occurred at program exit")

raise ex_group