Fix signalctl for Python 3.11 #4374
Conversation
Python 3.11 deprecated the `msg` param in `Future.cancel()`, we can no longer depend on the `msg` to mark the reason of cancellations. That means, it is impossible to match recorded signals to CancelledErrors and replace them with SignalErrors. The solution here is to raise an ExceptionGroup containing all signal errors, and its `__cause__` will be set to the cancellation error chain displaying where the signal hit the user code. User error will be merged into the same ExceptionGroup with signal errors, so be prepared to use the `except*` grammar to handle exceptions. One exception is CancelledError itself. If `SignalController.wait_for()` is cancelled from outside (and no signals captured after that), a naked CancelledError will still be raised. This made it possible to chain `SignalController.wait_for()` to `asyncio.wait_for()` for timeouts. Similarly, a group of signal errors can still be found in the __cause__.
fantix
force-pushed
the
signalctl-py311
branch
from
22fd235
to
4ae0aeb
Compare
|
Oh mypy! python/mypy#12840 |
|
Hm, is there a way to retain Python 3.10 compatibility for now without too much hassle? If not, then I think we should sit on this until Python 3.11 is out and all dependencies are in order. |
Yeah I think so, instead of using ExceptionGroup, we could just chain the signal errors with |
This partially reverts 35f1a54
fantix
force-pushed
the
signalctl-py311
branch
from
9cb6683
to
c8b36bb
Compare
| # Event 3: cancelled by outer code. | ||
| if not fut.done(): | ||
| fut.cancel(msg=e) | ||
| fut.cancel() | ||
| outer_cancelled_at_last = True |
There was a problem hiding this comment.
The name is a bit cryptic. Maybe just call it waiter_was_cancelled?
There was a problem hiding this comment.
waiter_was_cancelled means "event 3 happened at least once", while we need "event 3 is the last event", because without ExceptionGroup, we can only raise one error, and it should be the last one in user-code. For examplye, if an outer cancel - say asyncio.wait_for() expired - was handled in a "finally" block in user-code, but we captured a sginal in the "finally" block, we should raise the SignalError instead of the CancelledError, in which case this flag should be False even though "waiter was cancelled" once.
On the same page I'm still open to suggestions for naming.
Python 3.11 deprecated the `msg` param in `Future.cancel()`, we can no longer depend on the `msg` to mark the reason of cancellations. That means, it is impossible to match recorded signals to CancelledErrors and replace them with SignalErrors. The solution here is to chain all signal errors with `__context__` and raise the last error, and its `__cause__` will be set to the cancellation error chain displaying where the signal hit the user code. * Bump CI to use Python 3.11 * Compute CI cache key hash with major Python version * Fix cargo-test with venv cache key
Python 3.11 deprecated the `msg` param in `Future.cancel()`, we can no longer depend on the `msg` to mark the reason of cancellations. That means, it is impossible to match recorded signals to CancelledErrors and replace them with SignalErrors. The solution here is to chain all signal errors with `__context__` and raise the last error, and its `__cause__` will be set to the cancellation error chain displaying where the signal hit the user code. * Bump CI to use Python 3.11 * Compute CI cache key hash with major Python version * Fix cargo-test with venv cache key
Python 3.11 deprecated the
msgparam inFuture.cancel(), we can no longer depend on themsgto mark the reason of cancellations. That means, it is impossible to match recorded signals to CancelledErrors and replace them with SignalErrors.The solution here is to raise an ExceptionGroup containing all signal errors, and its
__cause__will be set to the cancellation error chain displaying where the signal hit the user code. User error will be merged into the same ExceptionGroup with signal errors, so be prepared to use theexcept*grammar to handle exceptions.One exception is CancelledError itself. If
SignalController.wait_for()is cancelled from outside (and no signals captured after that), a naked CancelledError will still be raised. This made it possible to chainSignalController.wait_for()toasyncio.wait_for()for timeouts. Similarly, a group of signal errors can still be found in the__cause__.UPDATE: the use of ExceptionGroup is dropped because mypy isn't ready yet, now we're chain the signal errors with
__context__.