Async kernel

Async kernel is an IPython kernel for Jupyter that provides concurrent message handling via an asynchronous backend (asyncio or trio).

The kernel provides two external interfaces:

1. Direct ZMQ socket messaging via a configuration file and kernel spec - (Jupyter, VScode, etc).
2. An experimental callback style interface (Jupyterlite).

Documentation

Highlights

• Debugger client
• anyio compatible asynchronous backend (asyncio (default) or trio)
• aiologic thread-safe synchronisation primitives
• Backend agnostic multi-thread / multi-event loop management
• IPython shell
• Per-subshell user_ns
• GUI event loops ¹²
  • inline
  • ipympl
  • tk host and asyncio³ or trio⁴ backend running as a guest
  • qt host and asyncio³ or trio⁴ backend running as a guest
• Experimental support for Jupyterlite (try it online here 👈)

Prevent asynchronous deadlocks

The standard (synchronous) kernel implementation processes messages sequentially irrespective of the message type. The problem being that long running execute requests will make the kernel non-responsive. Another problem exists when an asynchronous execute request awaits a result that is delivered via a kernel message - this will cause a deadlock because the message will be stuck in the queue behind theblocking execute request⁵.

Async kernel handles messages according to the channel, message type and subshell id. So widget com message will get processed in a separate queue to an execute request. Further detail is given in the concurrency notebook, a Jupyterlite version is available here.

Installation

pip install async-kernel

Kernel specs

A kernel spec with the name 'async' is added when async kernel is installed.

Kernel specs can be added/removed via the command line.

The kernel is configured via the interface with the options:

• name
• display_name
• Parameters on the kernel including:
  • interface.backend
  • interface.backend_options
  • interface.loop
  • interface.loop_options
  • shell.timeout

Backends

The backend set on the interface is the asynchronous library the kernel uses for message handling. It is also the asynchronous library directly available when executing code in cells or via a console⁴.

Example - overwrite the 'async' kernel spec to use a trio backend

pip install trio
async-kernel -a async --interface.backend=trio

Gui event loop

The kernel can be started with a gui event loop as the host and the backend running as a guest.

asyncio backend

# tk
async-kernel -a async-tk --interface.loop=tk

# qt
pip install PySide6-Essentials
async-kernel -a async-qt --interface.loop=qt

trio backend

pip install trio
# tk
async-kernel -a async-tk --interface.loop=tk --interface.backend=trio

# qt
pip install PySide6-Essentials
async-kernel -a async-qt --interface.loop=qt --interface.backend=trio

For further detail about kernel spec customisation see command line and kernel configuration.

Origin

Async kernel started as a fork of IPyKernel. Thank you to the original contributors of IPyKernel that made Async kernel possible.

Footnotes

1. A gui (host) enabled kernel runs a gui event loop with the asynchronous backend running as guest. The host must be set before the kernel is started. This was a deliberate design choice to to ensure good performance and reliability. ↩

2. It is also possible to use a caller to run a gui event loop in a separate thread (with a backend running as a guest) if the gui allows it (qt will only run in the main thread). Also note that pyplot will only permit one interactive gui library in a process. ↩

3. The asyncio implementation of start_guest_run was written by the author of aiologic and provided as a gist. ↩ ↩²

4. trio's start_guest_run. ↩ ↩² ↩³

5. Ipykernel solves this issue specifically for widgets by using the concept of 'widget coms over subshells'. Widget messages arrive in a different thread which on occasion can cause unexpected behaviour, especially when using asynchronous libraries. ↩