What is the consensus on building widgets that can work in the background?

[tswast]

In the Jupyter Widgets docs, I found this short guide on building widgets that can update and/or respond to user input asynchronously / non-blocking. https://ipywidgets.readthedocs.io/en/latest/examples/Widget%20Asynchronous.html#updating-a-widget-in-the-background

I was able to get the progress bar sample to mostly work with anywidget by doing something like the following:

import anywidget
import traitlets
import threading
import time

class BackgroundProgress(anywidget.AnyWidget):
    _esm = """
    export function render({ model, el }) {
        let output = document.createElement('div');
        output.classList.add('progress');
        let bar = document.createElement('div');
        bar.classList.add('progress-bar');
        bar.style.height = "20px";
        bar.style.backgroundColor = "blue";
        output.appendChild(bar);
        
        el.append(output);

        function update_output() {
            bar.style.width = `${model.get('value')}px`;
        }

        update_output();

        model.on('change:value', update_output);
    }
    """
    _css = """
    """
    value = traitlets.Int(0).tag(sync=True)

    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)

        # if msg.get('event') == 'increment':
        # Define the blocking work to be done in a separate thread
        def blocking_work():
          for _ in range(100):
            time.sleep(1.0)  # Simulate blocking work
            self.value += 1
        
        # Start a new thread to run the blocking work
        thread = threading.Thread(target=blocking_work)
        thread.start()

# Instantiate and display the widget
counter_widget = BackgroundProgress()
counter_widget

For responding to user input while another cell is running, I haven't been able to find a solution.

import anywidget
import traitlets

class ButtonWidget(anywidget.AnyWidget):
    _esm = """
    export default {
        render({model, el}) {
            const button = document.createElement("button");
            button.textContent = "Click me";
            button.onclick = () => model.send("hello, world");
            el.appendChild(button);
            
           let output = document.createElement('div');
           el.append(output);

            function update_output() {
                output.innerText = `${model.get('message')}`;
            }

            update_output();
            model.on('change:message', update_output);
        }
    }
    """
    message = traitlets.Unicode("").tag(sync=True)

button_widget = ButtonWidget()

counter = 0

def update_message(_, content, _buffers):
    global counter
    button_widget.message = f"{counter}: {content}"
    counter += 1

button_widget.on_msg(update_message)
button_widget

This gets close-ish, but the on_msg handler doesn't run until any other running cells have stopped. I've been using the following cell in Jupyter Lab to test this out:

import time

for i in range(100):
    time.sleep(1)
    print(i, end=" ")

While it is running, the button widget doesn't get any UI updates.

I'm curious to hear if folks have found a solution to this? Perhaps it has to do with the fact that Traitlets is pretty synchronous? I found jupyter-widgets/ipywidgets#3293 for a request to migrate off of it for better true async support.

I'm aware of 3 solutions for concurrency right now in Python:

1. Multiprocessing: I don't think this applies to anywidget, but correct me if I'm wrong.
2. Threading: Kinda works for long-running work, but I couldn't figure out how to get event handlers to run in a background thread.
3. Async: Jupyter provides an async event loop now, but I think issues like Replace traitlets with pydantic? jupyter-widgets/ipywidgets#3293 mean that we can't really take advantage of this.