Upstream widget improvements

[domoritz]

Quak is really exciting and I love how well it integrates into Jupyter and the general Python data science ecosystem. Right now, this package copies a lot of the code from the mosaic widget but also improves upon it. What do you think is the best way to upstream those improvements while also making the widget reusable so you don't need to repeat logic?

[manzt]

Thanks! Agreed, I'd initially hoped to extend from MosaicWidget, but quak's narrower focus required some APIs that I couldn't quite map conceptually onto the more general purpose case (uwdata/mosaic#449).

Maybe on the Python side, one potential solution is to extend from a mixin-like class with the custom handler or even directly from the MosaicWidget itself, just overriding the _esm. Or mosaic_widget could export a handle_mosiac_custom_message handler that I could wire-up on the Python side.

However, IMO the front-end and python code wiring up the connector over Jupyter comms should really be packaged together somehow. Unfortunately, I've been swamped with SciPy and wrapping up my PhD, which has delayed progress on this front. I initially hoped to tinker with a plugin-based system before making quak public, but it ended up becoming a blocker.

One crazy idea: exporting a module to be consumed on the front-end (like how anywidget sends ESM).

connector_esm = """
export function createMosaicConnector(model, { arrow }) {
  /* ... */ 
  return {
    query(model) { /* ... */ },
  }
}
"""

def handle_mosaic_front_end_message(conn, msg, buffers):
    # logic to handle the message and respond

Then in quak we could do something like:

import * as arrow from "npm:apache-arrow";
import * as mc from "npm:@uwdata/mosaic-core";

async function loadEsm(text) {
  let url = URL.createObjectURL(new Blob([text], { type: "text/javascript" }));
  let mod = await import(url);
  URL.revokeObjectURL(url);
  return mod;
}

export default () => {
  let coordinator = new mc.Coordinator();
  return {
    async initialize({ model }) {
      let { createMosaicConnector } = await loadEsm(model.get("connector_esm"));
      let connector = createMosaicConnector(model, { arrow });
      coordinator.databaseConnector(connector);
    },
    async render({ model, el }) {
      /* ... */
    }
  }
}

Just a rough idea... Seems boilerplatey... but at least this way we could try to nail down the Python/JS connection part together.