pyodide-worker raising error with pyodide 0.26.1

[MarcSkovMadsen]

I was trying to upgrade the pyodide and wasm code + documentation to latest version. I found that --pyodide-worker will not work with latest version of pyodide 0.26.1 and panel 1.5.0b2

Reproduce

• pip install panel==1.5.0-b.2
• Follow the instructions in the reference example https://panel.holoviz.org/how_to/wasm/convert.html
  • after the conversion manually update 0.25.0 to 0.26.1 in the script.js file.

See the error in the browser console

PythonError: Traceback (most recent call last):
  File "/lib/python312.zip/_pyodide/_base.py", line 574, in eval_code_async
    await CodeRunner(
  File "/lib/python312.zip/_pyodide/_base.py", line 394, in run_async
    coroutine = eval(self.code, globals, locals)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/lib/python312.zip/contextlib.py", line 158, in __exit__
    self.gen.throw(value)
  File "/lib/python3.12/site-packages/panel/util/parameters.py", line 51, in edit_readonly
    yield
  File "<exec>", line 8, in <module>
  File "/lib/python3.12/site-packages/param/parameterized.py", line 2319, in update
    restore = dict(self_._update(arg, **kwargs))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/lib/python3.12/site-packages/param/parameterized.py", line 2352, in _update
    self_._batch_call_watchers()
  File "/lib/python312.zip/contextlib.py", line 158, in __exit__
    self.gen.throw(value)
  File "/lib/python3.12/site-packages/param/parameterized.py", line 284, in _batch_call_watchers
    yield
  File "/lib/python3.12/site-packages/param/parameterized.py", line 2546, in _batch_call_watchers
    self_._execute_watcher(watcher, events)
  File "/lib/python3.12/site-packages/param/parameterized.py", line 2506, in _execute_watcher
    watcher.fn(*args, **kwargs)
  File "/lib/python3.12/site-packages/panel/reactive.py", line 412, in _param_change
    self._apply_update(named_events, properties, model, ref)
  File "/lib/python312.zip/contextlib.py", line 158, in __exit__
    self.gen.throw(value)
  File "/lib/python3.12/site-packages/panel/io/document.py", line 375, in unlocked
    yield
  File "/lib/python3.12/site-packages/panel/reactive.py", line 316, in _apply_update
    self._update_model(events, msg, root, model, doc, comm)
  File "/lib/python3.12/site-packages/panel/reactive.py", line 344, in _update_model
    model.update(**msg)
  File "/lib/python3.12/site-packages/bokeh/core/has_props.py", line 483, in update
    setattr(self, k, v)
  File "/lib/python3.12/site-packages/bokeh/core/has_props.py", line 336, in __setattr__
    return super().__setattr__(name, value)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/lib/python3.12/site-packages/bokeh/core/property/descriptors.py", line 332, in __set__
    self._set(obj, old, value, setter=setter)
  File "/lib/python3.12/site-packages/bokeh/core/property/descriptors.py", line 620, in _set
    self._trigger(obj, old, value, hint=hint, setter=setter)
  File "/lib/python3.12/site-packages/bokeh/core/property/descriptors.py", line 698, in _trigger
    obj.trigger(self.name, old, value, hint, setter)
  File "/lib/python3.12/site-packages/bokeh/model/model.py", line 571, in trigger
    super().trigger(descriptor.name, old, new, hint=hint, setter=setter)
  File "/lib/python3.12/site-packages/bokeh/util/callback_manager.py", line 186, in trigger
    self.document.callbacks.notify_change(cast(Model, self), attr, old, new, hint, setter, invoke)
  File "/lib/python3.12/site-packages/bokeh/document/callbacks.py", line 251, in notify_change
    self.trigger_on_change(event)
  File "/lib/python3.12/site-packages/bokeh/document/callbacks.py", line 423, in trigger_on_change
    invoke_with_curdoc(doc, invoke_callbacks)
  File "/lib/python312.zip/contextlib.py", line 158, in __exit__
    self.gen.throw(value)
  File "/lib/python3.12/site-packages/bokeh/io/doc.py", line 82, in patch_curdoc
    yield
  File "/lib/python3.12/site-packages/bokeh/document/callbacks.py", line 453, in invoke_with_curdoc
    return f()
           ^^^
  File "/lib/python3.12/site-packages/bokeh/document/callbacks.py", line 422, in invoke_callbacks
    cb(event)
  File "/lib/python3.12/site-packages/panel/io/pyodide.py", line 346, in pysync
    dispatch_fn(json_patch, pyodide.ffi.to_js(buffer_map), msg_id)
  File "https://mnr-jupyterhub.de-prod.dk/mt-uk/user/masma/vscode/proxy/8000/pyodide/script.js", line 4, in sendPatch
pyodide.ffi.JsException: DataCloneError: Failed to execute 'postMessage' on 'DedicatedWorkerGlobalScope': [object Map] could not be cloned.

    at new_error (pyodide.asm.js:10:9965)
    at pyodide.asm.wasm:0x16dbfd
    at pyodide.asm.wasm:0x177431
    at _PyEM_TrampolineCall_JS (pyodide.asm.js:10:125894)
    at pyodide.asm.wasm:0x1c2eaf
    at pyodide.asm.wasm:0x2c7c0f
    at pyodide.asm.wasm:0x20a884
    at pyodide.asm.wasm:0x1c359c
    at pyodide.asm.wasm:0x1c38ab
    at pyodide.asm.wasm:0x1c3929
    at pyodide.asm.wasm:0x29e95d
    at pyodide.asm.wasm:0x2a4f54
    at pyodide.asm.wasm:0x1c3a69
    at pyodide.asm.wasm:0x1c36d2
    at pyodide.asm.wasm:0x176a95
    at callPyObjectKwargs (pyodide.asm.js:10:64068)
    at Module.callPyObjectMaybePromising (pyodide.asm.js:10:65316)
    at wrapper (pyodide.asm.js:10:27006)
    at Cn.e.port1.onmessage (pyodide.asm.js:10:102154)

[MarcSkovMadsen]

I see the same issue when trying to run the latest version of pyscript with the py-editor. I.e. #6995 has not been solved completely.

<!DOCTYPE html>
<html>
  <head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width,initial-scale=1.0">
    <script src="./mini-coi.js" scope="./"></script>
    <script type="text/javascript" src="https://cdn.bokeh.org/bokeh/release/bokeh-3.5.0.js"></script>
    <script type="text/javascript" src="https://cdn.bokeh.org/bokeh/release/bokeh-widgets-3.5.0.min.js"></script>
    <script type="text/javascript" src="https://cdn.bokeh.org/bokeh/release/bokeh-tables-3.5.0.min.js"></script>
    <script type="text/javascript" src="https://cdn.jsdelivr.net/npm/@holoviz/panel@1.5.0-b.2/dist/panel.min.js"></script>

    <link rel="stylesheet" href="https://pyscript.net/releases/2024.7.1/core.css">
    <script type="module" src="https://pyscript.net/releases/2024.7.1/core.js"></script>
  </head>
  <body>
    <div id="simple_app"></div>
    <script type="py-editor" config='{"packages": ["https://cdn.holoviz.org/panel/1.5.0-b.2/dist/wheels/bokeh-3.5.0-py3-none-any.whl", "https://cdn.holoviz.org/panel/1.5.0-b.2/dist/wheels/panel-1.5.0b2-py3-none-any.whl"]}'>
      import panel as pn

      pn.extension(sizing_mode="stretch_width")

      slider = pn.widgets.FloatSlider(start=0, end=10, name='Amplitude')

      def callback(new):
         return f'Amplitude is: {new}'

      pn.Row(slider, pn.bind(callback, slider)).servable(target='simple_app');
    </script>
  </body>
</html>

Error raised while processing Document events: DataCloneError: Failed to execute 'postMessage' on 'DedicatedWorkerGlobalScope': [object Map] could not be cloned.

Please also note the css files that cannot be loaded.

[MarcSkovMadsen]

Even if I set the pyodide interpreter to 0.25.0 it does not work. The app renders but nothing happens when I drag the slider.

<!DOCTYPE html>
<html>
  <head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width,initial-scale=1.0">
    <script src="./mini-coi.js" scope="./"></script>
    <script type="text/javascript" src="https://cdn.bokeh.org/bokeh/release/bokeh-3.5.0.js"></script>
    <script type="text/javascript" src="https://cdn.bokeh.org/bokeh/release/bokeh-widgets-3.5.0.min.js"></script>
    <script type="text/javascript" src="https://cdn.bokeh.org/bokeh/release/bokeh-tables-3.5.0.min.js"></script>
    <script type="text/javascript" src="https://cdn.jsdelivr.net/npm/@holoviz/panel@1.5.0-b.2/dist/panel.min.js"></script>

    <link rel="stylesheet" href="https://pyscript.net/releases/2024.7.1/core.css">
    <script type="module" src="https://pyscript.net/releases/2024.7.1/core.js"></script>
  </head>
  <body>
    <script type="py-editor" config='{ "interpreter": "https://cdn.jsdelivr.net/pyodide/v0.25.0/full/pyodide.mjs", "packages": ["https://cdn.holoviz.org/panel/wheels/bokeh-3.5.0-py3-none-any.whl", "https://cdn.holoviz.org/panel/1.5.0-b.2/dist/wheels/panel-1.5.0b2-py3-none-any.whl"]}'>
      import panel as pn

      pn.extension(sizing_mode="stretch_width")

      slider = pn.widgets.FloatSlider(start=0, end=10, name='Amplitude')

      def callback(new):
         return f'Amplitude is: {new}'

      pn.Row(slider, pn.bind(callback, slider)).servable(target='simple_app');
    </script>
    <div id="simple_app"></div>
  </body>
</html>

[philippjfr]

I suspect this is issue with the coincident implementation that PyScript uses to sync data between worker and frontend, or, equally likely, an issue with the way we use it.

@WebReflection would be very grateful for your input. The way I understand the error is that _link_docs_worker calls the dispatch_fn which is a JS function which is executed on the frontend. So behind the scenes coincident uses message passing to send the arguments from worker -> frontend, which in turn triggers the error:

pyodide.ffi.JsException: DataCloneError: Failed to execute 'postMessage' on 'DedicatedWorkerGlobalScope': [object Map] could not be cloned.

[WebReflection]

@philippjfr this issue starts with --pyodide-worker which we don't use at all ... then it shows error caused but network but it claims the error is some error that can't be dispatched ... I am a bit confused to be honest ... and Maps, even recursive one, can be cloned ... what kind of data needs to travel around? if that includes Blobs or Files or not supported data then I am afraid you need your own conversion there and the issue should be gone.

Quoting that module used to postMessage via buffers:

not supported yet: Blob, File, FileList, ImageBitmap, ImageData, and (raw) ArrayBuffer, but typed arrays are supported without major issues, but u/int8, u/int16, and u/int32 are the only safely suppored (right now).

[WebReflection]

@MarcSkovMadsen out of curiosity, does this work if you add just worker attribute and use <script type="py" ...> instead of py-editor as type? it could be that PyEditor is lacking behind some needed check/fix although I am sure this has been previously tested already too.

[MarcSkovMadsen]

Here is a summary @WebReflection

• panel + py-editor 2024.7.1 + pyodide 0.25.0: works. See https://marcskovmadsen.pyscriptapps.com/panel-py-editor-not-working/v1/.
• panel + py-editor 2024.7.1 + pyodide 0.26.1: does not work. See https://marcskovmadsen.pyscriptapps.com/panel-py-editor-not-working/v2/
• panel + py-script 2024.7.1 without worker + pyodide 0.26.1: works. See https://marcskovmadsen.pyscriptapps.com/panel-py-editor-not-working/v3/
• panel + py-script 2024.7.1 with worker + pyodide 0.26.1: does not work. See the console of https://marcskovmadsen.pyscriptapps.com/panel-py-editor-not-working/v4/
• panel + py-script 2024.7.1 with worker + pyodide 0.25.0: does not work. See the console of https://marcskovmadsen.pyscriptapps.com/panel-py-editor-not-working/v5/. This one is rather surprising as the similar editor works?

[WebReflection]

The TL;DR of the issue is that latest pyodide introduced LiteralMap to make it possible to deal with Pyodide converted values (the default .to_js()) as if these were plain objects so that these can be used out of the box with all APIs in the JS world that pretty rarely expect a Map instead of an object literal.

These conversions might be done somewhere else (either panle or bokeh in this case) and the transformer was not accounting for these kind of proxied maps so it was failing.

It was also not recursively check for these kind of proxied maps or other PyProxy values within arrays so now polyscript takes care of that and once it lands in PyScript and we release this issue whould be eventually gone.