[JS] Wasm and worklets

[annevk]

I noticed that https://littledan.github.io/spec/document/JS.html doesn't expose Wasm in worklets. I suspect that might be useful, e.g., for audio worklets although maybe what we actually want is to not go through JavaScript at all in that scenario and start up a wasm-run worklet.

cc @padenot

[padenot]

In general, you'd want to prepare the sources outside of the worklet, so this is fine for now. However, it's critical to be able to import and export things, so yes, this spec will need to be modified. Thanks for the heads-up.

@hoch probably wants to know about this issue as well.

[littledan]

@annevk To be specific, are you suggesting that things be marked in WebIDL as Exposed=(Window,Worker,AudioWorklet) ? That's what I see in HTML at the moment. Will there be a way of marking things as available in any worklet, not just audio worklets?

[annevk]

@littledan I'm not a 100% sure what would be most useful so I haven't suggested anything on purpose. All worklets should be covered by using the keyword Worklet.

[littledan]

OK, I guess once you have a recommendation, I'll update the spec to use it.

[annevk]

I recommend talking with implementers. I just noticed this was wrong, but I won't be able to drive it for the foreseeable future.

[littledan]

Implementers, is there any worklet type that you wouldn't expose Wasm to?

[jfbastien]

I'd go further and expose / spec some WebAssembly-only worklets, without JS.

[hoch]

@littledan

things be marked in WebIDL as Exposed=(Window,Worker,AudioWorklet)

To clarify the change, it's because AudioWorklet was the only one requires bi-directional communication. AFAIK, the other two worklets (Paint/Animation) did not want MessagePort for V1.

FWIW, AudioWorklet with WASM (which is currently available in Chrome Canary with experimental flag) runs rather nicely. I don't see any reason for other worklets to reject the idea.

cc/ @bfgeek

[hoch]

One issue though (originally raised by @majido):

worklet.addModule() expects the code to be evaluated in WorkletGlobalScope synchronously but WebAssembly.instantiate() is asynchronous. Isn't this a conflict? Without a communication channel, the promise resolution from worklet.addModule() does not guarantee a valid WASM instance.

[majido]

Is there a plan to ever have Worklet.addModule supports WASM modules similar to ES6 Modules. I think that will be ideal. It will also address the issue below.

In general I don't see why we shouldn't expose WebAssembly API in worklet scope. In fact, it will be nice to use them in AnimationWorklet to boost performance. I have just one minor concern.

Worklet.addModule is async but it returns a promise that is resolved once the modules script is executed. At the moment, once that promise is resolved, it is reasonable to assume that all the additional hooks (e.g., painter, animator, processor) are properly registered in worklet execution context. In another word, main thread can now use those without concern.

With introduction of WebAssembly API that is no longer true given that the registration can happen asynchronously. So there may be a need for a method to signal back to the main js context that the module initial work is complete. It seems AudioWorklet can get away with this
using the message port but I cannot think of a similar thing in pain, layout, and animation worklet.

audioWorklet.addModule('xyz.js').then(()=> { 
  // XZY processor is registered in worklet so I can use it. 
  // This is no longer the case with WASM?
});

// In JS modules
registerProcessor(class XYZ{ })

//In WASM modules
WebAssembly.instantiate().then(() => registerProcessor(class XYZ{}))

[littledan]

@hoch Synchronous instantiation is available through the WebAssembly.Instance constructor. If you can pass over a compiled WebAssembly.Module instance through a communication channel like postMessage, this would be the most efficient way to start up Wasm running on a Worklet. But even if you couldn't, you can use the WebAssembly.Module constructor to synchronously compile Wasm code.

@majido ES6 module integration is planned but not coming immediately. See WebAssembly/design#1087

[jfbastien]

Note that some engines limit the size of modules which can be compiled synchronously on the main thread.

[chicoxyzzy]

WebAssembly in worklets will be very useful for Houdini specs worklets to speedup custom layouts, etc.

[littledan]

At this point, Wasm is exposed in worklets in the current JS API. I've made a proposal at WebIDL that we make a mechanism to expose to everything, but we haven't come to agreement on merging that yet. If another global object is added to the web platform, we should consider adding it to the WASM/JS API specification.