Attempting to get OfflineAudioContext.suspend to work

[orottier]

Hey @b-ma I could maybe use your help. I was trying to get suspend to work but I'm running into

Error: called `Result::unwrap()` on an `Err` value: Error { status: InvalidArg, reason: "", maybe_raw: 0x0, maybe_env: 0x0 }
    at OfflineAudioContext.startRendering (/Users/otto/Projects/node-web-audio-api-rs/monkey-patch.js:136:35)

This happens when the rust lib is trying to run the JsFunction

   4: node_web_audio_api_rs::offline_audio_context::_generated_suspend_and_generated_::{{closure}}
             at ./src/offline_audio_context.rs:506:9
   5: core::ops::function::FnOnce::call_once{{vtable.shim}}
             at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/core/src/ops/function.rs:250:5
   6: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
             at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/alloc/src/boxed.rs:2007:9
   7: web_audio_api::render::thread::RenderThread::render_audiobuffer_sync
             at /Users/otto/Projects/web-audio-api-rs/src/render/thread.rs:202:17

It is maybe because the function reference cannot be resolved. I'm trying to read https://nodejs.org/api/n-api.html#references-to-values-with-a-lifespan-longer-than-that-of-the-native-method but not sure if it is related and how to continue. Any ideas?

[orottier]

Note this requires the latest checkout of orottier/web-audio-api-rs#407 to compile. Otherwise we need thread safe jsfunction which complicates things further.

Solving this issue might also pave the way for the event listeners?

[orottier]

Also note that I'm aware this is not spec compliant in its current form, but I have a plan to make ctx.suspend(..).then(..) work via dark magic next

[orottier]

When I change the rust implementation to run the callback directly when suspend is called, like

     pub fn suspend_at<F: FnOnce(&mut Self) + 'static>(&mut self, suspend_time: f64, callback: F) {
        (callback)(self);
     }

there is no issue. The JsFunction is executed properly and the result is as expected. So it must have something to do with the async context not having access to the method, I believe.

[b-ma]

Hey, I managed to make the callback run without crashing by using the ThreadSafeCallContext, but the callback is actually executed after startRendering is fully executed so the buffer is full of silence... need to investigate this further

[orottier]

Alright, ThreadSafeCallContext is the way to go because we need to revert orottier/web-audio-api-rs@1c66068 at some point.

Indeed strange the callback does not seem to run instantly. The rust lib correctly schedules the callback to execute on render quantum 173 but the next render quantum it does not notice any new audio graph messages to be handled.. Perhaps we need to look into how to make the call context blocking or so..

[orottier]

Assuming that the JsFunction is handed to the event loop, which is blocked by the actual execution of start_rendering_sync, it could be that we need another design. I maybe should look into a more spec compliant way of suspend where we don't resume immediately after the closure is run. Perhaps this allows the JsFunction to run in time (after suspend, before resume).

However we are between a rock and a hard place here because that means that start_rendering needs to be an async method in rust which will have a lot of impact because we then need to bring an async executor (e.g. tokio) inside the node bindings...

I have another design up my sleeve where we model the rust start_rendering method with a dedicated render thread and a simple shim of the Promise API on the rust side backed by https://crates.io/crates/oneshot
Might be able to sketch a solution for it in the next few days

[b-ma]

Assuming that the JsFunction is handed to the event loop, which is blocked by the actual execution of start_rendering_sync, it could be that we need another design. I maybe should look into a more spec compliant way of suspend where we don't resume immediately after the closure is run. Perhaps this allows the JsFunction to run in time (after suspend, before resume).

Hum yup, that sounds quite logical indeed

However we are between a rock and a hard place here because that means that start_rendering needs to be an async method in rust which will have a lot of impact because we then need to bring an async executor (e.g. tokio) inside the node bindings...

No problem for me to go this way if you want to have a try, tokio is already integrated into napi-rs to handle async stuff from what I understand

[orottier]

Just wanted to let you know I am actually working on async rendering, but it is really hard. Async stuff is not my cup of tea

[b-ma]

Seems to work like a charm! Really nice!

[orottier]

That's great news!

I will merge this (after a tiny bit of polishing) into the wpt branch, which we can also clean up a bit further to be merged into the main branch.

[b-ma]

wpt branch, which we can also clean up a bit further to be merged into the main branch.

+1

[b-ma]

Sorry, I just trashed the diff by merging main, didn't realize there was so much change to merge

[orottier]

Alright, continuing this work in #42 now