Configurable AudioWorklet process block size (higher than 128 samples)?

[josh83abc]

Hello everybody!

As far as I see, the AudioWorkletProcessor process block size is 128 samples, like any AudioNodes.

I haven't really tested the robustness of the audio stream but this value seems to be pretty low to me if you compare it with what you see in desktop music softwares (process block size is usually more than 512 samples and can easily be 1024 or 2048 samples).

I don't know if it is related but I can hear frequent tiny audio glitches with the AudioWorklet sinus demo when I switch from tab to tab. https://googlechromelabs.github.io/web-audio-samples/audio-worklet/basic/hello-audio-worklet.html

Also in my app, the audio latency is not the most important aspect, so I prefer to have a higher latency allowing a better robust audio stream.

Would it be possible to change the AudioWorkletProcessor process block size in the future? Is there a workaround?

PS : this post also talks about it : #1466

[hoch]

The internal buffering (e.g. FIFO) in the processor might be the way to resolve the buffer size difference. Even if we change the spec to accommodate the variable buffer size, the implementation will do the buffering internally anyway because the other parts of WebAudio use 128 frames.

Having the same render quantum size is the key to the lower latency, and it was the one of design goals of AudioWorklet. I doubt that WG will change something fundamental like this now, but it can be revisited later for the V2.

[hoch]

Also the glitch is an implementation issue. If you have a repro case, please file a bug and cc me (hongchan@).

[sletz]

@josh83abc: you can get more info on glitch issues on this bug report: https://bugs.chromium.org/p/chromium/issues/detail?id=796330&can=2&start=0&num=100&q=component%3ABlink%3EWebAudio&colspec=ID%20Pri%20M%20Stars%20ReleaseBlock%20Component%20Status%20Owner%20Summary%20OS%20Modified&groupby=&sort=

[josh83abc]

Thanks @sletz for this link!! lot of very interesting stuff in it. Avoid audio glitches is the main priority of my music player, so I'm very interested in understanding all the details.

@hoch, thanks a lot for all these technical details. I'm aware that all the webaudio graph is processed with a render quantum of 128 samples and so does the AudioWorkletProcessor. For now I didn't really tested the audio robustness of the webaudio graph rendering, it is pretty good for sure but I will try to reach its limits. Also I will test on iOS and Android. Tell me if I'm wrong but I don't think a FIFO on top of the AudioWorkletProcessor will improve the robustness that much.

For the V.next, it would be interesting if the webaudio render quantum size could be ajusted by the programmer who doesn't need a very low latency. For instance, a render quantum of 1024 samples (23ms@44.1khz) would be way enough for my application. Actually Flash is using this value to process audio, the latency is ok and there is no audio glitch at all, even if Chrome crashes.

[padenot]

The internal block size of the Web Audio rendering graph provides an lower bound for the OS buffer size, not an upper bound (and even then, it's not strictly true).

The Web Audio API already has a mechanism to request a higher latency on an AudioContext, using AudioContextLatencyCategory.

Please open issues on the UA's bug tracker for issues about implementations.

[josh83abc]

Thanks a lot @padenot for letting me aware of the latencyHint option of the AudioContext!! I totally missed it when I read the spec, but this exactly what I was talking about : having an option to ajust the audio latency according the need of the app.
Amazing it already exists!! webaudio is dope :)

[rtoy]

Based on #1503 (comment), I think we can close out this issue. I don't think there's anything that needs to be done for the spec.

[fr0m]

Back then using ScriptProcessorOrNode, buffer size controls how frequently the audioprocess event is dispatched, well latencyHint doesn't affect that.

Since the AudioWorkletProcessor process buffer size is 128, the frequency of process is triggered is overwhelmingly higher than using ScriptProcessorOrNode with 1024 buffer size or higher.

This situation may cause high CPU occupation since expensive operation may be executed in the process, which is the situation in my project. So do we have another way to control the frequency of process triggered?

[padenot]

This situation may cause high CPU occupation since expensive operation may be executed in the process, which is the situation in my project. So do we have another way to control the frequency of process triggered?

No. If it's too expensive when computing 128 frames, why is it OK when computing 1024 frames? You exactly have the same amount of time per frame to compute the audio.

You probably simply need to optimize your code.

[sletz]

Because running the complete audio chain adds a fixed processing cost per buffer. When smaller buffers are used, the fixed cost is added more times, so finally takes more of the available CPU.

[fr0m]

Thanks for the reply.

We use audioWorklet to do audio live stream, encoded audioBuffer is sent via WebSocket in AudioWorkletProcessor. It's expensive because of the frequency, not frame.

I can cache the buffer, and send the buffer in a particular buffer size. But it will be much better if the unnecessary process will not be triggered at all. Will you take this situation into account?

[positonic]

@sletz that benchmark link for glitches is dead, has it moved somewhere?

[sletz]

Which link ?