Description
Bevy version
0.11.0
What you did
I compile my game to WASM.
When I try to play my games on mobile browser I noticed intemittent/broken sound.
On desktop browser everything "works" well.
Additional information
I investigated the issue.
First of all I created some patches audio_context_patch.js for original AudioContext
const audioContextList = [];
function constructorPatch(obj, key) {
obj[key] = new Proxy(obj[key], {
construct(target, args) {
const audioContext = new target(...args);
audioContextList.push(audioContext);
return audioContext;
},
apply(target, thisArg, args) {
console.log(`patch apply ${key}`);
return Reflect.apply(target, thisArg, args);
},
});
}
constructorPatch(globalThis, 'AudioContext');
constructorPatch(AudioBufferSourceNode.prototype, 'start');
const userInputEventNames = [
"mousedown",
"pointerdown",
"touchdown",
"keydown",
];
function resumeAudioContexts() {
console.log('resumeAudioContexts');
let count = 0;
audioContextList.forEach((context) => {
if (context.state !== "running") {
context.resume();
} else {
count++;
}
});
if (count > 0 && count === audioContextList.length) {
userInputEventNames.forEach((eventName) => {
document.removeEventListener(eventName, resumeAudioContexts);
});
}
}
userInputEventNames.forEach((eventName) => {
document.addEventListener(eventName, resumeAudioContexts);
});Above code does 2 things:
- Resume
AudioContextwhen user "interacted" with the page some how - Make interceptor/patch for
play()method forAudioBufferSourceNode
For example you can compile to WASM the simplest audio example:
//! This example illustrates how to load and play an audio file.
use bevy::prelude::*;
fn main() {
App::new()
.add_plugins(DefaultPlugins)
.add_systems(Startup, setup)
.run();
}
fn setup(asset_server: Res<AssetServer>, mut commands: Commands) {
commands.spawn(AudioBundle {
source: asset_server.load("sounds/Windless Slopes.ogg"),
..default()
});
}compile to WASM is pretty straightforward:
- Add WebAssembly support to your Rust installation
rustup target install wasm32-unknown-unknown- Install wasm-bindgen CLI
cargo install wasm-bindgen-cli- compile to WASM
wasm-bindgen --no-typescript --out-name wasm_bindgen_core --out-dir dist/ --target web target/wasm32-unknown-unknown/release/....your_file_name....wasm- run compiled example with following
index.html:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport"
content="width=device-width, height=device-height, initial-scale=1, minimum-scale=1, maximum-scale=1, user-scalable=no" />
<title>Example</title>
</head>
<body>
<script src="./audio_context_patch.js"></script>
<script type="module" src="./wasm_bindgen_core.js"></script>
</body>
</html>notice that audio_context_patch.js must be before regular wasm_bindgen_core.js to apply patches.
As a result in browser console I see following:
AudioBufferSourceNode.play() keeps calling each frame again and again!
This is too overwhelming for mobile browser, that is why I hear corrupted sound.
This is appropriate for desktop browser some-how, probably because of performant CPU.
Navigating through the call stack of WASM shows not much for me:
real (wasm_bindgen_core.js:199)
requestAnimationFrame (async)
(anonymous) (wasm_bindgen_core.js:399)
handleError (wasm_bindgen_core.js:227)
imports.wbg.__wbg_requestAnimationFrame_d082200514b6674d (wasm_bindgen_core.js:398)
$winit::platform_impl::platform::event_loop::runner::Shared<T>::apply_control_flow::hf7f85fd12e8e3b67 (wasm_bindgen_core_bg.wasm:0x4218f2)
$winit::platform_impl::platform::event_loop::runner::Shared<T>::run_until_cleared::h52fc249ce9680fb7 (wasm_bindgen_core_bg.wasm:0x4e3100)
$winit::platform_impl::platform::backend::timeout::AnimationFrameRequest::new::{{closure}}::hf77d92138eacb4da (wasm_bindgen_core_bg.wasm:0x783754)
$wasm_bindgen::convert::closures::invoke0_mut::hf7a6e5275be6b226 (wasm_bindgen_core_bg.wasm:0x7b5f70)
...
and the same loop again and again
...
I didn't investigate bevy 0.10.1 the same way however I heared the same corrupted sound in mobile browser - so I think this issue persisted a long time ago, nobody tested sound in mobile browser.
Metadata
Metadata
Assignees
Labels
Type
Projects
Status