On wasm, window.cursor.grab_mode is not updated when cursor is unlocked.

[sarahkittyy]

Bevy version

0.10.1

What you did

Using examples/input/mouse_grab.rs:

fn grab_mouse(
    mut windows: Query<&mut Window>,
    mouse: Res<Input<MouseButton>>,
    key: Res<Input<KeyCode>>,
) {
    let mut window = windows.single_mut();

    if mouse.just_pressed(MouseButton::Left) {
        window.cursor.visible = false;
        window.cursor.grab_mode = CursorGrabMode::Locked;
    }

    if key.just_pressed(KeyCode::Escape) {
        window.cursor.visible = true;
        window.cursor.grab_mode = CursorGrabMode::None;
    }
}

Left clicking on the canvas will lock the cursor. Hitting escape will unlock the cursor.

However, this is handled by the browser only, and the Escape key event is never actually sent to bevy. window.cursor.grab_mode will still be Locked. Left-clicking at this point will do nothing, as grab_mode is already Locked and no state is changed.

Clicking Hitting escape a second time will then set grab_mode = None, allowing you to re-lock by clicking.

What went wrong

• The browser detaching your cursor should be picked up by bevy and grab_mode should be set to None, or at the very least an event should be emitted.

Additional information

• In JS, you can listen to the change with document.addEventListener('pointerlockchange', /* ... */)

[bonsairobo]

Not sure if I should make a separate issue, but when the mouse is grabbed in the browser, it always hides it, regardless of the value of the visible flag.

[janhohenheim]

My workaround is running the following on Update:

fn capture_cursor(
    mut q_window: Query<&mut Window>,
    mouse_button_input: Res<ButtonInput<MouseButton>>,
) {
    let Ok(mut window) = q_window.get_single_mut() else {
        return;
    };

    // Has no real effect on Web anyways,
    // as `CursorGrabMode::Locked` and `CursorGrabMode::Confined` imply `visible = false`.
    // You could also just run it once for native. Eh.
    window.cursor.visible = false;

    // Because Bevy caches the last `grab_mode`, 
    // this does *not* permanently lock the cursor when running on `Update`.
    // In other words, the user can still press `Esc` to unlock the cursor in web
    // even when this runs every frame.
    // When pressing `Esc`, the `grab_mode` does not change: <https://github.com/bevyengine/bevy/issues/8949>
    window.cursor.grab_mode = CursorGrabMode::Locked;

    // Check if the user has clicked into our window.
    if mouse_button_input.just_pressed(MouseButton::Left) {
        // Clear Bevy's grab mode cache by setting a different grab mode
        // because an unlocked cursor will not update the current `CursorGrabMode`.
        // The next `Update` frame will reset the `grab_mode` to `CursorGrabMode::Locked` again.
        window.cursor.grab_mode = CursorGrabMode::Confined;
    }
}

[jf908]

The web's pointer lock system is quite problematic for winit's simple API. Winit only has a function for setting the grab mode, it doesn't have any functionality for getting the current mode or listening to changes.

There are currently 2 big problems with bevy's pointer lock on web:

1. Pointer lock can fail for a number of reasons (not supported, disabled in permissions, user didn't click recently, pointer lock was exited recently) but the only ways to know if it failed is to await the returned promise (only implemented on Chrome) or to listen to the document's pointerlockchange event to see if it worked. Since these can only be done asynchronously, winit cannot and doesn't return any error if pointer lock fails.
2. The user can unlock the pointer whenever they wish by pressing Escape which triggers the document's pointerlockchange event and doesn't emit any keyboard events. Winit and bevy do not listen to this event.

Bevy's grab_mode value is therefore just whatever you last set it to, not the cursor's actual state.

This doesn't really feel like a bug in winit, it's just missing behaviour to detect if the grab mode changes externally which I think only makes sense on the web. Therefore maybe bevy could implement this behaviour itself with web_sys? It's worth documenting at least.

[janhohenheim]

Wrote a hotfix for the moment; just add FixPointerUnlockPlugin and the grab mode will update correctly :)

[janhohenheim]

Marking as blocked per winit. Don't know if there's an issue for this open with them.

[jf908]

Made one! rust-windowing/winit#4222

[janhohenheim]

@jf908 winit also swallows up the Escape key event in this case, maybe that would be good to track somewhere as well