Device Events API Discussion

[Osspial]

The Device Events API currently exists, but there seems to be informal consensus between the maintainers that it needs to be reworked somewhat before we can declare it stable. This issue is meant to house discussion on what changes should be made.

[ishitatsuyuki]

What's the device event API's purpose? A typical application should only be interested in events that happens when it's focused. Apart from that, DeviceEvent as it stands is mostly useless because backend implementations are mostly incomplete.

[Osspial]

It's got a couple purposes:

• Currently, it's used to get raw mouse movement data without OS filtering, which is useful for camera controls.
• In the future, it will be used for gamepad and gyro/accelerometer data.

Apart from that, DeviceEvent as it stands is mostly useless because backend implementations are mostly incomplete.

I don't get what you're going for there? DeviceEvent is implemented on all the desktop platforms, and even if the feature as it stands now had an incomplete implementation that wouldn't mean it's useless in principal.

[ishitatsuyuki]

I think I got confused because the Key variants (and the now removed Text variants). The former is only implemented on X11, which is why I said it's inconsistent.

TBH I think these thing have nothing to do with window focus (except gamepad); do we really receive those mouse events when the mouse is outside the window? This seems to be another confusing factor that makes the documentation harder to understand at a glance.

[msiglreith]

Is this something which could be actually handled independently of winit in a separate crate?

[Osspial]

@ishitatsuyuki Ah, that makes sense. FWIW, keyboard device input is implemented on the gamepad-device-events branch on Windows.

TBH I think these thing have nothing to do with window focus (except gamepad); do we really receive those mouse events when the mouse is outside the window? This seems to be another confusing factor that makes the documentation harder to understand at a glance.

DeviceEvents aren't associated with any particular window, and our documentation reflects that by delivering them independently of WindowIds or focus. Could you further clarify what part you find confusing? I ask, since I have a lot of exposure to our APIs and as such am a bad judge of what's clear, what's not, and why.

@msiglreith From what I've seen, no. On Windows, device events are delivered as standard events in the platform event queue, and I'd assume that's the case on other platforms.

Design-wise, why would we want it in a separate crate? It seems fairly natural for us to expose this, given the high-level goals of this crate.

[msiglreith]

@Osspial On windows, one can just create a new hidden window and poll from there 😛
Not sure about other platforms or if this would be feasible at all.
FWIW I did some experiments there: https://github.com/norse-rs/norse-haptic/blob/master/src/action.rs#L170-L201

Design-wise, why would we want it in a separate crate? It seems fairly natural for us to expose this, given the high-level goals of this crate.

I'm not sure what's supposed to be the scope of winit, IIRC gamepad support for example was always treated as it should be supported by a separate crate.

Pros: Keep the scope of winit small, possible supporting a wider range of devices (VR?)
Cons: Work, coupling with current approaches, unknown if feasible for other OSes

[Osspial]

On windows, one can just create a new hidden window and poll from there 😛

We pretty much do that already, actually. The role of that window has been expanded to handle device events on the gamepad-device-events branch. That also allows us to wait for gamepad events, instead of having to constantly poll for them.

I'm not sure what's supposed to be the scope of winit, IIRC gamepad support for example was always treated as it should be supported by a separate crate.

We expanded our scope to include that a bit ago, though I can understand the confusion there since Tomaka was pretty insistent that it didn't belong in Winit. See https://github.com/rust-windowing/winit/blob/master/FEATURES.md

[ishitatsuyuki]

DeviceEvents aren't associated with any particular window, and our documentation reflects that by delivering them independently of WindowIds or focus.

This is completely against intuition, as that's not GUI input is typically exposed.

The reason I get confused with this is due to the fact that you spawn a hidden window - I can't predict such implementation from the docs, and therefore I would just think receiving events without focus is not possible.

After all, capturing inputs globally is a kind of keylogging - am I correct?

[Osspial]

that's not GUI input is typically exposed.

That's true, but device events don't behave the same way as traditional GUI input. The OS creates window events by aggregating together events from multiple devices and filtering the device input (mouse movement and acceleration being the most notable examples of this), but the device events themselves can still be retrieved and processed without that filtering. Said events aren't associated with a window, because the OS hasn't associated them with a window - instead, they're associated with the device that produced them.

The reason I get confused with this is due to the fact that you spawn a hidden window - I can't predict such implementation from the docs, and therefore I would just think receiving events without focus is not possible.

I think I phrased my previous comment poorly - we don't actually do that right now on master. The gamepad-device-events branch does since that fixed some bugs I noticed, but it isn't a requirement. On Windows, at least, the current reason device events are delivered even without window focus is because we ask for it, but I'm not entirely sure why that decision was made. Looking back, I'll agree with you that it probably isn't the right choice, and I'd be alright with pushing for us to move away from that. I'm pretty sure that'd require changing the implementation on Windows, macOS, and X11, but the change shouldn't be too much work.

[daxpedda]

We have recently discussed on Matrix to remove DeviceEvent::Motion.
Its unclear exactly what kind of information it provides and its not clear exactly how backends should implement it in relation to WindowEvent::CursorMoved or DeviceEvent::MouseMotion, so its not consistent between them at all.

Unless some useful use-case is presented, we plan to move ahead with it. We have already decided that we don't plan to cover Gamepads in Winit and only plan to expose UUIDs for external crates to be able to do all the work.

Cc @notgull because this is basically a X11 issue only.

EDIT: thanks @kchibisov, there is no DeviceEvent::Axis 😅

[kchibisov]

I think we've discussed DeviceEvent::Motion.

[mickvangelderen]

I just ran into the lack of winit::event::DeviceEvent::Key events on MacOS / OSX. I will try to use the window events instead. It is not entirely obvious that this does not work on OSX in https://github.com/rust-windowing/winit/blob/master/FEATURES.md. Especially if one is developing on a platform where it does seem to work (Linux in my case).