Skip to content

Commit 03721db

Browse files
Basic multi touch support (issue #279) (#306)
* translate touch events from glium to egui Unfortunately, winit does not seem to create _Touch_ events for the touch pad on my mac. Only _TouchpadPressure_ events are sent. Found some issues (like [this](rust-windowing/winit#54)), but I am not sure what they exactly mean: Sometimes, touch events are mixed with touch-to-pointer translation in the discussions. * translate touch events from web_sys to egui The are a few open topics: - egui_web currently translates touch events into pointer events. I guess this should change, such that egui itself performs this kind of conversion. - `pub fn egui_web::pos_from_touch_event` is a public function, but I would like to change the return type to an `Option`. Shouldn't this function be private, anyway? * introduce `TouchState` and `Gesture` InputState.touch was introduced with type `TouchState`, just as InputState.pointer is of type `Pointer`. The TouchState internally relies on a collection of `Gesture`s. This commit provides the first rudimentary implementation of a Gesture, but has no functionality, yet. * add method InputState::zoom() So far, the method always returns `None`, but it should work as soon as the `Zoom` gesture is implemented. * manage one `TouchState` per individual device Although quite unlikely, it is still possible to connect more than one touch device. (I have three touch pads connected to my MacBook in total, but unfortunately `winit` sends touch events for none of them.) We do not want to mix-up the touches from different devices. * implement control loop for gesture detection The basic idea is that each gesture can focus on detection logic and does not have to care (too much) about managing touch state in general. * streamline `Gesture` trait, simplifying impl's * implement first version of Zoom gesture * fix failing doctest a simple `TODO` should be enough * get rid of `Gesture`s * Provide a Zoom/Rotate window in the demo app For now, it works for two fingers only. The third finger interrupts the gesture. Bugs: - Pinching in the demo window also moves the window -> Pointer events must be ignored when touch is active - Pinching also works when doing it outside the demo window -> it would be nice to return the touch info in the `Response` of the painter allocation * fix comments and non-idiomatic code * update touch state *each frame* * change egui_demo to use *relative* touch data * support more than two fingers This commit includes an improved Demo Window for egui_demo, and a complete re-write of the gesture detection. The PR should be ready for review, soon. * cleanup code and comments for review * minor code simplifications * oops – forgot the changelog * resolve comment https://github.com/emilk/egui/pull/306/files/fee8ed83dbe715b5b70433faacfe74b59c99e4a4#r623226656 * accept suggestion #306 (comment) Co-authored-by: Emil Ernerfeldt <emil.ernerfeldt@gmail.com> * fix syntax error (dough!) * remove `dbg!` (why didnt clippy see this?) * apply suggested diffs from review * fix conversion of physical location to Pos2 * remove redundanct type `TouchAverages` * remove trailing space * avoid initial translation jump in plot demo * extend the demo so it shows off translation Co-authored-by: Emil Ernerfeldt <emil.ernerfeldt@gmail.com>
1 parent 0d71017 commit 03721db

File tree

11 files changed

+686
-15
lines changed

11 files changed

+686
-15
lines changed

CHANGELOG.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,9 @@ NOTE: [`eframe`](eframe/CHANGELOG.md), [`egui_web`](egui_web/CHANGELOG.md) and [
1515
* [Pan and zoom plots](https://github.com/emilk/egui/pull/317).
1616
* [Users can now store custom state in `egui::Memory`.](https://github.com/emilk/egui/pull/257).
1717
* Zoom input: ctrl-scroll and (on `egui_web`) trackpad-pinch gesture.
18+
* Support for raw [multi touch](https://github.com/emilk/egui/pull/306) events,
19+
enabling zoom, rotate, and more. Works with `egui_web` on mobile devices,
20+
and should work with `egui_glium` for certain touch devices/screens.
1821

1922
### Changed 🔧
2023
* Make `Memory::has_focus` public (again).

egui/src/data/input.rs

Lines changed: 61 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -96,7 +96,7 @@ impl RawInput {
9696
/// An input event generated by the integration.
9797
///
9898
/// This only covers events that egui cares about.
99-
#[derive(Clone, Debug, Eq, PartialEq)]
99+
#[derive(Clone, Debug, PartialEq)]
100100
pub enum Event {
101101
/// The integration detected a "copy" event (e.g. Cmd+C).
102102
Copy,
@@ -133,6 +133,22 @@ pub enum Event {
133133
CompositionUpdate(String),
134134
/// IME composition ended with this final result.
135135
CompositionEnd(String),
136+
137+
Touch {
138+
/// Hashed device identifier (if available; may be zero).
139+
/// Can be used to separate touches from different devices.
140+
device_id: TouchDeviceId,
141+
/// Unique identifier of a finger/pen. Value is stable from touch down
142+
/// to lift-up
143+
id: TouchId,
144+
phase: TouchPhase,
145+
/// Position of the touch (or where the touch was last detected)
146+
pos: Pos2,
147+
/// Describes how hard the touch device was pressed. May always be `0` if the platform does
148+
/// not support pressure sensitivity.
149+
/// The value is in the range from 0.0 (no pressure) to 1.0 (maximum pressure).
150+
force: f32,
151+
},
136152
}
137153

138154
/// Mouse button (or similar for touch input)
@@ -296,3 +312,47 @@ impl RawInput {
296312
.on_hover_text("key presses etc");
297313
}
298314
}
315+
316+
/// this is a `u64` as values of this kind can always be obtained by hashing
317+
#[derive(Clone, Copy, Debug, Eq, PartialEq, PartialOrd, Ord)]
318+
pub struct TouchDeviceId(pub u64);
319+
320+
/// Unique identifiction of a touch occurence (finger or pen or ...).
321+
/// A Touch ID is valid until the finger is lifted.
322+
/// A new ID is used for the next touch.
323+
#[derive(Clone, Copy, Debug, Eq, PartialEq, PartialOrd, Ord)]
324+
pub struct TouchId(pub u64);
325+
326+
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
327+
pub enum TouchPhase {
328+
/// User just placed a touch point on the touch surface
329+
Start,
330+
/// User moves a touch point along the surface. This event is also sent when
331+
/// any attributes (position, force, ...) of the touch point change.
332+
Move,
333+
/// User lifted the finger or pen from the surface, or slid off the edge of
334+
/// the surface
335+
End,
336+
/// Touch operation has been disrupted by something (various reasons are possible,
337+
/// maybe a pop-up alert or any other kind of interruption which may not have
338+
/// been intended by the user)
339+
Cancel,
340+
}
341+
342+
impl From<u64> for TouchId {
343+
fn from(id: u64) -> Self {
344+
Self(id)
345+
}
346+
}
347+
348+
impl From<i32> for TouchId {
349+
fn from(id: i32) -> Self {
350+
Self(id as u64)
351+
}
352+
}
353+
354+
impl From<u32> for TouchId {
355+
fn from(id: u32) -> Self {
356+
Self(id as u64)
357+
}
358+
}

egui/src/input_state.rs

Lines changed: 77 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,12 @@
1+
mod touch_state;
2+
13
use crate::data::input::*;
24
use crate::{emath::*, util::History};
3-
use std::collections::HashSet;
5+
use std::collections::{BTreeMap, HashSet};
46

57
pub use crate::data::input::Key;
8+
pub use touch_state::MultiTouchInfo;
9+
use touch_state::TouchState;
610

711
/// If the pointer moves more than this, it is no longer a click (but maybe a drag)
812
const MAX_CLICK_DIST: f32 = 6.0; // TODO: move to settings
@@ -15,9 +19,13 @@ pub struct InputState {
1519
/// The raw input we got this frame from the backend.
1620
pub raw: RawInput,
1721

18-
/// State of the mouse or touch.
22+
/// State of the mouse or simple touch gestures which can be mapped to mouse operations.
1923
pub pointer: PointerState,
2024

25+
/// State of touches, except those covered by PointerState (like clicks and drags).
26+
/// (We keep a separate `TouchState` for each encountered touch device.)
27+
touch_states: BTreeMap<TouchDeviceId, TouchState>,
28+
2129
/// How many pixels the user scrolled.
2230
pub scroll_delta: Vec2,
2331

@@ -55,6 +63,7 @@ impl Default for InputState {
5563
Self {
5664
raw: Default::default(),
5765
pointer: Default::default(),
66+
touch_states: Default::default(),
5867
scroll_delta: Default::default(),
5968
screen_rect: Rect::from_min_size(Default::default(), vec2(10_000.0, 10_000.0)),
6069
pixels_per_point: 1.0,
@@ -70,7 +79,7 @@ impl Default for InputState {
7079

7180
impl InputState {
7281
#[must_use]
73-
pub fn begin_frame(self, new: RawInput) -> InputState {
82+
pub fn begin_frame(mut self, new: RawInput) -> InputState {
7483
#![allow(deprecated)] // for screen_size
7584

7685
let time = new
@@ -84,6 +93,10 @@ impl InputState {
8493
self.screen_rect
8594
}
8695
});
96+
self.create_touch_states_for_new_devices(&new.events);
97+
for touch_state in self.touch_states.values_mut() {
98+
touch_state.begin_frame(time, &new, self.pointer.interact_pos);
99+
}
87100
let pointer = self.pointer.begin_frame(time, &new);
88101
let mut keys_down = self.keys_down;
89102
for event in &new.events {
@@ -97,6 +110,7 @@ impl InputState {
97110
}
98111
InputState {
99112
pointer,
113+
touch_states: self.touch_states,
100114
scroll_delta: new.scroll_delta,
101115
screen_rect,
102116
pixels_per_point: new.pixels_per_point.unwrap_or(self.pixels_per_point),
@@ -121,7 +135,13 @@ impl InputState {
121135
/// * `zoom > 1`: pinch spread
122136
#[inline(always)]
123137
pub fn zoom_delta(&self) -> f32 {
124-
self.raw.zoom_delta
138+
// If a multi touch gesture is detected, it measures the exact and linear proportions of
139+
// the distances of the finger tips. It is therefore potentially more accurate than
140+
// `raw.zoom_delta` which is based on the `ctrl-scroll` event which, in turn, may be
141+
// synthesized from an original touch gesture.
142+
self.multi_touch()
143+
.map(|touch| touch.zoom_delta)
144+
.unwrap_or(self.raw.zoom_delta)
125145
}
126146

127147
pub fn wants_repaint(&self) -> bool {
@@ -188,6 +208,52 @@ impl InputState {
188208
// TODO: multiply by ~3 for touch inputs because fingers are fat
189209
self.physical_pixel_size()
190210
}
211+
212+
/// Returns details about the currently ongoing multi-touch gesture, if any. Note that this
213+
/// method returns `None` for single-touch gestures (click, drag, …).
214+
///
215+
/// ```
216+
/// # use egui::emath::Rot2;
217+
/// # let ui = &mut egui::Ui::__test();
218+
/// let mut zoom = 1.0; // no zoom
219+
/// let mut rotation = 0.0; // no rotation
220+
/// if let Some(multi_touch) = ui.input().multi_touch() {
221+
/// zoom *= multi_touch.zoom_delta;
222+
/// rotation += multi_touch.rotation_delta;
223+
/// }
224+
/// let transform = zoom * Rot2::from_angle(rotation);
225+
/// ```
226+
///
227+
/// By far not all touch devices are supported, and the details depend on the `egui`
228+
/// integration backend you are using. `egui_web` supports multi touch for most mobile
229+
/// devices, but not for a `Trackpad` on `MacOS`, for example. The backend has to be able to
230+
/// capture native touch events, but many browsers seem to pass such events only for touch
231+
/// _screens_, but not touch _pads._
232+
///
233+
/// Refer to [`MultiTouchInfo`] for details about the touch information available.
234+
///
235+
/// Consider using `zoom_delta()` instead of `MultiTouchInfo::zoom_delta` as the former
236+
/// delivers a synthetic zoom factor based on ctrl-scroll events, as a fallback.
237+
pub fn multi_touch(&self) -> Option<MultiTouchInfo> {
238+
// In case of multiple touch devices simply pick the touch_state of the first active device
239+
if let Some(touch_state) = self.touch_states.values().find(|t| t.is_active()) {
240+
touch_state.info()
241+
} else {
242+
None
243+
}
244+
}
245+
246+
/// Scans `events` for device IDs of touch devices we have not seen before,
247+
/// and creates a new `TouchState` for each such device.
248+
fn create_touch_states_for_new_devices(&mut self, events: &[Event]) {
249+
for event in events {
250+
if let Event::Touch { device_id, .. } = event {
251+
self.touch_states
252+
.entry(*device_id)
253+
.or_insert_with(|| TouchState::new(*device_id));
254+
}
255+
}
256+
}
191257
}
192258

193259
// ----------------------------------------------------------------------------
@@ -517,6 +583,7 @@ impl InputState {
517583
let Self {
518584
raw,
519585
pointer,
586+
touch_states,
520587
scroll_delta,
521588
screen_rect,
522589
pixels_per_point,
@@ -537,6 +604,12 @@ impl InputState {
537604
pointer.ui(ui);
538605
});
539606

607+
for (device_id, touch_state) in touch_states {
608+
ui.collapsing(format!("Touch State [device {}]", device_id.0), |ui| {
609+
touch_state.ui(ui)
610+
});
611+
}
612+
540613
ui.label(format!("scroll_delta: {:?} points", scroll_delta));
541614
ui.label(format!("screen_rect: {:?} points", screen_rect));
542615
ui.label(format!(

0 commit comments

Comments
 (0)