Pinch and other Gestures

[hmaarrfk]

Describe the bug

Some of my users complained about "not being able to zoom". We couldn't figure it out until one of them tried to do so on my laptop, and we noticed they were trying to pinch on the trackpad to zoom.

Are these kinds of gestured captured in the WGPU/PyGFX stack? I tried to search of "pinch" in the codebase and couldn't find anything.

To Reproduce

Truthfully, its a little difficult for me to test this since I am linux and "pinch" is really only "supported" by "wayland" and xwayland is used to build the wgpu canvas.

Observed behavior

Users accustomed to apple's zoom in "gestures" try to pinch to zoom with 2 fingers on the trackpad with the PanZoomController on PyGFX and they say "well that doesn't work" and don't zoom into the images.

Your environment

 $ mamba list | grep -E "(wgpu|pyside|pygfx)"
pygfx                     0.2.0              pyhd8ed1ab_0    conda-forge
pyside6                   6.7.0           py310h7a3fa89_1    conda-forge
wgpu-native               0.19.3.1             h2b8f863_0    conda-forge
wgpu-py                   0.15.2             pyha804496_0    conda-forge

██ system:

             platform:  Linux-6.8.0-31-generic-x86_64-with-glibc2.39
python_implementation:  CPython
               python:  3.10.13

██ versions:

wgpu:  0.15.1

Generally running Wayland + PySide6

[hmaarrfk]

(probably not a bug, but a feature request or just general brainstorming)

[Korijn]

I believe our canvases emit pointer events and each event includes a pointer id. This id would be different for each finger. With that you should be able to implement pinch gesture (and others). Therefore I would think this is already supported sufficiently at the level of wgpu-py.

If your expectation is for pygfx controllers to support this, you would have to update those to handle multi pointer gesture events.

[hmaarrfk]

hmm interesting, I tried it on OSX and Wayland, and the event.pointer_id always reports 0.

to be clear I mean "pinch to zoom on the touchpad"

https://support.apple.com/en-us/102482

ps. no "expectations" for me, just trying to understand how one would go about to see if I can make it a pygfx wide change.

[almarklein]

Two-finger up-down scrolling seems to work though. Pinching does not seem to generate any events. This code is helpful to test this sort of thing: https://github.com/pygfx/wgpu-py/blob/main/examples/events.py).

[almarklein]

GLFW does not have pinch support, it seems glfw/glfw#90

For Qt seems to be viable to add support for pinch events. I'm having a stab.

[Korijn]

I'm going to guess that it does already work (even on mac and wayland) in jupyter rfb. It is going to be different for each combination of GUI backend and OS

[almarklein]

I put what I have now in pygfx/wgpu-py#515. Seems doable, although support will be limited to certain gui backends. And needs work in pygfx to support these events. Note that in addition to scaling, the pinch gesture also has rotation, which can be useful to e.g. manipulate objects in the scene.