This document and related code were written in 2021 and 2022 in the course of a master thesis. The thesis can be read in this repository while related code was published in the HIVE repository. A video of the working prototype which has been created for this thesis and the included qualitative study is available with this YouTube link.
Since Augmented Reality (AR) was first mentioned in the 1990s, the domain has evolved greatly in terms of available technologies and interaction possibilities. While users can view and explore three-dimensional data stereoscopically, interacting with such data in a three-dimensional space has proven to be difficult. Despite the development of many new input devices in the wake of the second wave of virtual reality, no standard has yet emerged for interacting with such objects in a mixed reality environment. Typical approaches which use controllers and gestures hold numerous drawbacks. Problems such as the gorilla-arm-effect and the lack of tactile feedback make good interactive handling in 3D space difficult.
This master thesis investigates the use of three-dimensional interactions using a touch-sensitive tablet, in order to determine if such are suitable for exploring a three-dimensional object in AR. The focus lies on the interaction possibilities, the usage of a tablet would offer, which are intended to make it easier for the user to deal with volumetric data in AR through spatial and touch-based input. This hybrid form of input offers an intuitive way to visualise modified volumetric data in the user’s real environment and to manipulate and explore it in a meaningful way through intuitively designed actions.
First, the foundation for the topic is laid by presenting related projects which also deal with interacting with 3D data. Then, different input devices are described before the interaction techniques themselves are discussed. The gathered information is then used to design a concept for an AR-based prototype using interactions based on touch-based and spatial gestures, while the panel-like shape of the tablet is used for intuitive inspection features. The design of this application is being incorporated into a corresponding prototype, which is evaluated in the course of a qualitative user study.
The focus of this study lies on how the user perceives the usage of the tablet as an input device and the possible actions which can be applied to virtual objects. The result showed that a tablet is predominantly perceived as a good and intuitive input device. In addition, the handling of touch-based and spatial gestures was understandable. When examining the data set, two-dimensional data could be stored from the three-dimensional model. All study participants agreed that the use of a marker to indicate the origin of the data was sufficient to draw a connection between 2D and 3D and to understand the information context of the two-dimensional data.