Reality fusion is a high performance and robust immersive robot teleoperation system that combines the best of both worlds: the high fidelity of neural rendering (3D Gaussian Splattings) and real-time stereoscopic point cloud projection.
- Reality Fusion Unity Project: Documentations for the
RealityFusionUnityUnity project and source code for VR robot control applciations. - Native Render Plugin: Instruction for compiling the original 3DGS Cmake project for a Unity native render plugin. Apre-compiled DLL files already avaliable in
Assest\Plugins\x86x64in the Unity project. The source code for the native renderer is in the3DGSUnityPluginfolder. - Reality Fusion Robot Setup: Documentation for setting up the robot, including guides in installing the required packages on Jetson TX2.
- Robot Startup Guide: Documentation for starting the modified turtlebot for teleoperation, after following the Robot Setup.
Link to paper: Reality Fusion: Robust Real-time Immersive Mobile Robot Teleoperation with Volumetric Visual Data Fusion
@INPROCEEDINGS{10802431,
author={Li, Ke and Bacher, Reinhard and Schmidt, Susanne and Leemans, Wim and Steinicke, Frank},
booktitle={2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
title={Reality Fusion: Robust Real-time Immersive Mobile Robot Teleoperation with Volumetric Visual Data Fusion},
year={2024},
volume={},
number={},
pages={8982-8989},
keywords={Visualization;Three-dimensional displays;Telepresence;Robot control;Virtual reality;Robot sensing systems;Rendering (computer graphics);Spatial resolution;Streams;Research and development},
doi={10.1109/IROS58592.2024.10802431}}
Contact: keli95566@gmail.com
This work was supported by DASHH (Data Science in Hamburg - HELMHOLTZ Graduate School for the Structure of Matter) with the Grant-No. HIDSS-0002.
Please check out INRIA's original license regarding the original implementation of 3DGS.