DroCo – V2

Branch DroCoV2 presents a completely new and clean version 2.0 of the original DroCo, currently still in development. Tested on Unity version 2022.3.23.

What's new

• Possibility to switch to AR.
• Synchronized video stream with flight data.
• New map base layer – ArcGIS.
• New GUI design.

DroCo – Multi-Drone Control Vizualization Tool

DroCo (VSTool) is a tool for effective drone remote control using mixed reality, that also supports communication and cooperation on a mission with multiple drones. The proposed solution is developed by Robo@FIT, Brno University of Technology research group, and is inspired by the high mental load of the pilot in the control of the drone, especially in the performance of more complex missions (multiple drones, remote target, proximity to infrastructure etc.). The system is based on the extension of the 3D virtual model with real data (augmented virtuality). It uses temporal and spatial registration of:

1. off-line data (map data, elevation data, 3D building models)
2. on-line data (video-stream, reconstructed 3D structures, location information, flight data)
3. virtual control objects (navigation points and directions, spatial areas and geo-fences, position of other drones, distance to nearby objects, preview map, or view from other drones).

The system thus allows you to pilot the drone in FPV (first-person-view), but at any time it can switch to TPV (third --person-view), so that one can look around freely in a situation with poor orientation, further directs the pilot to other mission objectives, points out close objects or other drones, etc. The system is currently being expanded with the functions of multiple drones, sharing more sensory information across the system, increasing the security of network communication. The development also aims to use a system for drone control training for pilots, increase the realism of drone behavior in simulated mode, more efficient mission management and visualization of the status for the operator of the whole event.

Installation

• Install UnxUtils to be able to patch ArcGIS scripts using patch_arcgis.bat script.
• Clone this repo:

  git clone git@github.com:robofit/drone_vstool.git

• Get submodules:

  cd drone_vstool
  git submodule update --init

• Create a symlink of the submodules to the Assets folder:

  .\scripts\link_submodules.bat

• Download multimedia files from LFS:

  git lfs install
  git lfs pull

• Patch ArcGIS scripts:

  .\scripts\patch_arcgis.bat

Setup ArcGIS

• Create ArcGIS developer account and create your API Key.
• Paste the API Key to ProjectSettings -> ArcGIS Maps SDK -> API Key.

Setup GStreamer (optional, not required)

• Install GStreamer 1.20.1 – install both, regular and devel version based on your computer's architecture (msvc and x86_64 works for me).
• Add gstreamer binary folder path to System Environment Variables – Computer -> System properties -> Advanced System Settings -> Advanced Tab -> Environment Variables... -> System Variables -> Variable: Path -> Edit -> New -> C:\gstreamer\1.0\msvc_x86_64\bin
• Create new system variable – New Variable: GST_SDK_PATH= C:\gstreamer\1.0\x86_64\
• If GStreamer is still not working inside Unity, try to install or reinstall the latest MSVC redistributable libraries.

Publications

• HUBINÁK, Róbert. Application for Efficient Drone Control Using Augmented Virtuality. Brno, 2020. Bachelor's thesis. Brno University of Technology, Faculty of Information Technology. Supervised by Beran Vítězslav.
• SEDLMAJER Kamil, BAMBUŠEK Daniel a BERAN Vítězslav. Effective Remote Drone Control Using Augmented Virtuality. In: Proceedings of the 3rd International Conference on Computer-Human Interaction Research and Applications 2019. Vienna: SciTePress - Science and Technology Publications, 2019, s. 177-182. ISBN 978-989-758-376-6.
• SEDLMAJER, Kamil. User interface for drone control using augmented virtuality. Brno, 2019. Master's Thesis. Brno University of Technology, Faculty of Information Technology. 2019-06-14. Supervised by Beran Vítězslav.