Real-Time Mapping for Teleoperation Systems in VR of Unmanned Aerial Vehicles

This study presents the development of virtual environments as control centers for remote teleoperation tasks of unmanned aerial vehicles (UAV s), Initially, our focus lies on reconstructing outdoor environments using a ZED mini stereo camera and reconstructing them through point cloud techniques. The virtual environment is hosted in UNITY, a widely recognized platform for designing virtual reality (VR) video games. Within this environment, a digital twin UAV is embedded, tasked with replicating the real positions and orientations of the vehicle. To achieve this, we propose using PID and PD control for the positions and rotations of the virtual vehicle, allowing it to follow the desired position, in this case, the real position of the vehicle. A series of experiments were conducted in outdoor and indoor environments to validate the functionality of this approach.