In-Device Feedback in Immersive Head-Mounted Displays for Distance Perception During Teleoperation of Unmanned Ground Vehicles

In recent years, Virtual Reality (VR) Head-Mounted Displays (HMD) have been used to provide an immersive, first-person view in real-time for the remote-control of Unmanned Ground Vehicles (UGV). One critical issue is that it is challenging to perceive the distance of obstacles surrounding the vehicle from 2D views in the HMD, which deteriorates the control of UGV. Conventional distance indicators used in HMD take up screen space which leads clutter on the display and can further reduce situation awareness of the physical environment. To address the issue, in this paper we propose off-screen in-device feedback using vibro-tactile and/or light-visual cues to provide real-time distance information for the remote control of UGV. Results from a study show a significantly better performance with either feedback type, reduced workload and improved usability in a driving task that requires continuous perception of the distance between the UGV and its environmental objects or obstacles. Our findings show a solid case for in-device vibro-tactile and/or light-visual feedback to support remote operation of UGVs that highly relies on distance perception of objects.

I. Introduction

Interest in improving real-time remote human-robot interaction is growing rapidly. First-person view (FPV) using VR for unmanned ground vehicle (UGV) with remote or (semi-) automatic control is increasingly used for search and rescue operations, in disaster recovery, and for terrain and object surveillance, especially in unsafe environments [1], [2]. Immersive VR displays for UGV teleoperation can improve the user's concentration and performance in obstacles avoidance tasks when compared to a normal display such as desktop monitor [3]. UGV can explore the surrounding environment and send information captured via sensors or cameras to remotely located users in real-time. However, most cameras attached to these robots have limitations, such as low degrees of freedom, narrow field-of-view, and poor photo-sensitivity, especially in dark and complex environments with interference from the objects or obstacles found in such environments.