Predictive XR Telepresence for Robotic Operations in Space

This paper proposes a new method for robotic teleoperation scenarios with significant time delay utilizing Extended Reality (XR) technologies, specifically for Lunar base assembly and maintenance. Traditional teleoperation interfaces interpret commands sent through a desktop computer screen and are limited in providing spatial awareness cues. Additionally, the software often has difficulty in accounting for a time delay or providing intuitive techniques for human-robot interaction (HRI). This paper presents a new XR interface for robotic teleoperation using a predictive 3D simulated virtual environment developed using the SpaceCRAFT platform. SpaceCRAFT is a multiuser Virtual Reality (VR) and Augmented Reality (AR) systems engineering toolbox software that allows for worldwide collaboration on design and test of space systems. SpaceCRAFT provides the operator a virtual interface to the robot represented through three different states, a Virtual Command State (VCS), Predicted Current State (PCS), and Time Delayed State (TDS). To improve situational awareness and the PCS representation of the robot, SpaceCRAFT processes LIDAR and RGB camera data to create a constantly updating 3D point cloud representation of the remote environment. An experiment involving teleoperation of a 3D printed robotic manipulator was run, with a simulated time delay similar to that between the Earth and the Moon. Results demonstrate that the predictive approach provides an effective and flexible method of robot teleoperation that helps achieve successful task completion, even with a large time delay.