Adaptive Navigation for Lunar Surface Operations Using Deep Learning and Holographic Telepresence

Current solutions for navigation and guidance on the lunar South pole have limited visual spatial cues to help guide astronaut crewmembers to mission objectives. Challenges include low solar elevation angles that create areas of high contrast, permanently shadowed regions (PSR), and a lack of radionavigation systems such as the Global Positioning System (GPS) on Earth. This paper proposes a novel and adaptive navigation system through the Space Communications, Operations, and User Telepresence (SCOUT) Augmented Reality (AR) assistant to support astronaut navigation on the Moon. SCOUT is an artificial intelligence (AI) system that can be integrated into an astronaut's visor to collaboratively identify points of interest, detect hazards, and guide a group of crewmembers towards mission waypoints using hologram overlays. SCOUT uses deep learning to iteratively improve its under-standing of the nearby environment, and can be trained using data from robotic systems deployed on the Moon before astro-nauts arrive. To assist an astronaut's navigation capability in extreme environments, SCOUT displays 3D spline holograms towards waypoints and objectives. For example, SCOUT can display a clear label during an extra-vehicular activity (EVA), such as a switch that should be flipped, or which direction they should walk. To effectively navigate through and explore any region of the lunar surface, SCOUT has been trained on a multitude of randomly generated Moon-like virtual environ-ments. Through this process, this AI system can quickly identify features that it has seen before as well as adapt to new situations it has never experienced before. Results show that SCOUT can map out an optimal path towards a waypoint, even without having previous data samples from that specific environment. SCOUT is designed to benefit the user as much as possible, through automated guidance and continuous learning of the surrounding environment.