3D Interactive Model of HERA to support ECLSS anomaly resolution using a Virtual Assistant

In future long-duration exploration missions (LDEMs), communication with mission control will be significantly delayed. Crews will often need to react to time-sensitive issues or hazards without relying on mission control for assistance. The need for a higher degree of crew autonomy without direct oversight from mission control introduces the implementation of virtual assistants (VAs) to aid the crew; however, we must first create standards and guidelines for VAs in this context. For this purpose, we have developed a VA called Daphne-AT (Anomaly Treatment) to investigate the interaction between astronauts and virtual assistants in the context of anomaly resolution related to the Environmental Control and Life Support System (ECLSS). A series of experiments will be conducted in a laboratory environment and at the Human Exploration Research Analog (HERA) at NASA Johnson Space Center (JSC) to study user's performance, situational awareness, cognitive workload, and trust in Daphne-AT. Subjects in a simulated LDEM will be given several ECLSS anomalies to identify and solve with and without the support of Daphne-AT. We based the scenarios we developed on existing ECLSS hardware and pre-existing anomaly resolution procedures from the HERA environment. Solving an anomaly requires the subject to complete the correct anomaly resolution procedure(s) for the appropriate anomaly, which could include various tasks such as swapping out component parts and activating or deactivating ECLSS systems. In experiment sessions without Daphne-AT, subjects will rely on telemetry feeds as well as background knowledge and training to solve anomalies. However, during experiment sessions with Daphne-AT, the VA will also assist subjects in detecting and diagnosing these anomalies. We conducted an initial set of experiments at Texas A&M University (TAMU) prior to those at HERA. However, the subject must complete anomaly resolution procedures during an experiment that require access to hardware and components not available at the TAMU location. To emulate the HERA ECLSS hardware elements in our laboratory at TAMU, we recreated a virtual 3D representation of HERA and its ECLSS systems using the game engine Unreal Engine 4 (UE4). This virtual model will act as an analog for subjects who are tested on TAMU's campus to complete relevant ECLSS procedures. All subsystems featured in the UE4 model are interactive and allow the user to perform steps in an anomaly resolution procedure similar to how the user would do it in HERA. The UE4 model allows the user to complete more than 20 anomaly resolution procedures, thus increasing the TAMU experiment's fidelity compared to those conducted at HERA. This paper describes the development of the UE4 model and how we used it to validate the experimental protocol implemented at HERA. The results of these experiments will inform future guidelines for VAs deployed on LDEMs.