Empirically Identified Gaps in a Situation Awareness Model for Human-Machine Coordination

Autonomous systems are a new frontier pushing socio-technical advancement. Such systems will be required to team with humans. Consequently, the ability to coordinate with teammates is critical. We have developed and empirically evaluated an autonomous synthetic teammate (AST) designed to operate in a task in which it receives information from a visual data display and chat messages from human teammates. Teams with the AST performed as well on most performance measures as teams without it. Further, the AST performed its piloting task well. Human participants performed their tasks as well with the AST piloting the system as they did with a human pilot. Nonetheless, we observed issues that show there remains room for improving human-AST coordination. These issues illuminate limitations in the AST's situation representation and point to directions for further improvement and future research.