Automatic Speech Semantic Recognition and verification in Air Traffic Control

The Federal Aviation Administration (FAA) and BrainVentions Corporation, Inc. have conducted a one-year cooperative research agreement to investigate the current state of the art Automatic Speech Semantic Recognition (ASSR) technology, to measure its accuracy, and to explore its potential applications in Air Traffic Control (ATC). The ASSR system used for this study was the ValsVox (BrainVentions) web-based Software as a Service (SaaS) application for real-time speech recognition and semantic parsing of ATC voice communications. Case studies were conducted using the enhanced ValsVox system with recorded audio from previously conducted ATC Human-in-the-Loop (HITL) simulations. The standard speech recognition metric of word error rate was calculated as well as semantic metrics based on the commands issued by the simulation pilots used as the “gold standard” interpretation of the voice clearances. The design of ValsVox is such that the speech recognition is constrained in real-time by the semantic parsing unlike other systems in which a transcript of the utterance is first generated then semantically parsed. For the phrases that were successfully recognized, the accuracy of the semantic recognition as measured against the “gold standard” was over 80%. These findings suggest that the technology may be ready for follow on research to incorporate ASSR into ATC decision support tools. For example, this might include tools that detect mismatches between verbal clearances and keyed commands or to alert the users of incorrect or missing read-backs. There is also a potential to use ASSR in simulation and training environments such as incorporating into simulation pilot workstations to assist overloaded pilots. In addition human transcriptionists could use ValsVox-generated transcripts to save time and costs when complete accuracy is needed.