Enhancing Cognitive Assistants with Low-Cost Computer Vision

From 2010 to 2017, the third-leading cause of fatal general aviation (GA) accidents was System Component Failure-Powerplant (SCF-PP) [1]. The General Aviation Joint Steering Committee SCF-PP Working Group identified several safety enhancements that should be pursued to reduce the risk of fatality during SCF-PP emergencies, including smart-cockpit technology that helps pilots identify and respond to emergency situations [2]. Previous iterations of smart-cockpit technology-such as The MITRE Corporation's (MITRE's) Digital Copilot concept [3] and offerings from companies such as ForeFlight, Garmin, X-Avionics, and Hilton Software-do not have features to proactively assist single pilots in identifying SCF-PP emergencies. One reason is that engine performance data are typically unavailable to applications running on external devices. Any assistance provided by these existing applications during power-loss emergencies must be manually requested by the pilot. Because built-in data output from instrument panels is rare in GA aircraft, there is an opportunity to leverage devices that are otherwise idle during flight. For example, a pilot's smartphone could be used to gather additional flight data for tactical safety algorithms that could independently identify SCF-PP emergencies using computer-vision techniques. Using devices typically present in the cockpit eliminates the need for a pilot to purchase dedicated camera and computing hardware, which can lower the barrier to entry for additional safety benefit. This paper provides an overview of a mobile application that makes engine-performance data available to other applications in a GA cockpit using computer vision. In addition to an overview of the concept, the paper contains an evaluation of a tactical safety algorithm using these data, as well as plans for further development.