Runtime Assurance for Safety-Critical Systems: An Introduction to Safety Filtering Approaches for Complex Control Systems

More than three miles above the Arizona desert, an F-16 student pilot experienced a gravity-induced loss of consciousness, passing out while turning at nearly 9Gs (nine times the force of gravity), flying over 400 kn (over 460 mi/h). With its pilot unconscious, the aircraft turn devolved into a dive, dropping from over 17,000 ft to lower than 8,000 ft in altitude in less than 10 s. An auditory warning in the cockpit called out to the pilot “altitude, altitude” just before he crossed through 11,000 ft, switching to a command to “pull up” around 8,000 ft. Meanwhile, the student’s instructor was watching the event unfold from his own aircraft. As the student’s aircraft passed through 12,500 ft, the instructor called over the radio “two recover,” commanding the student (“two”) to end the dive. As the student’s aircraft passed through 11,000 ft, the instructor’s “two recover!” came with increased urgency. At 9,000 ft, and with terror rising in his voice, the instructor yelled “TWO RECOVER!” Fortunately, at the same time as the instructor’s third panicked radio call, a new runtime assurance (RTA) system kicked in to automatically recover the aircraft. The Automatic Ground Collision Avoidance System (Auto GCAS), an RTA system integrated on the jets fewer than two years earlier, in fall 2014, detected that the aircraft was about to collide, commanded a roll to wings level and pull-up maneuver, and recovered the aircraft fewer than 3,000 ft above the ground. The event described here occurred in May 2016. A video from the event was declassified and publicly released in September 2016, and the footage can be found at
[1]
. While Auto GCAS monitored the behavior of a safety-critical cyberphysical system with a ­human ­providing the primary control functions, the same concept is gaining attention in the autonomy community looking to assure safety while integrating complex and intelligent control system designs.