Safety Validation for Deep Reinforcement Learning Based Aircraft Separation Assurance with Adaptive Stress Testing

Ensuring safe separation of aircraft has become a major challenge given the growing demand for air transportation. Recently, deep reinforcement learning (DRL) has been successfully applied to develop aircraft separation assurance systems. Though showing promising performance, the DRL systems are trained as black boxes with no safety guarantees. Given the catastrophic impact of potential failures, validations of these systems become critical. However, existing validation approaches focus on system failures caused by single factors and only work on the cases with one validated system, which greatly limits their use in real-world high-density air traffic where numerous validated systems exist and multiple factors cause the failures together. In this paper, we introduce a multi-agent adaptive stress testing formulation for the system validation task and propose a Multi-Agent Hybrid Adaptive Stress Testing (MAHAST) framework to comprehensively validate DRL-based aircraft separation assurance systems given high-density air traffic by detecting system failures caused by unsafe system dynamics and improper environment properties. Moreover, we introduce a novel hybrid adaptive stress testing approach with a hierarchical policy, detecting the failures caused by combinations of multiple factors. We conduct extensive numerical experiments in a real-time air traffic environment. Results empirically show that MAHAST can effectively detect diverse failures in DRL-based aircraft separation assurance systems with high-density air traffic.