Certification Considerations for Adaptive Stress Testing of Airborne Software

Adaptive Stress Testing (AST) has shown promise in identifying errant corner cases in complex software used in aerospace applications including Flight Management Systems (FMS). The strength of AST is performing test-based validation and verification of complex aerospace software-intensive systems at scale in simulated operational environments. Simulating and capturing the realistic operational complexities in integrated verification environments may expose flaws in the software prior to field deployment, whereas the software may perform just fine to traditional requirements-based unit and component-level testing. AST can be used to test code components or the entire system. Individual components may behave safely, but together can result in complex interactions and emergent failures, so it is important to test at the integrated system level. Motivated by the observed benefits at the prototype proof-of-concept scale, this paper considers how AST may be integrated into a production workflow and used to generate objective evidence in a process that delivers certified aerospace software. The research includes evaluation of alignment with both RTCA DO-178C and Overarching Properties (OP). The paper addresses questions such as "where should AST fit in the software development lifecycle, what aspects of AST do not fit, and what objectives does it satisfy?" The paper concludes that AST is useful at locating errors in complex airborne application software and in doing so provides benefits to manufacturers and end users. Furthermore, AST appears to align with in both DO-178C-based and Overarching Properties-based certification approaches.