Statistical Conformance Checking of Aviation Cyber-Physical Systems by Mining Physics Guided Models

Aviation systems such as aircrafts and Unmanned aerial vehicles (UAV) are cyber-physical systems (CPS) with several interacting automated control modules. The operational behavior of such CPS may change during deployment due to several reasons including: a) control system software bugs, b) environmental changes such as weather, and c) mechanical faults such as faulty sensor. Changes in operational characteristics can be exceptionally dangerous in civilian applications or when UAVs are used for military purposes or targeted precision tasks such as maneuvering in hostile environment. Moreover, safety violations are often not caused by a single fault but rather by a combination of faults that can potentially be avoided if the individual faults were pro-actively identified and mitigated. In this paper, we advocate for continuous verification of the operational characteristics to check its compliance with the specification. This compliance can be specified using Signal Temporal Logic (STL) on the output trajectories. However, unknown errors can cause operational changes that have little effect on the output characteristics but may manifest with fatal consequences when combined with other supervisory control systems. As such the problem of identifying such underlying changes is a difficult task specially when their impact on output characteristics is minimal. We propose a novel framework for analyzing the stochastic conformance of operational output characteristics of safety-critical CPS in terms of the coefficients of a physics guided surrogate model (PGSM) of the aerodynamics of the aviation CPS. We propose model guided hybrid recurrent neural networks (MGH-RNN) to mine a PGSM which is used to check the model conformance using STL on the model coefficients. The PGSM can also identify operational changes due to unknown errors. Additionally, we also derive a tractable input space partitioning mechanism based on PGSM characteristics to perform stochastic conformance checking by utilizing the conformal inference technique. We apply our technique to identify errors in pitch control system of aircrafts or UAVs. We have considered mechanical errors such as faulty Angle of Attack sensors which can affect the operation of supervisory pitch correction systems such as MCAS. This error identification can lead to detection of faults in advance and prevention of activation of controllers that rely on faulty sensors.