Stochastic Dynamic Agent-based Modelling for Evaluation of ACAS and DAA Systems

Validation studies for airborne collision avoidance systems (ACAS) and detect-and-avoid (DAA) systems have strongly relied on fast-time simulation of large sets of encounters, while the impact of sensor errors and (remote) pilot performance variability have been considered to a limited extent only. This paper shows the need of ACAS/DAA validation approaches for feedback to design and approval that account for extended variability in sensor input and human performance. Stochastic dynamic agent-based models are presented that describe the variability of interacting agents in encounter-scenarios. Monte Carlo (MC) simulation results are provided for encounters of manned aircraft equipped with TCAS II or ACAS Xa and for unmanned aircraft equipped with ACAS Xu. The results show that the nonlinear dynamics and stochastic influences in the sociotechnical ACAS/DAA systems can critically affect the aircraft manoeuvres, and the safety and efficiency of the operations. Pilot performance has the largest impact on the overall performance, which is much larger than the impact due to differences between TCAS II and ACAS Xa. Conventional estimates of near mid-air collision (NMAC) probabilities are often lower than the estimates achieved using MC simulation of agent-based models with sensor errors. In remotely piloted aircraft systems (RPAS), strictly following the remain-well-clear guidance of ACAS Xu can lead to livelock conditions, where the RPAS cannot effectively pass each other. It is concluded that stochastic dynamic models describing all elements of the sociotechnical systems encompassing the ACAS/DAA system are essential for reliable validation. This is especially important in the design of artificial intelligence-based systems like ACAS Xa/Xu, where there exists a close connection between evaluation of the performance of the overall system and tuning of meta-parameters of the optimization process.