Analytical Science for Autonomy Evaluation

Current directions in autonomous systems focus on collecting large amounts of data to verify, validate, test, and evaluate system operations. For multidomain and uncertain scenarios, data sampling may not be adequate to fully explore and represent the entire trade space for verification and validation (V&V). However, leveraging methods from test and evaluation (T&E), a hierarchy of analytics can be developed so as to narrow the trade space, while the opportunity cost of the remaining space is a risk-mitigated deployment strategy. Issues in V&V/T&E employ statistics, but could benefit from theoretical analytics, such as the ability to augment data for testing using simulated models or define tests to minimize operational risk. The use of modeling is not new; however, as analytics of artificial intelligence and machine learning (AI/ML) are designed to exploit data; then these methods are independent of the data developed from the first-principles physics models. The paper highlights the need for methods of analytical science for autonomy evaluation and presents three examples in structural, situation, and cyber awareness.