Assurance of Machine Learning/TinyML in Safety-Critical Domains

Machine Learning (ML) is being applied in a variety of systems, with outcomes that have been shown to outperform human ability under certain conditions [1] . In sectors such as healthcare [2] , automotive [3] , and manufacturing [4] , systems often feature high degrees of autonomy and safety, which causes them to be categorized as "safety-critical" (or, equivalently, "high assurance"). The appeal of adopting ML in safety-critical application domains is rising steadily in anticipation of the "intelligence bonus" expected of deploying ML-boosted applications operationally. Obviously, this drive has also caused much attention into the question of the level of assurance that can be achieved for such systems after they incorporate such additional components. Ways to reap the anticipated potential of ML and, more generally, of Artificial Intelligence (AI), are being explored also in materials science, robotics, and numerous other engineering systems. Application domains that are rich of field data (as in manufacturing, testing, and service) and that pursue multi-objective optimizations are especially suited for earning boosting from ML/AI data-driven algorithms. For the integration of data intensive ML algorithms, the aerospace industry provides a variety of unique opportunities and challenges. The influence of data science on the aerospace industry will be felt across the whole supply chain and development process, including in testing and assessment [5] . Owing to the computationally and storage intensive nature of current state-of-the-art implementations of ML, all data acquired from the field are transferred to the cloud for processing, which obviously incurs massive latency that renders the objective untenable for latency-sensitive embedded applications. TinyML is an emerging effort that integrates ML into embedded Internet of Things (IoT) devices with the potential to revolutionize aerospace industry and several other application domains with similar needs and characteristics. TinyML combines hardware and software fit for resource-scarce computation with the aim to enable ML models (particularly Deep Learning, DL, algorithms) on compact, relatively cheap, and power-efficient devices. Verifying function and performance assurances and confirming that the system meets certification and legal standards are all part of the testing process for a new product. One of the most important goals of contemporary data-driven flight testing is to increase aircraft safety and reliability while lowering testing time and cost in aerospace [6] . Modern IoT technology may combine several data sources to make informed and persists due on airside support jobs that require human intervention. Such data integration can result in considerable cost reductions, allowing human resources to be deployed more effectively and efficiently. Humans and ML-guided computers will need to interact in a safe, efficient, and effective manner even in a completely autonomous system scenario, which shall have to contemplate failure situations and assure awareness of human activity. For ML-based systems (MLS), quality indicators include performance measures such as hit and false alarm rates, as well as an assessment of the goodness (fitness for purpose) of the data and methods used to train and test the system. MLS have a data-driven algorithmic behavior that radically differs from the control-driven nature of their traditional predecessors, which the software verification body of knowledge rests upon. This particular trait of them puts the effectiveness of traditional verification approaches applied to MLS, including testing, into question, which in turn causes serious concern to those considering use in high-assurance application domains. While there is little doubt that the emerging technology of TinyML could bring several attractive benefits to aerospace, assurance challenges and barriers need to be overcome to improve and optimize maintenance and airside support.