Reliability Analysis of Architectural Safeguards for AI-enabled Systems

Although enormous progress has been made in Artificial Intelligence (AI), it entails new challenges. The growing complexity of learning tasks requires more complex AI models, which increasingly exhibit unreliable behaviour. This is of particular concern in safety-critical systems where AI is commonly used. Therefore, well-known architectural approaches have been adopted such as N-Version Programming or Simplex Architectures (denoted as Architectural Safeguards) to deal with the unreliable nature of AI. At design-time, however, it is difficult to identify an architectural safeguard that satisfies the reliability (or other quality) requirements of the system. In this paper, we present a model-based reliability analysis of AI-enabled systems to assess architectural safeguards at design-time taking into account the predictive uncertainty of AI components. We have validated our approach in a case study from the field of autonomous driving. Our results show that our approach not only enables the analysis of the impact of architectural safeguards on the overall reliability of an AI system but supports software engineers in decision-making.