Human Role in Failure of Autonomous Systems: A Human Reliability Perspective

Autonomous capabilities are increasingly applied to a variety of systems. These systems may operate with different Levels of Autonomy (LoAs). Systems designed with a LoA such that they can operate without human supervision or intervention are not expected to be launched in the near or intermediate future. Despite the expected dependence on human actions for a safe operation, the impact of possible interactions between the system and the human operator from a Human Reliability Analysis (HRA) perspective has not yet been explored. Given that (i) humans are involved in autonomous systems operations, and (ii) humans can fail and contribute to an accident, the autonomous systems field could benefit from HRA for achieving a safer operationMost of the HRA methods have been developed for Nuclear Power Plants operation (NPPs). Autonomous systems operations significantly differ from NPPs. The modifications of human-system interaction as a consequence of a higher LoA can present a challenge for current HRA methods. Autonomous systems include unique failure modes and causes of human failures, and these factors may not be correctly modelled and quantified through current HRA methods.Decreasing the chance of human failure in autonomous systems operation, identified through an HRA, is thus crucial for safety. As the LoA increases, operators will assume more of a monitoring role and can be the ultimate safety barrier if the autonomous system fails. The path for producing credible and robust HRA for autonomous systems operations is long. This paper proposes some high-level requirements, based on a discussion on the human role in failures of autonomous systems, Human Factors literature on these systems, and insights from how HRA methods respond to digital enhancements of NPPs control rooms.