Systems Modeling of Trust in AI-Enabled Medical Diagnosis

The progression of artificial intelligence, particularly in healthcare domain, has brought significant progress to treatment, drug discovery, disease diagnosis and more. The main contribution of this paper is the development of a scenario-based preferences risk register framework for quantifying AI related risks in disease diagnosis. Two case studies, one involving physician and the other one with involving patients as the main stakeholders, are compared to evaluate the effectiveness of the framework and to evaluate how the initiatives ranking orders will evolve based on the most and least disruptive scenarios for each cases. The framework is applied to realistic case studies on cardiac sarcoidosis, identifying success criteria, initiatives, emergent conditions, and the most and least disruptive scenarios. The success criteria align with the National Institute of Standards and Technology AI Risk Management Framework (NIST AI RMF) seven trustworthy AI characteristics. Finally, the framework identifies the most and least disruptive scenarios and the most important initiatives for both cases. Results show that Non-Interpretable AI and Lack of Human-AI Communications, Privacy Attacks, and Human Errors in Design, Develop, Measurement and Implementation are the most disruptive scenarios in both cases, however, scenario Cyber Security Threats also counted as other most disruptive scenarios to the initiatives ranked by patients. This adaptable framework has the potential not only to transfer findings to global healthcare systems but also to extend its effectiveness to various other domains, such as transportation, finance, environment, and design.