Towards International Standards for the Evaluation of Artificial Intelligence for Health

Healthcare can benefit considerably from advanced information processing technologies, in particular from machine learning (ML) and artificial intelligence (AI). However, the health domain only hesitantly adopts these powerful but complex innovations so far, because any technical fault can affect people's health, privacy, and consequently their entire lives. In this paper, we substantiate that international standards are required for thoroughly validating AI solutions for health, by benchmarking their performance. These standards might ultimately create well-founded trust in those AI solutions that have provided conclusive evidence to be accurate, effective and reliable. We give reasons that standardized benchmarking of AI solutions for health is a necessary complement of established assessment procedures. In particular, we demonstrate that it is beneficial to tackle this topic on a global scale and summarize the achievements of the first year of the ITU/WHO focus group on “AI for Health” that has tasked itself to work towards creating these evaluation standards.