Where Are We and Where Can We Go on the Road to Reliance-Aware Explainable User Interfaces?

With the widespread use of machine learning for algorithmic decision-making, Explainable Artificial Intelligence (XAI) systems are increasingly important. However, they are not always understandable and trustworthy for end-users, who should know when to rely on AI’s advice to make informed decisions. Hence, adequately presenting explanations in user interfaces (UIs) is essential. This paper investigates proposals in this direction and what is missing to support the design of explainable UIs aware of reliance issues. From a systematic literature review of 1,287 unique papers, we identified 120 secondary studies, of which we selected 22 for analysis. Our findings reveal that studies have been conducted to provide recommendations to specific application domains, and evidence regarding explanation effects on reliance remains inconclusive. We also found a lack of characterization of factors impacting reliance on XAI systems. Furthermore, we provide perspectives to foster appropriate reliance research on explainable UIs.