Robustness of Explainable AI Algorithms for Disease Biomarker Discovery from Functional Connectivity Datasets

Machine learning (ML) models have been used in functional neuroimaging for wide-ranging tasks, ranging from disease diagnosis to disease prognosis. There have been successive functional connectivity-based ML studies focused on improving model performances for disease detection. An increasing number of such studies use their trained models to detect and evaluate salient features that could be potential biomarkers of these neuro-logical conditions. The evaluation of these salient features is often qualitative and limited to cross-referencing existing literature for similar findings. In this study, we present objective quantitative metrics to evaluate the robustness of these salient features. Building upon existing generic evaluation metrics, we propose metrics that capture topological properties known to be characteristic of brain functional connectomes. Using existing and newly proposed measures on a set of baselines and state-of-the-art graph neural networks (GNN) models, we found that when GNNExplainer is used with models that incorporate attention, the scores produced are relatively more robust than other combinations. On datasets of patients with Autism Spectrum Disorder (ASD) or Attention-deficit Hyperactivity Disorder (ADHD), our proposed metrics highlighted that salient features identified in both disorders are highly involved in functional specialization, while salient ASD features expressed stronger functional integration than ADHD. We package these existing and novel metrics together in the RE-CONFIRM framework that holds promise to set the foundations for the quantitative evaluation of salient features detected by future studies.