Sanity Check for Shapley values-based explanations of Deep Neural Networks Predictions

Explanation methods have been introduced to un-cover the black-box of Deep Neural Networks (DNNs). The interpretability of the prediction is crucial in medicine, where the use of black-box algorithms has an impact on human life. An important class of explanation methods is based on the Shapley values. Since no ground truth explanation is available, assessing the explanation faithfulness is difficult. In this paper, a comparison among Shapley values-based explanations is performed, and a sanity check of their faithfulness from the literature is applied. The results show that these methods pass the check and they can be suited to explain DNN in healthcare scenarios.