Heatmaps for Visual Explainability of CNN-Based Predictions for Multivariate Time Series with Application to Healthcare

The need for explainable AI is becoming increasingly important for critical decision domains such as healthcare for example. In this context, this paper is concerned with explaining the predictions of Convolutional Neural Networks (CNNs) with particular focus on multivariate Time Series (TS) problems. The approach is based on heatmaps as a visual means to highlight the significant variables over the temporal sequence. Furthermore, a channel-wise CNN architecture is implemented to allow for considering the TS variables separately. The approach is applied to the problem of predicting the risk of in-hospital mortality. We use a dataset from the MIMIC-III database, which included about 13K ICU records. The experimental results demonstrate rational insights that conform with the common medical knowledge in this regard.