Improving Explainability for Single-Channel EEG Deep Learning Classifiers via Interpretable Filters and Activation Analysis*

Deep learning methods are increasingly being applied to raw electroencephalography (EEG) data. Relative to traditional machine learning methods, deep learning methods can increase model performance through automated feature extraction. Nevertheless, they are also less explainable. Existing raw EEG explainability methods identify relative feature importance but do not identify how features relate to model predictions. In this study, we combine well-characterized first layer filters with a novel post hoc statistical analysis of the filter activations that linearly relates properties of model activations with predictions. We implement our approach within the context of automated sleep stage classification, finding that the model uncovers positive relationships between waveforms resembling sleep spindles and NREM2, between high frequency filters and Awake samples, and between δ activity and NREM3. Our approach represents a significant step forward for raw EEG explainability and has the potential to provide many insights relating learned features and model predictions in future studies.