Enhancing Explainability in Affective Signal Processing with Interpretable Machine Learning

Machine Learning (ML) has achieved remarkable success, particularly in affective computing applications such as automatic pain estimation and mental health assessment. However, most ML frameworks function as black-box models, offering high performance but lacking interpretability. Traditional methods for interpretable affective signal processing, often relying on post-hoc techniques or map-based visualizations, tend to compromise either explainability or performance. Our recent work introduces concept-based explanations in affective signal processing, using Facial Expression Recognition (FER) as a benchmark application. Concept-based learning enables the model to understand both the classification label and the underlying intermediate concepts that contribute to predictions. Preliminary experiments have shown that this framework provides robust explanations while maintaining performance compared to the state-of-the-art black-box models. Despite promising results, there remain research gaps in defining explainability, enhancing conceptual explanations, and improving concept-based frameworks for generalizability. To address these problems, this paper outlines a two-stage research plan including refining concept-based FER and extending concept-based learning to broader affective signal processing applications.