Elicitation-Based Curriculum Learning for Improving Speech Emotion Recognition

Speech emotion recognition (SER) is an essential component of human-computer interaction systems, enabling machines to understand and respond to human emotions. To our knowledge, this is the first study that explicitly proposes emotion elicitation information to perform curriculum learning (CL) in SER tasks by incorporating both acted and spontaneous data. We utilize IEMOCAP and BAUM-1 databases that comprise both acted and spontaneous speech samples. We use wav2vec 2.0 transformer layer extracted representations as features and train them on ECAPA-TDNN classifier. We train separate SER models using acted and spontaneous training data and perform same-elicitation and cross-elicitation performances as curriculum generator tasks. Following that, our approach involves categorizing the data into easy (acted) and hard (spontaneous) training subsets. We first train the SER model with the easy subset after that, sequentially feed the hard subset using batch sampling and learning rate scaling. By employing CL, the model gradually adapts to the increased difficulty and complexity, leading to improved robustness against elicitation variations across the evaluation data. Our experimental results show that the proposed CL approach improves SER performance by 2.77% and 3.41% for the IEMOCAP and BAUM-1 databases, respectively, compared to systems trained without CL.