Emotion Recognition from Contextualized Speech Representations using Fine-tuned Transformers

In the last years, speech emotion recognition has represented an active research topic due to its large range of applications, from human-computer interaction to the diagnosis and analysis of behavioral changes during clinical treatments. Initial approaches targeting statistical models have constantly evolved into complex models based on the recent findings in the Deep Learning domain. Deep neural network models that achieved state-of-the-art performance in numerous speech recognition tasks, showed their potential in solving emotion recognition tasks. In this paper, we show that fine-tuning a Wav2Vec2.0 base model significantly outperforms other approaches for the prediction of natural emotions. The transformer-based neural network composing the Wav2Vec2.0 base model was initially trained for the speech-to-text task. The proposed approach is validated on the RAVDESS dataset, which is a publicly available dataset with eight discrete emotion categories, and we report 80.98 % overall accuracy using a 5-folds cross-validation procedure.