An analysis of large speech models-based representations for speech emotion recognition

Large speech models-derived features have recently shown increased performance over signal-based features across multiple downstream tasks, even when the networks are not fine-tuned towards the target task. In this paper we show the results of an analysis of several signal- and neural models-derived features for speech emotion recognition. We use pretrained models and explore their inherent potential abstractions of emotions. Simple classification methods are used so as to not interfere or add knowledge to the task. We show that, even without finetuning, some of these large neural speech models’ representations can enclose information that enables performances close to, and even beyond state-of-the-art results across six standard speech emotion recognition datasets.