Ensemble of Domain Adversarial Neural Networks for Speech Emotion Recognition

Domain-invariant class-discriminative representation across multiple emotional speech corpora is essential for multi-corpus speech emotion recognition. When training a model using multiple corpora, the generalization ability must be considered by hindering the memorization of domain-specific information, and recognition should be performed on features that cannot discriminate among domains. First, the performance degradation of multi-corpus speech emotion recognition systems was investigated, and a novel frame-work adopting an ensemble of several domain adversarial neural networks was proposed. Experiments were conducted on single-corpus and multi-corpus systems for two corpora, namely an English corpus (IEMOCAP) and a Japanese corpus (JTES). The findings of the experiments revealed that although individual domain adversarial neural networks exhibit information loss and inferior performance, an excellent comeback accuracy can be achieved by an ensemble of these individual domain adversarial neural networks. We confirmed that the proposed framework effectively learned both domain-invariant and class-discriminative representations.