Continuous Emotion Ambiguity Prediction: Modeling with Beta Distributions

Conventional continuous emotion prediction systems are typically trained to predict the ‘average’ of affect ratings obtained from multiple human annotators. These systems, however, ignore the ambiguity inherent in the perceived emotions, which is not captured by the ‘average rating’. This paper presents a novel ambiguity-aware continuous emotion prediction system that predicts the time-varying emotion state as a series of beta distributions. Our recent work has shown beta distributions to be an effective parametric model of a collection of affect ratings. This work develops an appropriate cost function that enables neural networks to be trained to predict beta distributions. It also investigates the choice of parameterization of the beta distribution, the choice of activation functions of the output layer, and the tractability of gradient definitions in combination with the loss function. The proposed framework is implemented using a Bag-of-Audio-Words front-end and an LSTM-based back-end and evaluated on the RECOLA dataset. In addition to comparison with baseline systems that only predict the ‘average rating’, the effectiveness with which the predictions represent ambiguity in perceived emotions is also evaluated. Experimental results reveal that the proposed approach outperforms other ambiguity-aware systems, especially when predicting valence.