Mutual information-based selection of audiovisual affective features to predict instantaneous emotional state

Automatic prediction of continuous level emotional state requires selection of suitable affective features to develop a regression system based on supervised machine learning. This paper investigates the performance of low-level dynamic features for predicting two common dimensions of emotional state, namely, valence and arousal instantaneously. Low-complexity features are extracted from audio and visual modalities independently and fused in the feature level. Features with minimum redundancy and maximum relevancy are chosen by using the mutual information-based selection process. The performance of frame-by-frame prediction of emotional state using the moderate length features as proposed in this paper is evaluated on spontaneous and naturalistic human-human conversation of SEMAINE database. Experimental results show that the proposed features selected by mutual information can be used for instantaneous prediction of emotional state with an accuracy higher than traditional audio or visual features that are used for affective computation.