Emotion recognition from audiovisual information

We report preliminary results on emotion recognition by machine from joint audiovisual input of facial video and speech. The results show potential advantages in using both modalities over either modality alone. The recognition rate for audio alone is about 75% and video alone about 70%. Using audiovisual data we achieved 97% without increasing the number of features. The improvement in performance is accredited to the complementary property between the two modalities. A possible application is in natural human-computer interfaces.