Recognizing emotions from videos by studying facial expressions, body postures and hand gestures

A system for recognizing emotions from videos by studying facial expressions, hand gestures and body postures is presented. A stochastic context-free grammar (SCFG) containing 8 combinations of hand gestures and body postures for each emotion is used and we show that increasing the number of combinations in SCFG improves the system's generalization for new hand gesture and body posture combinations. We show that hand gestures and body postures contribute to improving the emotion recognition rate with up to 5% for Anger, Sadness and Fear compared to the standard facial emotion recognition system, while for Happiness, Surprise and Disgust no significant improvement was noticed.