Multimodal decomposition for enhanced subtle emotion recognition

True human emotions may be revealed during brief moments of minute facial muscle movements. These so-called subtle emotions are low in intensity, sparse and short duration in nature. Despite such tiny presence, subtle emotions have shown to be useful for real-life applications including lie detection and clinical diagnosis. In this paper, we present a processing technique based on multimodal decomposition and emotion magnification that is viable to decompose and subsequently emphasize highly discriminative subtle emotions information. Through this method, promising results of 25-30% improvement in recognition performance are achieved. By further magnifying the decomposed emotion components, the subtle emotions intensity can be increased and further boost recognition performance by 3-4%.