Bimodal Physiological Emotion Recognition Using Information Contrastive Learning

Recently, several methods have been developed to enhance the performance of bimodal physiological emotion recognition. However, these methods have overlooked the need to identify common features related to emotions and to address the modality discrepancy. To address these issues, we propose a novel method called information contrastive learning (InfoCon), which extracts common features across different modalities and ensures that the extracted common features are highly related to emotions, thereby reducing the modality gap. We have compared the performance of InfoCon with state-of-the-art approaches on two public datasets, SEED-IV (EEG, eye movement) and MPED (EEG, GSR, RSP, ECG). Our results have shown that InfoCon achieves the best recognition accuracy of 92.86% on the SEED-IV dataset and promising results on the MPED dataset for all three protocols. Furthermore, we have visualized the features before and after the InfoCon transformation on the SEED-IV dataset and observed that the transformed features are more discriminative between different emotions. Overall, our proposed method provides a new approach for bimodal physiological emotion recognition by identifying common features and reducing the modality gap.