Multimodal Emotion Classification in Virtual Reality: An RMS-Optimized RNN Approach using ECG, GSR, and Eye-Tracking Signals

Emotion classification through physiological signals is an emerging field dedicated to objectively evaluating human emotional states by analyzing biological data. Physiological signals provide a more reliable means of emotion detection than traditional techniques that rely on speech, body language, or facial expressions because they capture involuntary reactions that are strongly associated with emotional events. This research aims to develop a sophisticated emotion classification system utilizing multimodal physiological signals, specifically Electrocardiogram (ECG), Eye Tracking data, and Galvanic Skin Response (GSR). The VREED dataset provided the physiological markers that were employed in this analysis. This dataset uses virtual settings based on immersive 360-degree videos to evoke emotions. In comparison to current state-of-the-art techniques, the proposed RMS optimized RNN model performs more effectively, including CNN and standard RNN, as measured by F1-score, recall, accuracy, and precision. The proposed model successfully demonstrated its superiority in emotion classification tasks by achieving an impressive accuracy of 94%, which was 3% higher than CNN and RNN when tested on the VREED dataset. This approach is designed to enhance the reliability and effectiveness of emotion recognition systems, contributing to advancements in psychological wellness, customized user experiences, and other emotion-sensitive applications.