A Publicly Available Multimodal Emotional Dataset Triggered by Both Videos and Odors

This study investigates multimodal emotion signal recognition by using odors as a stimulus material to explore the influence of the odor on human emotions. It is conducted in the context of traditional electroencephalography and electrooculography fusion for emotion recognition. We designed an experimental paradigm involving visual, auditory and olfactory senses, adding odors to traditional audiovisual medias to enhance emotion elicitation, and built a multimodal emotion dataset. The dataset contains electroencephalography and electrooculography data obtained from 15 subjects in video-odor stimulation and video-only stimulation experiments. 90 video clips and 10 odors were selected as stimulus materials. Besides, the Transformer classifier was adopted to classify the three categories of emotion, and give the average classification accuracy for the different features. Under differential entropy features, the multimodal average classification accuracy under video-odor stimulation was 3% higher than that under video-only stimulation. The results indicated that olfactory stimuli can be combined with video stimuli to enhance multimodal emotion recognition ability.