Emotion Detection Using Machine Learning, ECG Signals and Facial Features

This research study introduces a machine learning classifier for distinguishing 3 distinct emotional states using electrocardiogram (ECG) and facial features. A data pre-processing phase is performed by com bining the emotional data from the DREAMER (2015) dataset and the relatively newer DERCFF (2023) (Database for Emotion Recognition through Cardiac-Facial Features). Subsequently, the study leverages the power of the Discrete Cosine Transform to extract essential features, optimizing the model for enhanced accuracy. The DERCFF dataset plays a pi votal role in the study's approach as it provides a unique set of features extracted from facial and cardiac signals. Convolutional Neural Networks (CNNs) are employed to train upon these extracted facial images, facilitating a comprehensive analysis. The objective of this study is to discern the authenticity of the displayed emotions, ultimately enhancing the overall performance of the model. The target emotions consist of happiness, tension (anger and sadness) and calmness, with LightGBM, Bagged Decision Trees ensemble, Linear S VM and Gaussian Naive Bayes methods serving as the classifiers. This study's combined models (ECG and Facial Features) achieve a notable accuracy rate of 93.80/0, underscoring its efficacy in accurately identifying emotions from multimodal cardiac signals. This research contributes to the field of affective computing, with potential applications in mental health monitoring, human-computer interaction, and emotion-aware technologies.