A Deep Learning Technique for Emotion Recognition Using Face and Voice Features

Automatic emotion recognition is an emerging application in the field of artificial intelligence. Where the use of technology helps to curb various mental illnesses like anxiety, depression, schizophrenia etc. So, there is a need to create efficient artificial intelligent devices. In this experimental project, we developed a multimodal system using neural networks (CNN) for recognizing emotions from facial expressions and voice. We used an RGB images dataset for facial expression recognition. For voice emotion recognition, an audio recorded files dataset was used. We presented phases of automatic emotion recognition in the form of data acquisition, data analysis and data visualization. Comparative analysis among recent papers, features affecting the performance and proposed model in terms of accuracy have also been given in tabular form. The proposed model shows accuracy of 73% and 90.91% for face and voice features respectively. It's observed that variations in the factors of the feature-training phase of neural networks like epochs, batch sizes, training datasets, also affect the results.