An Emotionally Aware Friend: Moving Towards Artificial General Intelligence

Mental health is a leading cause of deaths, affecting over 450 million people globally. There are existing emotion recognition models to help and understand the state of a person but mainly via text. The proposed model in the paper is developed in a personalized multi-modal architecture to incorporate all the necessary aspects to predict the cumulative emotional status of a person by his/her text context, speech features, and facial expressions. There are mainly 3 different models: Bidirectional Encoder Representations from Transformers, Multi-layer Perceptron Classifier and Convolutional Neural Network working together in synchronization to cater to the need. Along with it, the advancement implemented includes General Adversarial Networks (GAN), to generate a human entity and help the human to cope up with their emotional state and practically save them from any kind of grave danger. The model with the help of GAN and lip-synced model manages to converse with the user after analyzing and considering their mental state, helping them to find a solution accordingly.