Artificial Intelligence System for Emotion Recognition and Text Analytics

Harnessing the power of emotional intelligence by analyzing a person’s behavioral and linguistic skills can help humans improve their approach to social interactions. In this paper, we propose an artificial intelligence-based stand-alone system that will allow us to classify and analyze facial expressions in real-time and perform sentiment analysis by examining the body of the text (extracted from audio) to understand the opinion expressed by it. This helps us provide a deeper understanding of how humans really feel at a given time. The proposed system uses a deep neural network (DNN) for classifying eight basic emotions based on features extracted from facial expressions and uses pretrained sentiment analysis tools to quantify text (extracted from audio) based on polarity. The system is implemented by extracting the audio and visual cues from real-time scenarios and using these extracted cues to perform facial expression recognition and text sentiment analysis. The integrated system extracts video and audio simultaneously with a frame rate of 4-5 fps. The facial emotion detection system successfully detects facial expressions of faces detected in real-time video with an accuracy of about 86.75%. The speech extracted is converted to text, cleaned, and processed to determine if the attitude of the speaker in a given situation is positive, negative, or neutral.