A Language-Independent Speech Sentiment Analysis Using Prosodic Features

Sentiment Analysis from audios is a challenging field of ongoing research because, though humans can recognize emotions from facial expressions, gestures, and tone of the ongoing conversation, it can be a challenging task for machines. In the following report and its corresponding research work, an audio emotion detection system has been proposed and is used to perform a comparative study to detect emotions from 3 datasets. In this research, a comprehensive study of diverse datasets (TESS and RAVDESS) along with a custom dataset is made. Our proposed model aggregates results from diverse baseline machine learning models trained on different parameters and hyperparameters and its performance is calculated and compared with existing research. The proposed model uses different features of the audio such as MFCC, Mel Spectrogram, and Chroma which are extracted from the datasets which make our model independent of language barriers. The efficacy of this model is evaluated using various evaluation metrics such as confusion matrix, overall accuracy, and F1-score. As for the outcome of the research and experiment, the overall accuracy is 99.46% and 89.62% for TESS and RAVDESS respectively. Furthermore, an accuracy of 78.28% has been reported for the custom dataset. It is also found that, the most predictable emotion is anger while the most misclassified is fear.