Emotion-Aware Music Recommendation using CNN, BERT, and DQN for Real-Time Personalization

The focus of this project is to work on a recommendation engine that learns and responds to the emotional needs of the listener. The conventional music recommender has been enduring for a very long time now, with the integration of AI in recent days. These algorithms learn the user preferences over time and analyze their listening patterns to follow up on the recommendation the next time they start with a listening session. To address this problem of suggesting songs based on emotions we concentrate on the emotion analysis of the listener. There are several ways and methods that can be used to detect the emotion of the listener, but the one easily available factor that can be read and analyzed is the facial expressions. Algorithms like Convolutional Neural Network (CNN) can be used to extract the different features of facial emotions expressed by the contraction and relaxation of different facial muscles. Bidirectional Encoder Representations from Transformers (BERT) and Mel-Frequency Cepstral Coefficients (MFCC) to categorize the songs into different emotions that aligns with the previously read facial emotions [happy, sad, neutral, angry, disgust]. The Pygame library of Python can then be used to combine the results obtained from the algorithms mentioned above to play the song that aligns with the emotion of the listener. To ensure that the algorithm functions well, and the user is satisfied with the recommendations, Deep Q Networks can be implemented so that the user preference are analyzed and when found that the song that is being played does not satisfy the users’ needs or is disliked by the user, the system will understand the uncertain decision made by the system must be avoided in the future. Since the system not only understands the emotions of the listener but also if the listener is satisfied with the recommendations of the system, it can alter its recommendation preferences over time, making it more personalized and appealing to the user.