DeepTunes - Music Generation based on Facial Emotions using Deep Learning

Approximately one among seven people in the world experience some form of mental health disorder. Listening to music has been found to substantially improve mental health via stress-reducing effects. A 2009 meta-analysis found that music-assisted relaxation can also improve the quality of sleep in patients with sleep disorders. In this paper we present DeepTunes, a system to generate music with lyrics, according to the detected emotion of the user. The user needs to submit a photo of himself for the Facial Recognition model to determine the user’s mood and provide some additional textual input describing how the user feels. The lyric generation model takes into consideration the mood of the user as determined by the Facial Recognition model and generates the lyrics accordingly with the first line provided by the user. Background music will also be generated that best suits the user’s mood. Deep Learning Techniques have been employed for Facial Recognition, Lyrics Generation and Music Generation models. For the facial recognition model, a Convolutional Neural Network (CNN) has been implemented, GPT-2 has been used to generate lyrics and stacked LSTM networks have been constructed to generate music. The goal of our study is to produce music and lyrics that generate a positive emotive response from the user, given any mood.