Multi Modal Genre Classification of Movies

With the emergence of platforms like Netftix which keep users engaged through their sophisticated recommendation algorithms, it is of growing importance to accurately predict the genre a movie belongs to, in order to push the right content to viewers. Until recently, content creators found their customers by communicating the plot of a movie through expressive posters or by publishing a short overview of the movie. For an algorithm to try and replicate the perceptions drawn by humans to predict which movies a consumer will be interested in, it is necessary to treat this as a multi-label problem and attribute multiple genres to a single movie. The principal aim of this paper is to reliably predict the different genres under which a movie can be classified, and further check whether a combination of poster and overview of a movie can provide more accurate results than a poster and overview used individually. We evaluate this task by performing a series of experiments that compare and contrast the importance of both, textual features from movie overviews and visual features from the poster. We propose that the ensemble use of both these features using a ResNet50 model and GloVe embeddings, outperforms conventional models that use these features individually, with an average F1-Score of 0.674.