Evaluating multimodal strategies for multi-label movie genre classification

We present the evaluation of different multimodal strategies to classify movies genres in a multi-label scenario. We used a dataset consisting of posters, synopses, subtitles, and trailers taken from 10,594 movies. For this purpose, we first built deep networks for each media. Then these classifiers were combined using two deep models and three late fusion strategies. Despite its simplicity, the late fusion strategies achieved better performance than the deep models. We obtained 0.649 of F-Score and 0.626 of AUC-PR using fusion by the sum rule. Furthermore, it was possible to notice the complementarity among the modalities and the improvement in performance obtained by combining them.