Comic Speaker Prediction Based on Visual Relations and Natural Language Processing Tasks

Automatic analysis and processing of comics has received increasing attention. Speaker prediction is a challenging task among the related tasks about comics. Previous methods focused on fixed rules based on human experience or combining multiple artificial features. In this work, we review previous methods, refer to relevant tasks, and propose a method that applies visual and linguistic information. The proposed method contains a visual information model and two replaceable language models. Unlike previous methods, it attempts to use fewer fixed rules or artificial features, and its language models do not require extra training data when predicting new books. To train these models, we merge existing datasets and apply different detection tasks. Experimental results show that the proposed method’s accuracy is 77.3%. It can perform better than most of the previous methods and shows the importance of combining linguistic information.