Understanding Multilingual Four-Scene Comics with Deep Learning Methods

Artificial intelligence has been developed tremendously in the last decades and is applied in many fields nowadays. On the other hand, understanding human creations is one of the most important subject concerning AI. Comic computing is a branch of computing dealing with comics in engineering. In this field, multimodal data with natural languages and pictures are used. Although tasks such as comic frame recognition in comic computing are performed using image processing techniques, artificial intelligence focuses on pictures in comics rather than the comic content to solve these tasks. To make models understand the contents in comics, we should deal with natural languages in comics in the form of character words. We used a dataset which was suitable for analysing comic contents and set up tasks for the analysis of four-scene comics. However, the current dataset was only in Japanese, therefore we have proposed an English version of the dataset in this paper. In order to extend our research to foreign countries, we consider that having a multilingual dataset is required. We did a comparison study between the Japanese version and the English version by the means of sentimental analysis. We demonstrated and analysed the results to determine the feasibility of computers understanding comics. From the results, we could remark that it was possible to obtain a good accuracy rate with our multilingual dataset. As a matter of fact, it was possible to show results similar to the ones from the original Japanese version. Hence, our research can be extended to multiple languages without affecting the output.