Key Information Extraction on Vietnamese Book Cover Images

Information extraction from images through OCR-based approach has been recently of high interest. In this paper, we implemented the task of extracting relevant information from text on Vietnamese book covers using Vi-BCI dataset consisting of over 7000 labeled book cover images. We constructed several different models according to four main approaches: text classification, text-box classification, named entity recognition, and combination model. We evaluated the models of each approach separately before comparing these approaches by selecting the model with the best result of each approach and evaluating these models on the Character Error Rate (CER) metric. All the proposed approaches have good results, with the named entity recognition approach achieving the highest overall prediction result, at 0.0973 with the Bi-LSTM model. In addition, we also propose a simple and effective method to arrange the extracted text from the image in the correct order. Finally, we discussed and analyzed the results for further improvement and proposed directions for future work.