Handwritten Character String Recognition Using Transformer and CNN Features

While character recognition systems applications with restrictions on writing conditions, such as zip code recognition and ledger sheet recognition, are in practical use, the accuracy of free handwritten character string recognition is still low and there are many challenges. Japanese free handwritten character strings are difficult to segment because some Japanese characters consist of multiple connected components and there are no spaces between words. Therefore, in this study, we propose a method for handwritten character string recognition method that does not require character segmentation using Transformer and CNN features. Transformer-based models usually require a large amount of annotated training data. However, our method can be trained with a small amount of annotated training data. Comparative experiments confirm the effectiveness of the proposed method by achieving CER=O.127 while CRNN [1] and TrOCR [2] achieve CER=O.562 and CER=O.174, respectively. The results show that the proposed method is effective for Japanese character string recognition, where the variety of character strings is large and it is difficult to secure a sufficient number of training samples for each character string.