LODENet: A Holistic Approach to Offline Handwritten Chinese and Japanese Text Line Recognition

One of the biggest obstacles in Chinese and Japanese text line recognition is how to present their enormous character sets. The most common solution is to merely choose and represent a small subset of characters using one-hot encoding. However, such an approach is costly to describe huge character sets, and ignores their semantic relationships. Recent studies have attempted to utilize different encoding methods, but they struggle to build a bijection mapping. In this work, we propose a novel encoding method, called LOgographic DEComposition encoding (LODEC), that can efficiently perform a 1-to-1 mapping for all Chinese and Japanese characters. As such, LODEC enables to encode over 21,000 Chinese and Japanese characters by 520 fundamental elements. Moreover, to handle the vast style variety of handwritten texts in the two languages, we propose a novel deep learning (DL) architecture, called LODENet, together with an end-to-end training scheme, that leverages auxiliary ground truths generated by LODEC or other radical-based encoding methods. We systematically performed experiments on both Chinese and Japanese datasets, and found that our approach surpassed the performance of state-of-the-art baselines. Furthermore, empirical evidence shows that our method can gain significantly improvement using synthesized text line images without the need for domain knowledge.