Deep Convolutional Neural Network Based Hidden Markov Model for Offline Handwritten Chinese Text Recognition

Recently, an effective segmentation-free approach via deep neural network based hidden Markov model (DNN-HMM) was proposed and successfully applied to offline handwritten Chinese text recognition. In this study, to further improve the modeling capability, we adopt deep convolutional neural networks (DCNN) to calculate the HMM state posteriors. First, on the frame basis, the DCNN-HMM can automatically learn the features from the raw image of the handwritten text line via the convolutional architecture rather than the handcrafted gradient features using in the DNN-HMM. Second, we examine several important factors of DCNN to the recognition performance, namely the kernel size, the number of blocks and convolutional layers. We also improve the language modeling by using more text data and high-order N-gram. Tested on ICDAR 2013 competition task of CASIA-HWDB database, the proposed DCNN-HMM could achieve a character error rate (CER) of 4.07%, yielding a relative CER reduction of 30.8\% over the DNN-HMM approach. To the best of our knowledge, this is the best published result of the segmentation-free approaches. Furthermore, we explain why DCNN-HMM is more effective than DNN-HMM via the visualization of feature learning and the error pattern analysis.