Handwritten Chinese Character Recognition in Ancient Books Based on Improved YOLOv7

In recent years, deep learning-based methods have achieved great success in handwritten Chinese character recognition, however, not many studies have been conducted on handwritten Chinese character recognition for rare characters in the Chinese classics, which are difficult to be inputted by computers due to the large number of strokes and the more complex features with unknown readings. In this paper, we construct a sample dataset containing 290 Chinese character categories, totaling 71277 labeled samples, by extracting the rare characters in the Guofeng chapter of the Book of Songs. Compared with the CASIA-HWDB dataset, our self-constructed dataset are more targeted, focusing on the rare characters that are uncommon daily, and they can be used as a supplement to this dataset. In order to recognize handwritten Chinese characters, previous methods usually use the Convolutional Neural Network (CNN) model, but the model is still lacking in data augmentation and balancing, model lightweight, as well as robustness and generalization ability. On the contrary, this paper proposes to recognize difficult-to-detect characters based on the improved YOLOv7 model, and the experimental results show that the classification accuracy of the improved YOLOv7 model is 100%, the recall is 99.89%, and the mAP can reach 99.6%, and the recognition accuracy can reach up to 99% in a single character, and based on ensuring the accuracy of recognition, it can also quickly and efficiently recognize multiple Chinese characters of different categories at the same time. Meanwhile, the YOLOv7 model has a faster detection speed compared with traditional Convolutional Neural Network models through the design features of single forward inference, unified detection framework, and compact model structure, which can be deployed in the Chinese character recognition system to help people eliminate reading barriers and experience the deep historical flavor and unique literary charm of classical literature in an immersive way. The performance of Resnet18, YOLOv5, YOLOv7, improved YOLOv7, and YOLOv8 in handwriting difficult-to-detect word recognition is compared experimentally. The results show that the improved model improves the accuracy by 6.32 percentage points and recall by 5.95 percentage points compared to the Resnet18 model, the detection speed on GPU is improved by 9.65 f/s, the mAP value is improved by 0.1 percentage points compared to the original YOLOv7 model, and the number of parameters and FLOPs is reduced by 47.87% and 44.90% respectively.