Decoding the Past: Solving Challenging Oracle Bone Characters Recognition Problem by Integrating Vision Transformer and Generative Adversarial Image Restoration Technique

Oracle Bone Characters (OBC), one of the earliest forms of writing in China, are crucial for understanding ancient religious practices and daily lives. Inscribed on animal bones and shells, these artifacts, known as Oracle Bone Scripts (OBS), offer pivotal historical and cultural insights. Yet, the identification of OBC poses substantial challenges due to factors such as Cracks, Weathering, Similarities and the Sheer Volume of Characters. Therefore, we built a model including three parts to help archaeologists improve the efficiency of Oracle exploration. Firstly, specific to Ligature situation, an automatic OBS detection system (OBS-DET) was developed, helping archaeologists extracting every OBC accurately. This model is trained on Oracle Bone Inscription Detection Dataset downloaded from YinQi WenYuan and run on RTX3090Ti, achieving the AP of 47.3% with the FPS of 53.4. Moreover, we built a set of generative adversarial based image restoration system (OBC-Trans-GAN) to transcribing scanned version OBC into clearer Hand-Printed version. OBC-Trans-GAN trained on Oracle-241 dataset can generalize to complex oracle recovery scenarios fine, further simplify the OBC recognition task. Finally, to solve Long-Tailed Problem and Open-Set Recognition Problem, we designed a transformer-based recognition model (OBC-VIT). Trained on the HUST-OBS dataset, OBC-VIT can achieve the accuracy of 97.2 %, the highest among the existing models. This project is made open-source on: https://github.com/zhz5687/Transformer-OBS-Recognition. We also develop an APP using the above AI models to assist archaeologists which was demonstrated and used in the Oracle Bone Research Center of South China Normal University.