Improvement of Small Object Detection of Handwritten Answer Sheet

The rapid advancement of technology has enabled numerous applications, including the automated assessment of student answer sheets. Document Layout Analysis (DLA) is a key process in detecting handwritten answer sheets, as it identifies the positioning of handwritten content for further analysis and serves as an essential preliminary step for optical character recognition systems. However, analyzing handwritten documents poses significant challenges due to variations in handwriting styles and irregular structures. These challenges are particularly evident in mathematics answer sheets, where students often arrange their answers freely. Despite advancements in DLA research, the analysis of unstructured or unformatted handwritten mathematics answer sheets is unexplored. This research proposes a modified YOLOv8 model for DLA on unformatted handwritten answer sheets. This method involves adding the Convolutional Block Attention Module (CBAM), getting rid of one C2f layer (a new setup), and using Particle Swarm Optimization (PSO) to fine-tune the hyperparameters. These modifications enhance the model's sensitivity to some features, reduce complexity, and maintain high accuracy. Experimental results reveal that the modified YOLOv8 with CBAM achieves comparable and superior accuracy in some features. After the implementation of PSO, across various evaluation metrics, the model consistently demonstrates improved performance for incorrect text. The most notable enhancement is a 9.6% increase in precision and 4.37% increase in mAP@50. Additionally, the model's computational efficiency is significantly improved, with GFLOPs reduced by 4.88% leading to a 14.52% reduction in model size.