Enhancing Gastrointestinal Stromal Tumor (GIST) Diagnosis: An Improved YOLOv8 Deep Learning Approach for Precise Mitotic Detection

Gastrointestinal stromal tumor (GIST) is the most common malignant tumor originating from interstitial cells in the gastrointestinal tract. Different grades require various surgical interventions and adjuvant treatments, which are closely linked to the patient’s prognosis. The current clinical risk stratification method relies heavily on the identification and counting of mitotic figures, which serve as important criteria. However, manual evaluation of pathological slides in clinical practice is often limited by the shortage of experienced clinicians and the subjectivity in interpreting results. Therefore, in this paper, we propose an enhanced YOLOv8 network framework for the automatic detection of mitotic cells in GIST. We substituted the backbone network with VanillaNet, known for its simplified model complexity in feature extraction. This change facilitated the identification of specific targets and improved network performance. Additionally, we introduced the Advanced Feature Pyramid Network (AFPN) to further enhance the model’s accuracy. Experimental results show that the proposed model achieved an accuracy of 0.816, a recall rate of 0.858, and an F1-score of 0.837 on the test dataset. It demonstrates superior efficacy in identifying mitotic cells, outperforming the original YOLOv8 model in overall performance. This augmented model has the potential to significantly reduce reading time while ensuring consistent diagnostic results, thereby greatly improving diagnostic efficiency. Future large-scale validation is necessary for the clinical adoption of this model.