Wireless Capsule Endoscopy Diagnosis Using Prototype Self-Attention and Dynamic Curriculum Learning

Wireless capsule endoscopy is a non-invasive and painless approach for diagnosing gastrointestinal diseases. An automated medical decision support system can significantly enhance clinician efficiency and reduce the incidence of misdiagnosis when analyzing lesions within the 50,000 to 100,000 image frames generated for each individual. However, only a small number of previous studies have focused on fine-grained recognition characteristics, and very few have simultaneously addressed both fine-grained recognition and class imbalance issues in wireless capsule endoscopy. To address these issues, we propose a novel medical decision support network that includes a prototype attention enhancement module and a dynamic curriculum learning approach. The prototype attention enhancement module improves lesion-sensitive compact representation learning by leveraging cosine similarity between the representation and a prototype memory with multi-class centers. The dynamic curriculum learning adopts triplet loss and weighted cross-entropy, facilitated by a progressive factor that controls between fine-grained representation learning and class-sensitive learning. Through extensive comparative experiments on three public datasets, the proposed network demonstrated competitive performance, outperforming previous methods with an F1-score of 96.6% on the 10-class Kvasir-Capsule dataset, an accuracy of 98.3% on the CAD-CAP dataset, and an accuracy of 93.7% on the mixed KID dataset. Two additional gastrointestinal histopathology datasets confirm the generalization of the proposed network. The code will be available at https://github.com/Xingcun-Li/MDSN-WCE Note to Practitioners—The primary aim of this study is to address diagnostic challenges in automated wireless capsule endoscopy (WCE). WCE provides a non-invasive diagnostic method for gastrointestinal diseases, but manual analysis of the resulting large volume of image data is time-consuming and error-prone. To this end, we propose a novel medical decision support network, which is also applicable to similar medical image analysis issues characterized by class imbalance and fine-grained recognition. Our approach integrates a prototype attention enhancement (PAE) module and dynamic curriculum learning (DCL), enabling the model to accurately identify lesions within WCE images and exhibit favorable performance in gastrointestinal histopathology image recognition beyond the WCE domain. Additionally, our approach can be applied to various common deep learning models without incurring significant additional computational costs, thereby aiding endoscopists in significantly reducing the time required for diagnosing WCE video frames. The code will be uploaded to GitHub for direct use in the automated diagnosis of WCE video frames. While our method shows promise, its main focus is on rapidly identifying a large number of frames within WCE videos. However, further work is needed to extract key frames and improve the interpretability of outputs from WCE videos, which is also a future research direction.