Automatic Endoscopy Classification by Fusing Depth Estimations and Image Information

Gastric cancer ranks as the fifth leading cause of cancer mortality worldwide. The quality of upper gastrointestinal endoscopy is crucial towards early identification of premalignant conditions and relies on the endoscopist’s skill and thorough examination of stomach landmarks. Unfortunately, it has been observed that existing cancerous lesions may go undetected during examination. To standardize the quality of this procedure, meticulous protocols have been proposed. To support this process, we focused on developing a model to identify the anatomical locations in esophagogastroduodenoscopy images. This study advances endoscopic image classification by incorporating depth map estimation, essential for measuring distances to specific landmarks. This method, analyzing 2,054 images from 96 patients across 13 gastric regions using the ConvNeXT architecture with information fusion techniques, achieved an 87% F1 macro score. This approach suggests that depth map integration can improve stomach region classification, boosting prediction accuracy and potentially reducing missed gastric lesions.