Perceptual Variation Stacking: Test Time Augmentations in Endoscopy Image Segmentation

The rapid advancement in machine learning is the force behind developmental strides in computer-aided diagnosis and treatment. Methods like Convolution Neural Networks, Reinforcement Learning etc., are making it possible for rapid diagnosis of diseases, development of new drugs and personalized medicine. One of such applications is the semantic segmentation of structures or pathology. Semantic segmentation makes it easy to detect and isolate abnormalities from normal human anatomy and thus is essential for computer-assisted robotic surgeries. Most of the existing work on medical image segmentation focuses on modalities like MRI, CT scans and pathology images. In this study, we use methods like U-Net, PAN and DeepLabv3+ to segment endoscopy images. We train and test the models on a 7-class human larynx dataset consisting of 536 endoscopic images from 2 patients. The main highlight of our paper is the image stacking based data augmentation strategy. We stacked perceptual augmentations like brightness, contrast, and sharpness of the image along channel dimension to extract more information from a single image. We use the IoU metric to evaluate the performance of the models and data augmentation strategies. Upon testing, stacked data augmentation strategy consistently outperformed traditional data augmentation strategy and improved the model performance. The datasets for endoscopy images are limited; thus, further improvements in model performance can be achieved by adding more images to datasets or further domain-specific refinement of the data augmentation strategy or using a semi-supervised learning approach.