Temporal Correlation Network for Video Polyp Segmentation

Accurate polyp segmentation from colonoscopy images is essential for identifying colorectal cancer. Recently, segmentation methods based on convolutional neural networks and transformers have represented excellent performance for image polyp segmentation. However, these methods are mostly designed for individual images rather than the entire video datasets, which results in the absence of sequential relationships among lesion images and neglects the significant intrinsic property of continuous video. In this work, we propose a temporal correlation network (TC-Net) for video polyp segmentation. In TC-Net, the temporal correlation is unprecedentedly modeled based on the relationship between the original video and the captured frames to be adaptable for video polyp segmentation, and the network is also calibrated for the corresponding time correlation output. Furthermore, we design a dual-track learning strategy for the optimization method in TC-Net to ensure the independence of TC-Net during the learning process to adequately exploit the optimization effect of temporal correlation. The network’s effectiveness is demonstrated by extensive experiments on five publicly available biomedical datasets, and TC-Net achieves state-of-the-art (SOTA) performance.