TCBFormer: A General Architecture Based on Dual-Branch Feature Fusion for Polyp Segmentation

Polyp segmentation is crucial for early detection of colorectal cancer. With the development of deep learning, training end-to-end networks to achieve fully automatic polyp segmentation has become the mainstream nowadays. However, these methods often face difficulties when dealing with out-of-distribution datasets, missing boundaries, and small polyps. In 2022, the introduction of FCBFormer marked a significant advancement in vision tasks, enhancing multi-task computer vision performance while effectively tackling challenges such as out-of-distribution datasets, absent boundaries, and small polyps. This innovative approach not only outperformed traditional Vision Transformer and CNN backbones but also mitigated their inherent limitations. But there are still many areas for improvement in FCBFormer architecture such as the feature fusion method, decoder architecture and the basic modules stacked by its Transformer branch. Our idea based on FCBFormer, using the dual branch parallel network structure, proposing a more efficient feature fusion module named FCT and a more reasonable progressive decoder architecture to get the more refined semantic segmentation results. Our architectture gained the State of the Art on the Kvasir-SEG, CVC-CLINICDB dataset, which provided the effectiveness of our proposed architecture for polyp segmentation.