Gland Segmentation in Colon Histology Images via Attention-based Multimodal Information Fusion

Integrating pathological image and multimodal information such as patient metadata can help improve the segmentation performance. Most existing segmentation methods overlook the importance to exploit patient metadata, and suffer from suboptimal segmentation performance. To tackle these challenges, a novel multimodal feature fusion module (MFFM) is proposed. By incorporating a cross-modal attention mechanism and a self-attention mechanism, MFFM effectively and efficiently integrates information from different modalities. Experiments are conducted on GlaS, a glandular segmentation dataset, and the experimental results demonstrate that the method outperforms the state-of-the-art segmentation networks.