Anatomical Attention Guided Dual-Stream Encoding Multi-Task Learning Model for 3D Medical Image Registration and Segmentation

Medical image registration and segmentation constitute two fundamental tasks within the domain of medical image processing, characterized by their high relevance and complementarity. Consequently, the implementation of multi-task learning (MTL) holds the potential to facilitate their mutual benefits. Nonetheless, the relevance and complementarity between these tasks remains largely unexplored, and an adequate MTL feature-sharing mechanism has yet to be developed. In this study, a novel anatomical attention guided dual-stream encoding multi-task learning model for 3D medical image registration and segmentation is proposed to address the aforementioned challenges. Specifically, a dual-stream transformer-based shared encoder (DTSE) is proposed to bridge the encoding gap of MTL for registration and segmentation, facilitating a more efficient and accurate across-task shared feature representation. Furthermore, a multi-scale shared anatomical attention module (MS-SAAM) is proposed to address insufficient feature sharing, promoting complementarity of beneficial features across tasks. Compared to the state-of-the-art methods, our proposed method exhibits superior registration and segmentation performance, indicating the potential of it to formulate an efficient MTL model for 3D medical image registration and segmentation.