Self-attention-enhanced 3D Convolutional Dual-stream Pyramid Registration Neural Network

Medical image registration is mainly used for 3D reconstruction of images, which has important clinical research significance. Therefore, accurate medical image registration is an important and highly challenging research topic. Current state-of-the-art methods employ pyramid registration neural network models to learn the anatomical relationship between moving images and fixed images, achieving breakthrough progress. However, these methods overlook the spatial relationship between non-adjacent elements in the images, leading to a small receptive field problem. To address this problem, we propose a self-attention enhanced 3D convolution dual-stream pyramid registration neural network model based on Transformer-ConvNet, named TransPyramid++. In TransPyramid++, Self-attention Blocks based on attention mechanisms and PatchMerging based on image block fusion are exploited to capture the anatomical relationship between moving images and fixed images, considering the non-adjacent elements. Then, by utilizing the upsampling and skip connections to decode this relationship step by step, while maintaining the residual relationship between the encoder and decoder. Finally, we propose the PR++ Module based on 3D convolution operations to calculate correlations between feature maps at different positions, which is an enhancement for the relation between non-adjacent elements. As a result, the problem of small receptive fields is alleviated. We conducted extensive experiments on the publicly available brain image registration datasets, Mindboggle101 and LPBA40. The experimental results show that compared to state-of-the-art methods, the performance of our model exceeds that of existing models, with an average improvement of 6.7% on the ASSD (Average Symmetric Surface Distance) index.