A Pyramidal Feature Selection and Translation Approach for Large Deformation Image Registration

In recent years, CNN-Transformer hybrid networks, particularly those with dual-stream pyramid encoders, have gained significant attention in medical image analysis for their ability to combine the strengths of both CNNs and Transformers. However, a critical issue has been largely overlooked: the initial misalignment or inaccurate pre-alignment of image pairs, which can degrade the performance of these networks and limit their effectiveness on clinical datasets. To address this, we propose a novel multi-scale network framework for joint affine and deformable image registration tasks. Our approach introduces an additional branch to the dual-stream pyramid encoder, utilizing the Feature Selector to establish more precise spatial mappings for initially misaligned image pairs. This improves the network’s ability to correct large spatial misalignments during joint affine registration. Additionally, the Feature Translator in the decoder captures regions of interest based on image intensity, leveraging the inductive biases of convolutional layers to enhance pixel-level alignment. Our model demonstrates robust performance on registration tasks using two publicly available brain MRI datasets and is effective in addressing inter-frame mismatches in brain PET images. The code for our framework will be publicly available at https://github.com/ZZU-zy228/FSTNet.