Image-to-Volume Deformable Registration by Learning Displacement Vector Fields

Two-dimensional/three-dimensional (2D/3D) image registration is a problem that solves the deformation and alignment of a pre-treatment 3D volume to a 2D projection image, which is available for treatment support and biomedical analysis. 2D/3D image registration for abdominal organs is a complicated task because the abdominal organs deform significantly and their contours are not detected in 2D X-ray images. In this study, we propose a supervised deep learning framework that achieves 2D/3D deformable image registration between the 3D volume and a single-viewpoint 2D projection image. The proposed method uses latent image features of the 2D projection images to learn a transformation from the input image, which is a concatenation of the 2D projection images and the 3D volume, to a dense displacement Vector Field (DVF) that represents nonlinear and local organ displacements. The target DVFs are generated by registration between 3D volumes, and the registration error with the estimated DVF is introduced as a loss function during training. We register 3D-CT volumes to the digitally reconstructed radiographs generated from abdominal 4D-CT volumes of 35 cases. The experimental results show that the proposed method can reconstruct 3D-CT with a mean voxel-to-voxel error of 29.4 Hounsfield unit and a dice similarity coefficient of 89.2 % on average for the body, liver, stomach, duodenum, and kidney regions, which is a clinically acceptable accuracy. In addition, the average computation time for the registration process by the proposed framework is 0.181 seconds, demonstrating real-time registration performance.