Scale-Consistent Learnable PnP Network for Space Target Pose Estimation

Due to constraints from imaging devices, the most effective method for estimating the pose of space target RGB images is to establish a 2-D–3-D correspondence and then use the perspective-n-point (PnP) algorithm for pose recovery. However, traditional PnP algorithms are not differentiable, which hinders their integration with neural network training. Although recent work attempts to make PnP partially differentiable during the 2-D–3-D matching stage by mathematical methods, this leads to increased inevitably computational costs. To this end, we propose a scale-consistent learnable PnP (SCLP) network that facilitates end-to-end pose estimation for space targets. Our method incorporates a sparse keypoint learnable PnP (SKL-PnP) layer within a multiscale network, enabling PnP to function as a differentiable layer that integrates seamlessly with preceding neural components. Additionally, we also sample the 2-D–3-D correspondences to obtain sparse keypoint pairs, achieving a lightweight single-stage 6-D pose estimation algorithm. To manage the significant scale variations in space target images, we introduce Gaussian perception sampling (GPS) by assigning instances to different pyramid levels based on size. Furthermore, we propose a scale consistency regularization (SCR) module that aligns downsized feature maps with original ones to better address scale differences. Experimental results demonstrate that our approach achieves superior accuracy and efficiency on the SPEED and SwissCube datasets, showing significant improvements over state-of-the-art methods.