TAP-Track: Generalizable Spacecraft Pose Tracking by Tracking Any Points

Recent learning-based spacecraft pose tracking methods have demonstrated impressive improvement in estimation accuracy and potential scalability to a complex space environment. However, most of them still rely on the detection of discriminative keypoints on a known 3-D model, limiting the generalization to unknown spacecraft. To this end, we propose, to the best of our knowledge, the first generalizable spacecraft pose tracking method. Instead of requiring a known model, we only assume the existence of at least one planar structure, for example, solar panels, which holds for most satellites in general scenes. Additionally, the proposed method tracks any points on the plane across multiframe followed by a re-projection error minimization, rather than detecting keypoints between image pairs, allowing robust capture of “long-term” temporal information among frames even for textureless surfaces without sufficient keypoints. Moreover, we also constructed the first large-scale dataset, G-SPET, for generalizable spacecraft pose estimation and tracking. It covers 174 satellites with diverse structures and rich annotations, increasing the number of targets in previous datasets by almost two orders of magnitude. Extensive evaluations on the proposed dataset have demonstrated the superiority of our method over state-of-the-art (SoTA) methods. The code and dataset will be made publicly available soon.