Self-supervised online learning of appearance for 3D tracking

This paper presents a self-supervised online learning approach for 3D object tracking that requires no pretraining of appearance. Our method focuses on selecting the most relevant parts of the RGBD input by continuously updating appearance classifiers in conjunction with the spatial occupancy of the target. Fine-grained regions selected via the learned bottom-up saliency, together with spatial cues of the 3D shape model, are used to identify and localize the target via shape registration. The subsequent 3-D pose estimate along with positive and negative labels from the registration are used for online learning appearance. The proposed method outperforms competing model-based tracking algorithms on public datasets as well as on a new motion scene dataset that we have collected.