UBPT: Unidirectional and Bidirectional Prompts for RGBD Tracking

RGBD tracking addresses the limitations of single RGB modality tracking in complex environments by incorporating depth images. In recent years, the widespread use of depth sensors has made RGBD data available, greatly promoting the development of RGBD tracking. However, existing RGBD datasets still suffer from small scale and scarcity, as well as the dominance of modalities in the tracking process often fluctuates with changes in the open environments. Most existing RGBD trackers struggle to adaptively extract information from both modalities, hindering the models’ capacity to leverage the combined strengths of RGB and depth data. To address this issue, we propose a novel prompt-based RGBD tracking framework named unidirectional and bidirectional prompts for RGBD tracking (UBPT). UBPT employs a dual-stream transformer architecture and implements both unidirectional and bidirectional prompt learning to extract RGB and depth modality information, fully achieving cross-modality feature interaction. It can dynamically perform unidirectional intramodality feature enhancement and bidirectional intermodality feature fusion. This approach can dynamically and adaptively enhance dominant features while suppressing nondominant features. Experimental findings reveal that our proposed tracker achieves advanced results compared to recent state-of-the-art methods on various RGBD tracking benchmarks, demonstrating its superior tracking performance.