An Attention-Guided Matching Association Network for Hyperspectral and RGB Fusion Tracking

RGB-based trackers are prone to drift in some challenging scenarios. Hyperspectral data can provide more material information to address these challenges. Therefore, using hyperspectral information to supplement the RGB modality defects to improve tracking performance is worth exploring. However, there is almost no relevant work about this valuable issue. In addition, the two modality data in the existing hyperspectral-RGB dataset are not strictly matched and aligned, which brings significant challenges to the multi-modality tracking task. Therefore, we propose a simple but effective scheme to alleviate the problem of the difficulty of utilizing multi-modality information in tracking caused by the spatial difference between two modalities. In addition, to promote the development of the hyperspectral-RGB multi-modality tracking field, we propose a novel network that adaptively captures the relationship between the two modality information using the attention mechanism in Transformer to improve the tracking performance. Experimental results demonstrate the proposed method’s effectiveness.