FcFNet: A Challenge-Based Feature Complementary Fusion Network for RGBT Tracking

RGBT tracking technology often struggles in noncooperative challenges, such as illumination variation, scale variation, fast motion, occlusion, and thermal crossover. In this work, we propose a novel RGBT tracking model, called challenge-based Feature complementary Fusion Network (FcFNet), especially aiming at noncooperative scenarios. In particular, we first decouple the mixed challenging attributes and adaptively combine features at each layer using the challenge branch fusion module (CBFM), forming more discriminative target representations. Subsequently, we employ two nonshared convolution kernels in each layer and one shared convolution kernel to, respectively, extract individual and common features of different modalities. By applying the dynamic convolution fusion module (DCFM), the nonshared and shared convolution kernels are reweighted dynamically. Finally, we embed the feature-enhanced aggregation and dynamic kernel weights into the backbone network, which forms the FcFNet. Experimental evaluations conducted on the GTOT and RGBT234 datasets demonstrate a notable enhancement in detection performance, with an absolute improvement of 2.4% in success rate and 3.3% in precision rate when leveraging the CBFM and DCFM. The proposed FcFNet exhibits competitive performance compared with other state-of-the-art tracking algorithms, especially in continuous tracking tasks for noncooperative targets.