IMTrack: Interlayer Interoperability and Multi-scene Optimization for Visual Multimodal Target Tracking

In the domain of target tracking, leveraging auxiliary modalities such as depth, thermal, and event data to enhance tracking robustness has garnered substantial attention. Due to the scarcity of visual multimodal datasets, state-of-the-art approaches primarily rely on parameter-efficient fine-tuning to adapt models. However, existing studies often neglect the adaptation of fine-tuning to specific datasets, frequently focusing only on non-RGB modalities or employing them as prompts. Additionally, current methods overly emphasize modal balance while disregarding modality adaptation to environmental conditions. To address these limitations, we propose a unified visual multimodal detection framework named IMTrack. This model incorporates LoRA (Low-Rank Adaptation) and Adapter for fine-tuning RGB and auxiliary modalities, respectively, and employs confidence-based cross-attention to strengthen feature interaction and adaptability. Furthermore, we introduce a complementary masking multi-scene optimization strategy to enhance robustness in complex environments. Extensive experiments on five datasets validate the efficacy of our model. The results demonstrate that IMTrack surpasses existing methods in many indicators, achieving state-of-the-art performance. Notably, the robustness of our model to auxiliary modes is improved by more than 2%. Our source code is available at: https://github.com/cxy-lzk/IMTrack.