RGBT Salient Object Detection: A Large-Scale Dataset and Benchmark

Salient object detection (SOD) in complex scenes and environments is a challenging research topic. Most works focus on RGB-based SOD, which limits its performance of real-life applications when confronted with adverse conditions such as dark environments and complex backgrounds. Since thermal infrared spectrum provides the complementary information, RGBT SOD has become a new research direction. However, current research for RGBT SOD is limited by the lack of a large-scale dataset and comprehensive benchmark. This work contributes such a RGBT image dataset named VT5000, including 5000 spatially aligned RGBT image pairs with ground truth annotations. VT5000 has 11 challenges collected in different scenes and environments for exploring the robustness of algorithms. With this dataset, we propose a powerful baseline approach, which extracts multilevel features of each modality and aggregates these features of all modalities with the attention mechanism for accurate RGBT SOD. To further solve the problem of blur boundaries of salient objects, we also use an edge loss to refine the boundaries. Extensive experiments show that the proposed baseline approach outperforms the state-of-the-art methods on VT5000 dataset and other two public datasets. In addition, we carry out a comprehensive analysis of different algorithms of RGBT SOD on VT5000 dataset, and then make several valuable conclusions and provide some potential research directions for RGBT SOD.