HFMDNet: Hierarchical Fusion and Multilevel Decoder Network for RGB-D Salient Object Detection

Vision-based measurement techniques are required in the quality inspection process of various products. However, most of the existing research methods focus on the use of a single modality (red green blue (RGB) image or depth map) for defect detection. In this article, we propose a potential defect detection technique by introducing red green blue-depth (RGB-D) salient object detection (SOD) as a measurement method and presenting a hierarchical fusion and multilevel decoder network (HFMDNet). The key to the recently popular multimodal SOD lies in effectively acquiring cross-modal complementary information and realizing the interaction between cross-level information. Most existing methods attempt to employ various fusion strategies for cross-modal fusion or implement feature enhancement before fusion. However, these methods ignore the hierarchical distinctions between RGB and depth maps in cross-modal fusion, resulting in suboptimal performance in some cases of challenging situations. We fully take the cross-level information interaction both in the fusion and decoding stages into account and propose an HFMDNet. Specifically, we design a hierarchical fusion module (HFM) to compensate for modal differences between multimodal data, including a low-level feature fusion (LFF) module and a high-level feature fusion (HFF) module. Then, a multilevel refinement decoder (MRD) is designed to enhance, refine, and decode the fusion features to generate saliency maps with high quality. In addition, we introduce the edge features in the decoding phase as the auxiliary information to generate salient objects with clear boundaries. Extensive experiments conducted on nine publicly available datasets demonstrate that our HFMDNet delivers competitive and excellent performances.