IPFNet: Interactive Perceptive Fusion Network for Self-Supervised Monocular Depth Estimation

Monocular depth estimation (MDE) is an essential challenge in computer vision, and deep learning has provided numerous solutions. Despite advancements, deep learning models frequently encounter limitations, including the ambiguity in feature representation, particularly around object boundaries and fine structures, as well as the dependency on large-scale, accurately labeled datasets. The latter are costly and difficult to acquire, especially in outdoor and dynamic environments. Building upon recent research, we introduce a novel method for an interactive perception fusion network(IPFNet) framework that uniquely leverages semantic and inertial measurement unit(IMU) cues in complex environments. Our approach emphasizes the precise fusion of semantic and depth features particularly at object boundaries, proposing the symmetric extraction propagation(SEP) that facilitates a deep interaction of rich contextual information across multiple scales between semantic and geometric domains. Meanwhile, we introduce a visual-inertial fusion filter, which jointly models the IMU-based motion propagation and a deep learning-based pose estimation network. The resulting posterior pose is employed to construct an IMU-guided photometric consistency loss. By fusing multimodal data, this module enriches the diversity of supervision signals and effectively mitigates the limitations of monocular vision in handling illumination changes, occlusions, and dynamic objects. Our model undergoes extensive evaluation on established benchmark datasets, with comparisons to existing state-of-the-art (SOTA) methods. The results indicate enhanced depth estimation accuracy, improved model robustness, and better performance under challenging lighting conditions.