Semi-Supervised 360° Depth Estimation from Multiple Fisheye Cameras with Pixel-Level Selective Loss

In this paper, we study a practical omnidirectional depth estimation with neural networks that enables effective learning on real world data obtained using wide-baseline multiple fish-eye cameras. Most previous approaches only used synthetic data providing dense and accurate depth ground truth (GT). However, it is unrealistic to acquire such high quality GT data in real world due to limitations of the existing depth sensors. We first introduce two critical problems that can reduce the accuracy of depth estimation: depth GT sparsity and sensor calibration error. We then propose a novel semi-supervised learning method using pixel-level loss that selectively uses supervised loss and unsupervised re-projection loss according to existence of GT. Empirical results demonstrate that our method efficiently reduces the performance degradation in both simulation on synthetic data and real world data using sparse depth sensor.