3D Reconstruction with Spherical Cameras

The goal of image-based 3D reconstruction is to establish a high-quality 3D expression from images. In order to achieve a high-resolution and real-time 3D model, inspired by the open source COLMAP, we propose a novel framework (3DMAP) to reconstruct 3D scenes based on spherical cameras. Unlike traditional methods which focus on building a 3D plane by Poisson distribution function, our method illustrates the key processes of 3D reconstruction: locating the camera based on global feature, estimating the scene’s relative depth from monocular panoramic images, and obtaining a high-quality 3D surface reconstruction. In the camera locating part, we use a global descriptor augmentation model to build a labeled panorama dataset GDAP, in which the images are captured by our designed spherical cameras; In the depth estimation part, we propose a new network UMDE that can estimate the depth of both indoor and outdoor scenes; Finally, in the 3D surface reconstruction section, we turn the reconstruction problem to a graph optimization problem, called GraphFit, in which, we optimize the point clouds with s-t graph and smoothing method successively. We conduct experiments on our own dataset to demonstrate the effectiveness of our proposed 3DMAP framework. Experimental results show that our 3DMAP has achieved good evaluation scores and visual effects.