Weakly Supported Plane Surface Reconstruction via Plane Segmentation Guided Point Cloud Enhancement

Most of the widely used multi-view 3D reconstruction algorithms assume that object appearance is predominantly diffuse and full of good texture. For the objects that violate this restriction, the surface can hardly be reconstructed because such area lacks sufficient support from dense point clouds. To tackle this problem, we introduce a novel two-stage prior-guided method based on point clouds enhancement to enable the application of multi-view reconstruction approaches in such scenes. In the first stage, we optimize the original PlaneNet plane segmentation priors by taking advantage of the estimated depth map and confidence map from multi-view stereo. In the second stage, we correct and supply 3D point clouds for the weakly supported plane surface on the basis of the upgraded priors. Furthermore, we utilize a slight disturbance of the enhanced point clouds to facilitate the subsequent mesh reconstruction. The proposed point cloud enhancement approach is evaluated on the large-scale $DTU$ dataset. Our method significantly outperforms previous multi-view stereo state-of-the-arts. We also demonstrate weakly supported plane surface reconstruction results from real-world photos that are unachievable with either the methods aiming at preserving weakly supported surfaces or the traditional state-of-the-art 3D reconstruction systems.