Dense 3D Reconstruction with Fusing Structure from Motion and Dynamic Photometric Stereo

Structure from Motion is a 3D reconstruction technique based on image sequences, which estimates the camera motion and the 3D structure of the scene. However, in practice, Structure from Motion reconstructs the point cloud sparsely and lacks the necessary high-frequency details. To solve this problem, this paper proposes a densification method based on triangulation interpolation by fusing Structure from Motion and Dynamic Photometric Stereo. In detail, photometric stereo is used to compute the normal vector of the sparse points calculated by Structure from Motion. Then, we use the triangular dissections method to thicken the sparse points for dense 3D reconstruction with their positions and normals. Finally, we propose a corresponding optimization scheme to restore the 3D shape of the target object well. Experiments show that the proposed method greatly improves the accuracy in 3D reconstruction compared with the previous methods.