Viewpoint Recommendation Based on Object-Oriented 3D Scene Reconstruction

Viewpoint recommendation can recommend several viewpoints for taking aesthetic photographs of a place-of-interest (POI) and is of great importance for photography assistance. In this paper, we propose a system that can assist a user in choosing good viewpoints for taking high-quality photographs. Our system is based on social media and 3D reconstruction. To reduce the time cost and improve the quality of 3D reconstruction, we propose a weakly supervised object detection method that is used before 3D reconstruction. The camera pose of images is recovered by the subsequent 3D reconstruction pipeline. We use a convolutional neural network (CNN) to extract 2D image features, and we fuse them with 3D camera pose features to learn their relationships to image aesthetics. The trained model is utilized to evaluate the aesthetics of images. Finally, the 3D space of all possible camera poses is divided into 3D grids, and the aesthetics score of each grid is evaluated. We combine the aesthetics and diversity of all viewpoints and recommend several high-quality viewpoints. Experimental results indicate that our approach can help users choose viewpoints that will result in high-quality photographs while maintaining diversity.