Free-viewpoint image synthesis from multiple-view images taken with uncalibrated moving cameras

In this paper, we propose methods for free-viewpoint image synthesis from multiple-view images taken with uncalibrated cameras. In our method, two viewpoints are selected as basis images for defining a projective 3D coordinate of the object scene in which the scene structure is recovered from the input multiple-view images. The 3D coordinate is defined by the image coordinates of the selected two basis images according to the epipolar geometry between those two images, which is represented by a fundamental matrix. The multiple-view images are also related to the projective 3D coordinate by their epipolar geometry to the basis images. Based on such a framework, we do not need to strongly calibrate the cameras, so we can recover 3D structure of the scene without effort for the camera calibration. In addition to that, we can also render free-viewpoint images from hand-held moving cameras. We will demonstrate the effectiveness of the proposed method by showing free-viewpoint images from multi-view images taken with hand-held moving cameras.