Free Viewpoint Image Generation Synchronized with Free Lisning-Point Audio for 3-D Real Space Navication

In this paper, we present a method that generates a free view/listening point image/audio by using a completely synchronized capturing system, named one hundred camera and microphone system. This system can capture audio/image at 100 points with synchronized signal in a three-dimensional scene. This paper introduces an image interpolation method between nearest cameras to generate a free viewpoint image. The step of interpolating image is: calibrating all camera intrinsic/extrinsic parameter, rectifying each nearest stereo image pair, adjusting color balance among all images, stereo matching with occlusion detection, and generate middle image with interpolation along angle with disparity and occlusion map. The experimental results show that we can generate free view point image with free listening point audio. We have succeeded to render this data into 3-D cylindrical display.