3-D scene reconstruction with viewpoint adaptation on stereo displays

We propose a generic algorithm for the geometrically correct reconstruction of 3-D scenes on stereo displays with viewpoint adaptation. This forms the basis of multiviewpoint systems, which are currently the most promising candidates for real-time implementations of 3-D visual communication systems. The reconstruction algorithm needs 3-D tracking of the viewers' eyes with respect to the display. We analyze the effect of eye-tracking errors. A simple bound is derived, below which reconstruction errors cannot be observed. We design a multiviewpoint system using a recently introduced image-based scene representation. The design formed the basis of the real-time multiviewpoint system that was recently built in the European PANORAMA project. Experiments with both natural and synthetic scenes show that the proposed reconstruction algorithm performs well. The experiments are performed by computer simulations and the real-time PANORAMA system.