Virtual-view-based stereo video composition

Given two sets of videos captured by two sets of multiple cameras, we like to combine them to create a new stereo scene with the foreground objects from one set of video and the background from the other set. We address the camera parameter mismatch and camera orientation mismatch problems in this paper. We propose a floor model to adjust the camera orientation. Once we pick up the landing point (of foreground) in the background scene, we need to adjust the background camera parameters (position etc.) to match the foreground object. The depth information is needed in the above calculation. Thus, new background scenes may have to be synthesized based on the calculated virtual camera parameters and the given background pictures. Plausible results are demonstrated using the proposed algorithms.