Fast interpolated cameras by combining a GPU based plane sweep with a max-flow regularisation algorithm

The work presents a method for the high speed calculation of crude depth maps. Performance and applicability are illustrated for view interpolation based on two input video streams, but the algorithm is perfectly amenable to multicamera environments. First a fast plane sweep algorithm generates the crude depth map. Speed results from hardware accelerated transformations and parallel processing available on the GPU. All computations on the graphical board are performed pixel-wise and a single pass of the sweep only processes one input resolution. A second step uses a min-cut/max-flow algorithm to ameliorate the previous result. The depth map, a noisy interpolated image and correlation measures are available on the GPU. They are reused and combined with spatial connectivity information and temporal continuity considerations in a graph formulation. Position dependent sampling densities allow the system to use multiple image resolutions. The min-cut separation of this graph yields the global minimum of the associated energy function. Limiting the search range according to the initialisation provided by the plane sweep further speeds up the process. The required hardware is only two cameras and a regular PC.