Improving local stereo algorithms using binary shifted windows, fusion and smoothness constraint

Stereo cameras are a viable solution for reconstructing 3D scenes and are well suited for advanced driver assistance systems, autonomous driving and robotics applications. Modern stereo reconstruction algorithms offer good results, but require very much memory and their real time capabilities are limited on modern day processors. On the other hand, local window aggregation algorithms have a small memory footprint, they are very fast and can be ported to embedded devices, although they provide a lower number of 3D reconstructed points and are more error prone in the case of occluded and slanted surfaces. In this paper we propose a novel, local block matching method which has increased quality and is suitable for real time processing with hardware acceleration (satisfying running time). Our first contribution consists in the introduction of two new binary descriptors used for block matching. The second contribution lies in the shifting method implemented for the matching windows, in order to capture surfaces which are slanted, together with the fusion of the results obtained for fronto-parallel surfaces. Here we propose and compare two fusion methods: a naive and a gradient based approach. The final contribution consists in a smoothness constraint applied to neighboring pixels. The results have been tested on images from the Middlebury benchmark and also on real traffic scene.