Implementation of auto-rectification and depth estimation of stereo video in a real-time smart camera system

In this paper, we present a real-time, low power, and wireless embedded stereo vision system. The system consists of WiCa board (Wireless Camera board) and the methods developed using a smart camera processor, IC3D. IC3D is an SIMD video-analysis processor that has 320 processor elements. The proposed auto-rectification method is suitable for a parallel stereo system like WiCa. It is based on the matching of a planar background. After rectification, the two images of the background are coincident with each other, i.e., both the vertical and horizontal disparity of the background plane is removed. Then a dense matching method is implemented to achieve the depth map of the foreground object. Left-to-right checking and reliability checking is applied to reduce the error of the estimated depth. The system runs at 30fps and handles disparity up to 37 pixels in CIF (320×240 pixels) mode.