Real-time stereo vision implementation on the Texas Instruments KeyStone II SoC

Accurate calculation of a disparity map in real-time is a challenge in computer vision and autonomous applications due to the expensive computation cost. This paper describes a local temporal stereo method, using normalized cross correlation (NCC) as the matching cost function. To increase the performance in terms of speed without degrading the quality, a down sampling of the disparity map in the y-axis has been used, with a restricted search range along the y-axis. In addition to these, the search range is further restricted by referencing the disparity map from the previous frames. The algorithm is evaluated on a Texas Instruments (TI) KeyStone II embedded SoC with four ARM cores and eight DSP cores. The application has been written using OpenMP, OpenCL and IPC/MessageQ for parallelising the program. The core of the application has been optimised for this specific platform using linear assembly code in order to take advantage of SIMD instructions available with the DSP. The application achieved 77 FPS on a 1242×375 resolution video with 80 disparity levels using one ARM core and eight DSP cores.