A Vision Processor With a Unified Interest-Point Detection and Matching Hardware for Accelerating a Stereo-Matching Algorithm

In this paper, a unified interest-point detection and matching hardware with an optimized memory architecture is proposed for a real-time stereo-matching system. In order to support a stereo-matching algorithm, the unified datapath in the hardware performs not only interest-point detection and matching algorithms such as features from the accelerated segment test (FAST) and binary robust independent elementary feature (BRIEF) in real time but also a census transform, which is widely used in stereo matching. To achieve maximum performance, we propose two special memory architectures: 1) reconfigurable image memory (RIM) and 2) point cloud index memory system (PCIM). RIM is a unified memory architecture that loads pixel values from a raw image patch. Since FAST, BRIEF, and the census transform have different and complex memory access patterns, the miss rate of the memory access may increase. To optimize the memory operation, RIM can change its memory configuration according to the algorithm. PCIM is a dedicated memory system that utilizes the geometric information of the cameras in order to reduce the off-chip memory bandwidth. Based on the geometric information, PCIM removes most of the redundant candidates. Since PCIM minimizes the off-chip memory bandwidth using a dedicated cache, the performance degradation is negligible compared with the exact-nearest-neighbor method. Area-based stereo matching is accelerated based on the general-purpose computing on graphics processing units (GPGPU) architecture because the search range is adaptively reduced according to the disparity of the matched correspondences. The overall hardware consists of 1.20-M logic gates and consumes a maximum of 185 mW. Interest point detection and matching accelerator achieves 106 frames/s in 1080p full high-definition video (HD) resolution at a 200-MHz operating frequency with 3500 descriptors per image.