A 43.7 mW 94 fps CMOS image sensor-based stereo matching accelerator with focal-plane rectification and analog census transformation

The depth information is actively utilized for many applications such as mobile gesture user interface (UI). However, the previous stereo vision systems are unsuitable for the mobile gesture UI due to the long latency and the high-power consumption of external image sensor in embedded environments. In this paper, we propose a CMOS image sensor-based real-time stereo matching accelerator with low power consumption. For real-time operation, the focal-plane rectification is proposed to perform the image readout, the rectification, and the matching cost generation at the same time. Also, a low-power analog census transformation is implemented by simple comparator circuits. The proposed stereo matching CIS, implemented in 65nm CMOS technology, consumes 43.7 mW at 94.1 fps frame rate. It achieves 5.30×103 MDE/J energy efficiency.