An On-Chip Binary-Weight Convolution CMOS Image Sensor for Neural Networks

A CMOS image sensor (CIS) that can perform on-chip binary convolution is presented. The CIS can greatly reduce memory usage and computational complexity by directly generating a feature map for a binary neural network. The pixel readout of the CIS is performed in the column-parallel fashion using incremental delta-sigma analog-to-digital converters (ADCs). The CIS operates in two different modes: convolution and normal modes. When the column ADC is working in the convolution mode, it works as a first-order delta-sigma ADC and generates convolved images using a binary kernel. In the normal operation mode, the ADC is switched to a second-order delta-sigma ADC with little hardware modification and used to capture high-quality images. To demonstrate the CIS architecture, a 192 × 128-pixel CIS, which occupies an active die area of 14.44 mm
²
, is fabricated in a 0.18 μm standard CMOS process. The performance of the CIS is evaluated through measurements and network simulations. In the normal operation mode, the CIS achieves a read noise of 14.79 e
^-
_rms
and a full-well capacity of 6,420 e
^-
with a resulting dynamic range of 53 dB. The power consumptions of the CIS are 49.2 and 52.5 mW during the normal and convolution modes, respectively.