A General-Purpose Computing Core With Cooperative Motion Detection and Feature Extraction for Always-On PWM Image Sensors

This article presents a mixed-signal general-purpose computing core designed to support real-time inference applications in always-on low-power pulsewidth modulation (PWM) CMOS image sensors (CISs), functioning as a processing-in-sensor (PIS) circuit. This core can be integrated into the columns of CISs to perform low-power edge processing on images, without affecting the pixel fill factor and imaging quality. It employs a coordinated mechanism in which motion detection (MD) triggers the activation of feature extraction (FE), thereby achieving an organic integration of MD and FE functionalities, and maximizing the system’s power efficiency. MD is implemented via in-column frame difference (FD), while FE is performed using a programmable-weight $3\times 3$ convolution, a rectified linear unit activation function, and a $2\times 2$ max-pooling (MP) operation. Both functionalities are computed based on real-time PWM signals from the CIS and the principle of current integration, with partial circuit reuse achieved through different switching operations. A 0.8-V computing core prototype, with an area of 720 × 272 $\mu$ m, was fabricated and verified using 0.18- $\mu $ m standard CMOS technology. The experimental results at an image frame rate of 250 fps demonstrated an average power consumption of $5.23~\mu $ W for MD and $17.53~\mu $ W for FE. The prototype core computes the first two layers of an ultra lightweight convolutional neural network (CNN) for the task of MNIST digit classification, achieving an accuracy loss of only 0.86% compared to the ideal scenario. This analog computing core can be used in multimode, low-power, edge-intelligent vision sensors.