Denoise on Sensor: A Near-Sensor Compute-in-Memory Macro for Visual Perception Denoising via Concatnation-Eliminating

Noise is one of the most common and significant factors leading to image degradation. In recent years, due to the rapid development of neural networks, the performance of denoising algorithms has seen a substantial improvement. However, state-of-the-art denoising models often entail large-scale models and heavy computational requirements, making deployment challenging. Additionally, we have observed that the location of denoisers in the entire image processing pipeline has a significant impact on resource consumption and denoising effectiveness. Deploying the denoiser closer to the image acquisition stage will be more effective in separating noise from the image. In this paper, we propose a near-sensor compute-in-memory macro for visual perception denoising (Denoise on Sensor, DoS) along with its corresponding Edge Denoise U-net (EDU) architecture. DoS employs a mixed-signal circuit implementation for neural network inference, offering a notable advantage in terms of high speed and low power consumption compared to FPGA or GPU based approaches, making it feasible to deploy denoising tasks at the near-sensor edge. The simulation results show that the energy efficiency of DoS can reach 21.98 TOPS/W, and EDU deployed on DoS can achieve around 30dB PSNR and 0.83 SSIM on KODAK, BSD300 and SET14 datasets.