A 0.52mJ/Frame 107fps Super-Resolution Processor Exploiting Pseudo-FP6 Sparsity for Mobile Applications

Super-resolution (SR) is essential for enhancing digital image quality and is increasingly employed across various domains. The ability to recover fine details is especially critical in mobile applications such as gaming, video, and photography [1]. However, mobile devices are usually sensitive to latency and power consumption, so SR processors that drain a lot of battery are not practical. Designing energy-efficient hardware for convolutional neural network (CNN)-based SR (SRCNN) [2] to meet these constraints poses three key challenges, as shown in Fig. 2.10.1. Firstly, prior works [3], [4] have introduced substantial on-chip memory for multi-dimentional (multi-dim) overlapping regions (OR) in layer-fusion dataflow to achieve low external bandwidth. However, to further improve energy efficiency, multi-dim OR and external memory access should both be reduced. Secondly, novel low-bit-width but high-dynamic-range computation is required, since it is difficult for a SRCNN to obtain high image quality under traditionallow-bitwidth quantization methods, which are widely used for energy reduction [1]. Thirdly, deconvolution accounts for a large proportion of the workload in a SRCNN. Reusing the convolution datapath for deconvolution operations can improve area efficiency but leads to many redundant calculations. Alternatively, additional datapaths or registers are required to support deconvolution. It is important to remove additional both hardware overhead and redundant calculations to obtain high energy efficiency and improve hardware utilization.