SuperHCA: A Super-Resolution Accelerator with Sparsity-Aware Heterogeneous Core Architecture

Deep learning-based super-resolution (SR) models have emerged as a potential approach to achieving high-quality images. The large SR networks can achieve a high peak signal-noise ratio (PSNR), a metric to evaluate the quality of the image. However, the SR networks typically contain large amounts of parameters, inducing high computation capacity and memory bandwidth requirements, which are difficult to deploy on embedded hardware. In this work, we develop the Anchor-Based Shuffle Net (ABSN) oriented to develop a hardware accelerator with a dynamic-scale fixed-point (DSFP) quantization method. In addition, we implement the dynamic quantization adaption in hardware. We design a Super-resolution Heterogeneous Accelerator, namely SuperHCA, employing a sparsity-aware heterogeneous architecture to distinguish between dense and sparse workloads to improve inference efficiency. Furthermore, we provide Slice Layer Fusion (SLF) computation in the heterogeneous cores to reduce external memory access and on-chip buffer sizes. The SuperHCA achieves 91 FPS with the lowest area overhead compared to the state-of-the-art works.