Hardware Design of a Context-Preserving Filter-Reorganized CNN for Super-Resolution

This paper presents a hardware design of a CNN for single image super-resolution (SISR) processing. Very deep convolutional network for image super-resolution (VDSR) is a promising algorithm for SISR generating high-quality images but it is too complex to be implemented in hardware for commercial products. The paper presents a novel hardware design that is applicable to commercial products with low hardware resource usage while avoiding a significant degradation of image quality. The proposed design employs sub-pixel convolution layers to support various scale factors. A context-preserving 1D reorganization scheme is proposed to reduce the number of multipliers. Additional optimization technique is proposed to reduce the internal SRAM to store the input feature map. Thanks to the proposed optimizations along with additional techniques popular in hardware design, the number of multipliers and the size of SRAMs are reduced to 1/118.27 and 1/47.59, respectively, in comparison with a straightforward implementation of VDSR. The proposed CNN hardware synthesized with TSMC 65-nm CMOS technology, and it requires 8153.1K gate counts and 333.1kB SRAM to process a full-HD image. The proposed context-preserving 1D reorganization and memory reduction optimizations are also applicable to denoising and image classification without a significant performance degradation.