Lightweight Single Image Super-Resolution With Multi-Scale Spatial Attention Networks

Convolutional neural networks (CNNs) generally provide higher performance gain for single image super-resolution (SISR) as the depth and number of parameters are increasing. However, just increasing the layers of straightforward deep networks has a problem that it requires an impractically large number of parameters for obtaining state-of-the-art performance. Instead, some researchers proposed lightweight networks, which is designed with more sophisticated network structures for achieving better performance than the straightforward networks at the same parameter requirement. In this paper, we propose new lightweight Multi-scale Spatial Attention Networks (MSAN) for SISR, which attempt to bring out a better performance from the relatively small number of parameters. Specifically, we adopt a dense connection with feature fusion layers to broadcast abundant features to every level of layers, and propose a double residual structure that provides an additional skip-connection. We also design a Multi-scale Spatial Attention Block (MSAB) to exploit multi-scale spatial contextual information. Furthermore, we introduce a spatial attention module which adaptively focuses on the most informative feature scale in a given region of the image. In the experiments, we validate that the proposed MSAN achieves significant accuracy compared to recent lightweight models and comparable performance to the state-of-the-art methods.