Densely Connected Transformer With Linear Self-Attention for Lightweight Image Super-Resolution

Image super-resolution (SR) is the process of restoring high-resolution (HR) images from low-resolution (LR) ones. Recent Transformer-based SR methods have achieved impressive results by utilizing the self-attention (SA) mechanism, which allows modeling long-range dependencies among input features in spatial dimensions. However, the computational complexity of SA increases quadratically with respect to the feature size, which makes Transformer-based methods inefficient. Additionally, despite the success of dense connections in convolutional neural network (CNN)-based methods, they have not been fully explored in Transformer-based methods. In this article, we propose a novel approach for lightweight SR, called densely connected transformer with linear SA (DCTLSA) network. Our method addresses the efficiency issue of SA by designing a new linear SA (LSA), which calculates the similarities in spatial dimension with linear complexity. Moreover, we leverage dense connections to integrate multiple levels of features and provide rich information for SR. Our experimental results demonstrate that DCTLSA outperforms state-of-the-art lightweight SR methods in terms of SR performance, model complexity, and inference speed. The code of the proposed method is available at https://github.com/zengkun301/DCTLSA.