Expectation-Maximization Attention Cross Residual Network for Single Image Super-resolution

The depth of deep convolution neural network and self-attention mechanism is widely used for the single image super-resolution (SISR) task. Nevertheless, we observed that the deeper network was more hard to train and the self-attention mechanism is computationally consuming. Residual learning has been widely recognized as a common approach to improve network performance for deep learning, but most existing methods did not make the best of the learning ability of deep CNN, thus hindering the ability of representative CNN. In order to tackle these problems, we introduce a deep learning network namely expectation-maximization attention cross residual network (EACRN) to tackle the super-resolution task. Particularly, we propose a cross residual in cross residual (CRICR) structure that makes up very deep networks consisting of multiple cross residual groups (CRG) with global residual skip connections. Every cross residual group (CRG) consists of some cross residual blocks with cross short skip connections. At the same time, CRICR allows network focused on capturing high-frequency patterns by connecting rich low-frequency patterns to be bypassed and several short skip connections. In addition, we introduce various convolution kernel size so that adaptive capture the image pattern in different scales, which make these features get the more efficacious image information through interacting with each other. The introduced Expectation-Maximization Attention (EMA) module is robust to the variance of input and is also friendly in memory and computation. Extensive experiments demonstrate our EACRN obtains superior performance and visual effect relative to the most advanced algorithm.