Edge-constrained Residual Semantic Attention for Image Restoration

Currently, image restoration models have achieved significant advancements. However, in cases where the missing regions are too large, it often leads to erroneous structures and inconsistent blurry textures with the surrounding areas. To further enhance the performance of image restoration algorithms, we propose a two-stage model—based on edge constraints and residual semantic attention for image restoration. In the first stage, an edge structure generation network is employed to learn and restore missing edge structure information. Subsequently, in the second stage, the complete edge structure information assists a content-texture restoration network in generating the entire image. This involves initially using gated convolutions to extract effective features. Furthermore, a Residual Semantic Attention module (RSA) is designed, replacing regular convolutional kernels with regions known from the image itself. It applies residual combination and additional convolution to enhance the semantic coherence of foreground and background regions while maximizing the retention of original feature information.