SFFTNet: Sparse Feature Fusion Transformer Network for Image Deblurring

The U-Net structure, with its an encoder-decoder architecture, has been widely adopted by many deep learning methods for image deblurring. Most methods concentrate on the design of encoder and decoder block and use skip connections to connect them. However, this simple skip connection strategy is insufficient to fully exploit the correlation of multi-scale features, which can result in a potential loss of deblurring performance. To address this issue, we design an effective Sparse Feature Fusion Transformer capable of integrating multi-scale features to replace the skip connections in the U-Net framework. Specifically, We propose a cross-attention mechanism with a learnable top-k selection operator to adaptively preserve highly correlated cross-attention values for feature fusion. This approach ensures that the fused multi-scale features effectively integrate contextual information, resulting in high-quality image deblurring. Additionally, we introduce the position encoding generator scheme to ensure that our deblurring network can be applied to images of any size. Comprehensive experimental results demonstrate that our proposed method outperforms the the state-of-the-art methods.