RMemVD: A Simple Restricted Multi-Memory Bank for Video Deblurring

Video deblurring relies on leveraging information from adjacent frames in the video sequence to restore the blurred regions in the current frame. The previous methods mainly employ Convolutional Neural Network (CNN) encoder-decoder architectures, Recurrent Neural Network (RNN) feature propagation, attention mechanisms, or their combination to leverage video sequence information. However, these methods typically employ information from only a portion of the input frames and are generally unable to effectively capture the complex spatiotemporal relationships present in blurry videos. To address these issues, we construct a compact and effective pixel-level feature memory bank to filter and store all temporal frame information and design a specific memory update algorithm for refreshing it. To enrich the memory bank, we also introduce the semantic-level feature memory and sensory memory. The former is used to enhance the model’s attention on semantic information in the video, enabling it to aid in final pixel-level deblurring and the latter compensates for the temporal locality deficiency in the memory bank. Furthermore, we design a semantic-level cross-attention module for extracting semantic information and a cross-attention reinforcement module to enhance the retrieved pixel-level and semantic-level memory features. Qualitative and quantitative experimental results demonstrate that our model achieves more competitive performance. Especially, it surpasses the CDVD-TSPNL+ in terms of Peak Signal-to-Noise Ratio (PSNR-it is used to measure the pixel-wise differences between two video frames, with higher values indicating better performance) by over 0.25dB on the GOPRO dataset. The source code is available at https://github.com/chuang-lq/RMemVD.