A Motion Refinement Network With Local Compensation for Video Frame Interpolation

Video frame interpolation (VFI) is a challenging yet promising task that involves synthesizing intermediate frames from two given frames. State-of-the-art approaches have made significant progress by directly synthesizing images using forward optical flow. However, these methods often encounter issues such as occlusion and pixel blurring when handling large motion scenes. To address these challenges, this paper proposes a novel approach based on the concept of local compensation. By adopting this approach, more refined optical flow estimation can be obtained, leading to higher-quality video frame interpolation results. Specifically, we introduce two modules, namely the Comprehensive Contextual Feature Extraction (CCFE) module and Motion-Guided Feature Fusion (MGFF) module, to enable local compensation of optical flow estimation. The CCFE module is designed to be embedded in each layer of the image pyramid structure. It aims to encourage the model to extract clean and sufficiently rich contextual information from the input images. On the other hand, the MGFF can guide the multi-source features fusion based on motion features, making the feature fusion of moving objects more precise, thus providing local compensation for optical flow estimation. Extensive experimental results demonstrate that incorporating our proposed modules into the baseline network significantly enhances the performance of video frame interpolation.