Multi-Level Video Frame Interpolation: Exploiting the Interaction Among Different Levels

This paper proposes a novel multi-level frame interpolation scheme by exploiting the interactions among different levels. The proposed scheme includes three major stages that work at block level, pixel level, and sequence level, respectively. Effective algorithms are designed for each stage, i.e., block-level motion estimation with dropping unreliable motion vectors, pixel-level motion vector-guided partial scale-invariant feature transform flow matching, and sequence-level 3-D total variation regularized completion. Compared to traditional methods that focus mostly at one single level, the proposed scheme manages to recognize and utilize the interactions among the three levels based on their distinct characteristics and intertwined relationships. With a proper exploitation of interactions, unique advantages for each level can be effectively preserved while inherent limitations of a given level can be overcome by utilizing information from other levels. Extensive experiments have confirmed its superior performance over several classical schemes, in both subjective visual quality and objective peak signal-to-noise ratio/structure similarity measurements, and typical artifacts can be significantly reduced.