High-Resolution Projection Network combining High-Resolution Optical Flow Compensation for Video Super-Resolution

The Super-Resolution (SR) task is to generate high-resolution (HR) images/videos using low-resolution (LR) ones. At present, the Single Image Super-Resolution (SISR) methods have achieved superior performance. However, Video Super-Resolution (VSR) methods still have some disadvantages such as edge blurring, high-frequency details missing and motion artifacts. This paper proposes an end-to-end HR feature projection VSR network HOF-HRPN, which consists of a High-Resolution Optical Flow (HOF) and a High-Resolution Projection Network (HRPN). The HOF network estimates the HR optical flow and compensates it to the neighboring frames to achieve accurate frame alignment. The HRPN is composed of a multi frame feature projection channel and a single frame SR channel in parallel. HRPN takes into account the advantages that single frame SR and VSR can extract missing high-frequency details from intra-frame and inter-frame, respectively. It fuses high-frequency details obtained from multi-scale LR projection learning and single frame SR results. A large number of comparative experiments based on public datasets verify that HOF-HRPN is robust and can recover accurate pixel values, clear edges and rich textures.