JVCSR+: Adaptively learned video compressive sensing reconstruction with joint in-loop reference enhancement and out-loop super-resolution

Recently, deep learning-based video compressive sensing reconstruction (VCSR) technologies have significantly improved reconstructed video quality by taking advantage of spatial and temporal correlations. However, existing VCSR work mainly focuses on improving deep learning-based motion compensation without optimizing local and global information, leaving much room for further improvement. This paper proposes a novel VCSR method, JVCSR+, which focuses on optimizing feature information, removing reconstruction artifacts, and increasing the resolution simultaneously. Specifically, the measurement matrix in the proposed compressive sensing (CS) module can learn adaptively, so that sampled measurements retain more image structure information for better reconstruction. An average search module is also proposed to detect more suitable areas for references, thereby attaining superior motion compensation performance. Within the loop, the enhanced frame is utilized as a reference to improve recovery of the current frame. Furthermore, we propose an out-loop super-resolution module for VCSR to obtain high-quality images at low bitrates. The results of extensive experiments demonstrate that our proposed JVCSR+ obtains promising performance compared to state-of-the-art CS methods within the same bitrate range.