Joint Video Denoising and Super-Resolution Network for IoT Cameras

Internet of Things (IoT) cameras have widely been deployed over the last few years. These cameras are often with limited hardware so that they can only capture noisy videos in low resolution. In this work, we propose the joint video denoising and super-resolution network for IoT cameras, which consists of the noise-robust moving-attention (NRMA) module and the noise-eliminated upsampling (NEU) module. In NRMA, we adopt a coarse-to-fine approach by first extracting the coarse flow and then refining through bi-directional feature propagation among adjacent frames. In NEU, we further utilize inner-frame features for noise-elimination and upsampling. Through this approach, we avoid the negative effects brought by applying denoising and super-resolution in tandem, and enhance the reconstruction of moving objects by the embedded attention layers in NRMA. We conduct our experiments on both synthetic data sets, which utilize existing data with additive white Gaussian noise (AWGN), and a realistic data set captured using a pair of IoT and professional cameras. Our extensive experimental results demonstrate that our proposed method significantly reduces noise and enhances detail in both types of data sets. Notably, our approach outperforms the state-of-the-art benchmark (RealBasicVSR) by an average of 5.24 dB on the existing data sets (with noise level $\sigma{=}20$ ) and by 0.95 dB on the realistic data set in terms of peak signal to noise ratio.

I. Introduction

Internet of Things (IoT) cameras have widely been developed and deployed over the last decade. To support a range of anywhere anytime applications (e.g., video surveillance) in a cost-effective way, a massive amount of IoT cameras are geo-distributed, but each IoT camera is with low cost. However, due to the limited hardware (bottlenecked by the cost of photosensor), significant noise is perceived (especially during the night [1]) and the video resolution is low. With these noisy and low-resolution (LR) videos, the perceived quality is far from satisfactory. Even if the majority of the footage could be less useful, in the case of an incident (e.g., missing person, traffic accident, etc.), there is a need to recover noise-free and high-quality video to aid the investigation of the incident.