Joint Feature and Texture Coding: Toward Smart Video Representation via Front-End Intelligence

In this paper, we provide a systematical overview and analysis on the joint feature and texture representation framework, which aims to smartly and coherently represent the visual information with the front-end intelligence in the scenario of video big data applications. In particular, we first demonstrate the advantages of the joint compression framework in terms of both reconstruction quality and analysis accuracy. Subsequently, the interactions between visual feature and texture in the compression process are further illustrated. Finally, the future joint coding scheme by incorporating the deep learning features is envisioned, and future challenges toward seamless and unified joint compression are discussed. The joint compression framework, which bridges the gap between visual analysis and signal-level representation, is expected to contribute to a series of applications, such as video surveillance and autonomous driving.

I. Introduction

Recent years have witnessed an explosion of video big data, especially for the surveillance video [1] which is becoming the biggest big data in the digital universe. In particular, according to the prediction from NVIDIA [2], there will be 1 billion cameras deployed in 2020 producing extreme high-volume data in real-time. The large-scale visual data are of paramount significance for social security, smart city and intelligent manufacturing. However, it is usually impractical to rely on manpower only for the utilization of the video big data, especially in the scenario of safeguarding which requires real-time monitoring and rapid response. Recently, the advances of computer vision have substantially promoted the performance of visual analysis and enabled them to be applied in practical application scenarios. As such, in these circumstances, efficient management of the large scale visual data is highly desired.