Layered Conceptual Image Compression Via Deep Semantic Synthesis

Motivated by the insight of Marr on generative image representations, we propose a layered conceptual image compression scheme by integrating the advantages of both variational auto-encoders (VAEs) and generative adversarial networks (GANs). In particular, the image is represented by two layers: the low-dimensional codes of the stochastic textures encoded by the VAE and the geometric structures characterized by edge maps. Subsequently, the edge maps and latent codes are compressed individually such that the final bit streams are formed in a combined manner. At the decoder side, the GAN synthesizes the decoded images on the basis of the latent codes and the reconstructed edge maps. Experimental results demonstrate that our proposed scheme achieves better visual reconstruction quality than the traditional image compression algorithms such as JPEG, JPEG2000 and HEVC (intra coding) in the low bit rate coding scenarios.

1. INTRODUCTION

The popular lossy image and video compression standards (e.g., JPEG [1] and HEVC [2]) adopt the block-based compression architecture to remove the redundancy. Despite the great success achieved over the past decade, they employed signal level visual representations without fully considering visual perception. Following the insight of Marr [3], geometric structures (e.g., edges and ridges) and stochastic textures are two prominent components that compose the visual scene. As such, it is highly expected that the compression algorithms follow such representations of an image, which can greatly promote the coding efficiency and representation capability.