Subjective Quality Driven Image Encoding Method Using Image Completion

This paper presents a still image coding method using deep learning-based image completion. Deep learning-based image completion can restore skipped areas of images in high quality. When we introduce image completion to image coding, it is possible to reduce the coded-bit amount compared with normative codec-based methods by replacing complex areas, such as textures, with simple signal values at the encoder and completing them at the decoder. However, there is no method for automatically detecting the skipped areas at the encoder because we cannot evaluate the quality of completion by objectively comparing the signal difference between the original and completed images. To resolve this issue, we propose an image quality estimation model without referencing original images. Our key idea is to obtain the model by adversarial training with a completion network assuming that original images have higher quality than the completed ones. We also propose a detection algorithm for skipped areas. Our algorithm detects skipped areas by giving priority to complex areas where a large coded-bit amount is required for the normative codec-based method to increase coding efficiency. The proposed method reduced the coded-bit amount by 25% compared with an HEVC-based method while maintaining the subjective quality for particular images.