Inactive Region Filling Method for Efficient Compression Using Reinforcement Learning

Based on the massive advancements in modern hardware technologies, beyond high-resolution images, immersive and interactive videos have become an important next-generation technology. Immersive video includes 3D information having a huge number of bits. Thus, a special coding standard is required to compress immersive video e.g., 360 and point cloud videos. To compress immersive videos, the 3D information is projected into 2D space which creates several empty spaces called inactive regions. Even though the codec model can efficiently compress the immersive video, the inactive regions excessively degrade the overall performance. Recently, several model-based methods have suggested filling the inactive regions to prevent performance degradation. In this paper, we propose a novel approach to fill the inactive region using deep reinforcement learning. We perform block-wise inactive region filling in which each block has an agent, and overall pixel values are controlled by using a control point-based block-wise selective filtering method. The control point controls the pixel intensity to generate rough patterns and reduce computational complexity. In addition, each control point and the active region (attribute or pixel information) are propagated to the inactive region by considering their own information and neighboring information. Besides, each agent can learn the information of neighboring blocks with a modified rate-distortion (R-D) cost reward. We evaluate the proposed method on the sphere segmented projection (SSP), point cloud compression (PCC) attribute, and PCC geometry video formats. The proposed method achieves an average bitrate reduction of −10.9% for the SSP, −9.1% for the PCC-attribute, and −11.8% for the PCC-geometry, which are much better than the conventional method.