FICNet: An End to End Network for Free-View Image Coding

Free-view image compression has attracted the gaze of people due to the rapid development of 3D vision applications. However, as far as we know, no end-to-end learned compression model is proposed for free-view image sequences. Most existing learned compression models are limited and only applicable to image sequences with simple horizontal and vertical translations, such as stereo and light field image compression models. In this paper, we first propose an end-to-end network FICNet to improve free-view image compression performance, effectively eliminating the spatial redundancy among multiple views. In our methods, a differentiable depth prediction module is introduced to our model for exploring spatial correlation and achieving end-to-end training. Besides, we demonstrate a strategy of multi-view reference to alleviate the hole problem in depth-based prediction, and a filter network is designed to improve the prediction accuracy further. A residual fusion network with multi-level complementary features is also utilized to enhance the reconstruction quality. Extensive experiments show that our model can perform favorably in generating more refined predictive images and achieves up to a 16.23% BD-rate improvement compared to the state-of-the-art method 3D-HEVC.