MiNL: Micro-Images based Neural Representation for Light Fields

With the freedom and flexibility, light field has shown its great potential in a wide range of applications such as virtual reality (VR) and augmented reality (AR). Traditional representations for light fields can be separated into two types: explicit representation and implicit representation. Unlike explicit representation that represents light fields as Sub-Aperture Images (SAIs) based arrays or Micro-Images (MIs) based lenslet images, implicit representation treats light fields as neural networks, which is inherently a continuous representation in contrast to discrete explicit representation. In this paper we propose MiNL, a novel MI-wise implicit neural representation for light fields. Given the micro-image’s coordinate, MiNL outputs the corresponding micro-image’s RGB values. Light field encoding in MiNL is just training a neural network to regress the microimages and the decoding process is a simple feedforward operation. With such a representation, all information of light fields are stored in parameters of neural networks, which can realize several light field-related tasks at the same time. For example, compared with mainstream light field compression methods, our proposed method transform the light field compression task into model compression task and can achieve comparable performance with state-of-the-art methods, with about 1~2dB PSNR improvement over HEVC/H.265 at the same bit rate. In addition to light field compression, MiNL can also be generalized to light field denoising, with more than 7dB PSNR improvement over original noisy light field images for different kinds of noises.