Learning Neural Light Field with Projective Embedding Network

Neural Radiance Fields (NeRF) is a novel view synthesis method that uses neural networks for high-quality image rendering. However, it is slow when rendering due to numerous network evaluations per pixel. In contrast, Neural Light Fields (NeLF) improves rendering efficiency with a single evaluation per pixel, but suffers from poor multi-view consistency. Learning NeLF with Ray-Space Embedding (RSEN) introduces an embedding network to learn local affine transformations of rays, thereby significantly enhancing multi-view consistency and effectively modeling specular reflections and refractions. However, the two-plane parameterization used in RSEN is insufficient to represent 360° scenes. In this paper, we propose a ray re-parameterization method based on Plücker representation. We utilize an embedding network to learn multiple homography matrices of projective transformation, which are then used to map rays into a latent space. This approach helps the alignment and interpolation of rays. Moreover, block normalization is applied to enhance the internal correlation within these homography matrices and their relative independence from each other. Experiments on Realistic Synthetic 360° show that our method outperforms NeRF in both rendering speed and quality, R2L in modeling non-diffuse phenomena, and RSEN using Plücker representation in terms of rendering quality and convergence.