TUDF-NeRF: Generalizable Neural Radiance Field via Truncated Unsigned Distance Field

Novel view synthesis is a classical research problem in computer vision and has been well studied in various ways. While recently proposed NeRF model achieves photo-realistic novel view synthesis result, traditional NeRF can represent only a single scene and is not able to generalize to novel scenes, which limits its application. This paper proposes TUDF-NeRF, a generalizable neural radiance field representation for high-fidelity novel view synthesis. This is achieved by modeling the volume density as the function of truncated unsigned distance field (TUDF) and predicting radiance field by feature matching. Compared with the existing methods that generalize NeRF to novel scenes, the TUDF based representation enforces the rendering results to be influenced only by samples near the surface instead of the whole space, which reduces the ambiguity during feature matching over low-texture areas and improves view synthesis quality. To further improve the quality of view synthesis over arbitrary views on novel scenes, this paper design a per-scene fine-tuning scheme that utilizes the gated recurrent unit (GRU) to fuse local information for global optimization. Extensive experiments demonstrate that TUDF-NeRF outperforms state-of-the-art view synthesis methods, and photo-realistic view synthesis over arbitrary view direction is achieved after per-scene fine-tuning.