Polygon Mesh Recovery via Segmentation Priors and Neural Radiance Fields

Neural Radiance Fields (NeRF) has made significant advancements in view synthesis and 3D reconstruction. However, the reconstructed outputs often lack semantic information, limiting its applicability for scene understanding tasks. Additionally, NeRF-based implicit representations focus on novel view synthesis, leading to coarse, non-editable surface meshes. This paper addresses the asset generation task within NeRF, incorporating prior semantic information to guide the reconstruction process. We propose a texture mesh recovery method that combines learned segmentation priors with NeRF. By leveraging the Segmentation Anything Model (SAM) for extracting object masks from RGB images, we ensure semantic consistency. Camera poses, derived from a multi-view reconstruction algorithm, are integrated into a mesh-based NeRF framework to learn explicit scene representations. Using an optimization approach, we disentangle texture and mesh, thereby enabling the creation of editable 3D assets. Experimental results demonstrate the effectiveness of the proposed method across multiple reconstruction datasets.