Generating 3D Models for Prototyping of Virtual Environments using NeRF

In the current technological landscape, technologies such as Metaverse, 3D printing, AR/VR applications, and game development require 3D models to build the environment around us for greater immersion. Thus, there is a need to speed up the 3D modelling process. There exist 3D reconstruction models like Neural Radiance Fields (NeRF) and Structure From Motion (SfM). However, pure NeRF converges extremely slowly on the model as it is generic (i.e it works on all types of objects). A way to optimize the generation of 3D models is to take advantage of a standard structure of the desired object class itself. Hence, a model is proposed, consisting of SlowFast CNN for structural feature extraction, Edge Collapsing for rejecting outliers and reducing complexity, which can generate 3D models of any desired class of objects, by training the model on the desired class of objects. The method samples the points to generate a point cloud, identifies the structure to which it most strictly adheres and generates a consistent mesh from these points and the defined structure, and to further reduce complexity, uses Edge Collapsing algorithm to fit unusual points to their nearest consistent fit. For comparison with other 3D reconstruction methods, a CLIP Similarity Score of 0.542 was obtained in an average execution time of 3.27 minutes.