RapidSim: Enhancing Robotic Simulation with Photorealistic 3D Environments via Smartphone-Captured NeRF and UE5 Integration

To address the challenge of bridging the gap between the virtual and real-world environments in robotics, this study proposed an innovative approach that leverages Luma AI’s Neural Radiance Fields (NeRF) and Unreal Engine 5 (UE5). More specifically, standard smartphone cameras were used to capture video footage and efficiently create photorealistic 3D virtual environments. These environments were then integrated with UE5 to simulate realistic robotic navigation and manipulation tasks. The method’s effectiveness was validated through objective evaluations using Navigation2 for operational efficacy and YOLO for accurate object recognition in both virtual and real settings. A significant reduction in the time and resources required for environment creation compared to existing methods like NeRF2real and Matterport3D was demonstrated. This advancement facilitates simulation-driven development and testing in robotics and establishes a new standard for virtual environment generation.