CFSA: An Efficient CPU-FPGA Synergies Accelerator for Neural Radiation Field Rendering

Neural Radiance Fields (NeRF) have great potential applications in 3D spatial modeling, scene reconstruction, and related fields, such as AR/VR. However, the compute-intensive and memory-intensive characteristics of NeRF present significant obstacles for real-time applications. Therefore, this work presents CFSA, a CPU-FPGA synergies hardware accelerator that enhances the deployment of NeRF in resource-constrained environments. A multilayer perceptron (MLP) processor has been designed on an FPGA with 89% utilization for the multi-resolution hash encoding NeRF algorithm. Additionally, a grid occupancy and early termination computation unit has been created, which reduces computational effort by over $\mathbf{9 0 \%}$. An overall pipeline scheduling scheme for the CPU and FPGA has also been designed. The experimental data shows that the accelerator can render synthetic data at 7.1 FPS on the Xilinx VC709 board with 100 MHz. Compared to ICARUS, a specialized architecture for NeRF-based rendering, there is a $338.1 \times$ speed improvement and a $8.4 \times$ increase in energy. Compared to the GTX 1080 Ti GPU, there is a $1.3 \times$ speed improvement and a $20.2 \times$ increase in energy.