A Novel Hardware Accelerator of NeRF Based on Xilinx UltraScale and UltraScale+ FPGA | IEEE Conference Publication | IEEE Xplore

A Novel Hardware Accelerator of NeRF Based on Xilinx UltraScale and UltraScale+ FPGA


Abstract:

Neural Radiance Fields (NeRF) has shown its superiority in various fields, including 3D reconstruction and inverse rendering. However, due to its high computational deman...Show More

Abstract:

Neural Radiance Fields (NeRF) has shown its superiority in various fields, including 3D reconstruction and inverse rendering. However, due to its high computational demands, NeRF typically requires implementation on high-performance GPUs, which is costly and power-intensive, thus limiting its applicability. Compared to GPUs, FPGAs offer a potential solution to implement NeRF at lower cost and power. However, FPGA-based NeRF designs are still rare. To address this issue, a novel hardware NeRF accelerator based on Xilinx UltraScale and UltraScale+ FPGA has been proposed. The proposed design is based on the Multiresolution Hash Encoding NeRF algorithm and comprises Feature Reader, Multilayer Perceptrons (MLP), and Volume Renderer. The Feature Reader and MLP calculate the coordinates and directions of the sampling points on the rays, which are then used to determine the colors and densities of these points. To prevent timing problems and routing congestion caused by high DSPs utilization, we use registers and cascade paths between DSPs to construct MACC matrices in MLP. The Volume Renderer processes these colors and densities to obtain the colors of rays. The proposed accelerator runs at 300MHz on Xilinx Alveo U250, achieving an average 1.60 × performance improvement compared to NVIDIA GTX 1080 Ti. Additionally, the accelerator has reduced energy consumption per image by an average of 5.21x and 4.64x compared to NVIDIA GTX 1080 Ti and NVIDIA RTX 3090, respectively.
Date of Conference: 04-08 September 2023
Date Added to IEEE Xplore: 02 November 2023
ISBN Information:

ISSN Information:

Conference Location: Gothenburg, Sweden

Funding Agency:


References

References is not available for this document.