Space-Mate: A 303.5-mW Real-Time Sparse Mixture-of-Experts-Based NeRF-SLAM Processor for Mobile Spatial Computing

Simultaneous localization and mapping (SLAM) provides crucial ego-pose information and 3-D maps of the user environment, which are fundamental to emerging mobile spatial computing devices. Dense 3-D mapping and accurate pose estimation are particularly necessary for applications like augmented reality (AR) and autonomous navigation. However, existing SLAM processors are typically limited to sparse 3-D maps due to constrained resources in mobile settings. This article presents space-mate, a low-power, real-time dense SLAM processor. At the algorithm level, a sparse mixture-of-experts (SMoE) based neural radiance field (NeRF) is introduced as a novel representation for dense 3-D mapping, optimized for mobile devices due to its reduced computational complexity and compact map size. Space-mate executes both tracking and mapping on a unified hardware with the following optimizations: 1) out-of-order SMoE (OoO-SMoE) router schedules MLP operations for multiple experts, enabling multi-batch processing and reducing data transactions; 2) heterogeneous coarse-grained sparse core (HCG-SC) accelerates various MLP training stages, including feed-forward (FF), back-propagation (BP), and weight-gradient (WG) calculation, enhancing energy efficiency; and 3) 2-D sampling unit (2D-SU) predicts and removes unnecessary mapping workload through loss prediction, significantly reducing energy per frame with minimal accuracy loss. Space-mate, fabricated in a 28-nm process, achieves 32.85 frames/s at 303.5-mW power consumption under a 125 MHz, 0.83 V operating condition. Compared to prior state-of-the-art SLAM ASICs, space-mate processes $30.72\times $ – $930.9\times $ more points per frame for tracking and $4.8\times $ – $7.2\times $ more points for mapping while consuming $2\times $ – $15.4\times $ less energy per pixel.