A Mobile Semantic LiDAR SLAM Processor with AI-based 3D Perception and Spatio-temporal-aware Computing

A low-power artificial intelligence (AI)-based semantic LiDAR SLAM processor is proposed to expand autonomous driving into emerging mobile robots. It combines point neural network (PNN)-based 3D segmentation with LiDAR SLAM to minimize pose errors due to the lack of perception in previous SLAM. The proposed processor is designed with a heterogeneous multi-core architecture utilizing SIMD and reconfigurable processing elements to fully support the three main operations: k-nearest neighbor (kNN); PNN; and non-linear optimization. It accelerates kNN operations with spherical bin partitioning optimized for the distribution of LiDAR data to eliminate unnecessary search spaces. In addition, the proposed spatio-temporal-aware computing minimizes excessive memory overhead and workload imbalance in kNN and PNN operations. Consequently, fabricated with 28-nm CMOS technology, the processor achieves 8.245 mJ/frame of energy consumption and a maximum performance of 20.7 ms latency, successfully demonstrating real-time semantic LiDAR SLAM system with 99.86% lower power consumption compared to modern CPU+GPU platforms.