A Mobile Semantic Lidar SLAM Processor With Artificial-Intelligence-Based 3-D Perception and Spatiotemporal-Aware Computing

A low-power artificial intelligence (AI)-based semantic lidar simultaneous localization and mapping (SLAM) processor is proposed to expand autonomous driving into emerging mobile robots. It combines point neural network (PNN)-based 3-D segmentation with lidar SLAM to minimize pose errors due to the lack of perception in previous SLAM. The proposed processor is designed with a heterogeneous multicore architecture that utilizes single instruction, multiple data and reconfigurable processing elements to fully support its three main operations: k-nearest neighbor (KNN), PNN, and nonlinear optimization. It accelerates KNN operations with spherical bin partitioning optimized for the distribution of lidar data to eliminate unnecessary search spaces. In addition, the proposed spatiotemporal-aware computing minimizes excessive memory overhead and workload imbalance in KNN and PNN operations. Consequently, fabricated with 28-nm CMOS technology, the processor achieves 8.245 mJ/frame of energy consumption and a maximum performance of 20.7-ms latency, successfully demonstrating real-time semantic lidar SLAM system with 99.86% lower power consumption compared to modern CPU+GPU platforms.