A LiDAR–PNN Pipelined Processor With Cylindrical Bin Partitioning and Halo Indexing for 3-D Perception in Outdoor Autonomous Driving Applications

With the rapid attention of real-time 3-D perception in outdoor autonomous driving applications, 3-D point-cloud neural network (PNN) using LiDAR data has been actively developed to capture the accurate semantic surrounding information. Previous 3-D PNN processors solely focused on the acceleration of algorithm structure [e.g., nearest neighbor searching, shared multi-layer perceptron (MLP), etc.] required due to the unordered and unstructured characteristics of point-cloud with the assumption that the entire point-cloud data are stored in external memory. However, they were impractical for real-time PNN in outdoor environments with LiDAR, because the LiDAR takes ~100 ms of sensing latency to obtain the 360° fully scanned LiDAR points, causing a waste of system latency and additional delays due to external memory accesses. Moreover, their Cartesian-based bin partitioning method exacerbates workload imbalance in outdoor applications due to spatial sparsity as the point distribution becomes increasingly sparse with distance from the sensor, causing severe performance degradation. To address these challenges, this paper proposes an efficient 3-D PNN processor with LiDAR–PNN processor pipelined structure that eliminates the external memory between LiDAR and processor and hides sensing latency behind processing time by utilizing the mechanical characteristic of LiDAR with the following hardware features: 1) cylindrical partitioning core with novel cylindrical bin partitioning method as well as halo indexing to process with the partially scanned ( $2{\theta }$ ) data and mitigate the PNN accuracy loss; 2) pseudo-random number generator-based sampling unit and k-nearest neighbor searching cores with unified neighbor searching algorithm to reduce the computation cost; and 3) linked-list memory management unit and predicted memory allocator to efficiently manage the point-cloud and support the parallel processing of neighbor searching and shared MLP. As a result, the proposed PNN processor achieves 2.18 M point/s of peak performance and $0.40~{\mu }$ J/point of energy with 108 K points.