A fusion of graph and grid based hybrid model of object detection and semantic segmentation for 4D millimeter wave radar

The next generation 4D millimeter wave radar can provide rich information and dense point cloud, and perceive the environment under all-weather and all-operating conditions, making it very suitable for autonomous driving systems. However, the existing object detection and semantic segmentation models based on 4D millimeter-wave radar are usually directly transplanted from LiDAR, which cannot effectively adapt to the millimeter-wave radar point cloud features, resulting in poor detection performance. Concerning this, a hybrid model for 4D millimeter wave radar object detection and semantic segmentation is developed here via fusing graph and grid. A graph neural network module named radar adaptive multi-channel graph neural network (RAMGNN) is firstly designed, which leverages topology graphs and feature maps to propagate and update point cloud features. The node embeddings outputted from RAMGNN can be directly used for semantic segmentation and serve as a point cloud feature encoder for subsequent object detection. In what follows, the point cloud is projected onto a two-dimensional bird’s-eye view grid and its multi-scale features can be extracted exploiting a backbone network with channel attention mechanism. Finally, multi-scale features are fused to achieve effective object detection and semantic segmentation. Experimental results conducted on the publicly available dataset View-of-Delft demonstrate that the proposed model outperforms state-of-the-art algorithms in terms of both object detection performance and semantic segmentation quality.