Bird’s-Eye View Perception Based on Two-Stage LiDAR Compensation in Complex Traffic Scenarios

Establishing 3-D perception capabilities for self-driving cars is a key research problem. Recent research has differentially “lifted” features from multicamera images onto a 2-D ground plane to produce a bird’s-eye view (BEV) feature representation of the 3-D space around the vehicle. However, this is currently challenging due to the inability to accurately reproduce the sizes and positions of truncated objects as well as drag tails and long tails in cameras with different fields of view. In this article, we propose a BEV sensing method based on two-stage light detection and ranging (LiDAR) feature compensation. First, the initial BEV features are obtained by fusing image features with LiDAR voxel features. Second, a two-stage LiDAR feature compensation method is proposed to synthesize the point-voxel features by using voxel features and point cloud features. This method also calculates the similarity between the initial BEV features and the point-voxel features to reject and replace feature points in the image features that have insufficient similarity with the point-voxel features on a large scale. Again, through the compensated BEV features, the BEV features with time series are input into the time-domain BEV feature fusion module, to query the same vehicle’s position, size, and other physical states at different times. Finally, the features are fed into tasks such as detection and segmentation to complete the output. In this study, a comparative validation is carried out on the BEV-aware dataset nuScenes. The experimental results show the effectiveness of truncated target detection and segmentation.