Heterogeneous Multiscale Cooperative Perception for Connected Autonomous Vehicles via V2X Interaction

Connected autonomous driving and vehicle-to-everything (V2X) technology brings new opportunities for precise perception in occluded and sight-limited environments. The emergence of cooperative perception via V2X interaction positively impacts the safe and efficient driving of connected autonomous vehicles (CAVs). However, given the diversity and heterogeneity of V2X data, effectively processing such data to enhance cooperative perception remains a key and challenging task. This article introduces a heterogeneous multiscale coopera-tive perception (HM-CoPept) framework with bird’s eye view features. It pays more attention to crucial cooperative data that affects driving to avoid blocked areas and extend the sensing range. For the heterogeneity of V2X data, a bidirectional cross-attention is innovatively proposed to fuse LiDAR and camera data complementarily. Furthermore, the multiscale cooperation of V2X interaction data is proposed to break perception occlusion and limitation, considering the benefit of multiscale features with different spatial importance. Cooperative perception is enhanced by learnable spatial confidence weight of safety distance constra-int and foreground estimation. Test and validation are conducted on standard benchmarks, simulated (OPV2V) and real (DAIR-V2X). HM-CoPept results show that performance increases by more than 16% compared to single-vehicle perception. Through extensive experiments and critical analysis, we demonstrate that our approach advances competitive methods and state-of-the-art in average precision. HM-CoPept enables precise and broad perception in complex driving environments and promotes the intelligent and autonomous development of CAVs.