A Collaborative Perception Network based on Dynamic Multi-scale Fusion

Collaborative perception can improve perception performance by aggregating perception information from different perspectives of multiple agents, while solving the problems of obstacle occlusion or limited perception distance that may occur in single agents. However, when facing the inevitable transmission delays and localization errors in real-world communication, existing collaborative perception methods cannot effectively solve the problem of temporal-spatial misalignment, leading to serious decline in detection performance and robustness. In this paper, we propose a novel collaborative perception framework DynMSF(Dynamic Multi-Scale Fusion), that utilizes multi-scale strategies and dynamic information fusion to enhance both of the temporal and spatial robustness and improve the detection precision. Firstly, we introduce multi-scale collaboration (MSC) module, which collaborates on the perception information of agents at multiple scales to obtain spatial correlations at different scales, eliminating the negative effects caused by spatial misalignment. On the basis of multi-scale collaborative features, we propose a dynamic temporal fusion (DTF) module that dynamically fuses historical frame features stored in memory banks, enhances the feature and compensates for the transmission delay of the current frame. We conduct experiments on publicly available OPV2V and V2XSet datasets, and our model achieves the best performance compared to the baseline of existing methods. We also verify the strong temporal-spatial robustness of our model and the effectiveness of our proposed modules through noise robustness experiments and ablation study.