Research on multimodal information fusion dynamic target recognition method for autonomous driving

In order to improve the ability of autonomous driving to sense the surrounding environment and promote the development of autonomous driving technology, this paper proposes a multimodal information fusion dynamic target recognition method for autonomous driving. First, in order to solve the distributed modal gap caused by the heterogeneity of different signals, this paper proposes to construct two different subspaces of modes for multimodal information. The first subspace is mode-invariant, in which cross-modal representations learn their commonalities and narrow the modal gap. The second subspace is modality-specific, private to each modality, and guarantees their characteristics. Second, in video signals, existing methods use completely independent approaches to individual modal signals or treat all modal signals equally using the same approach, which do not fully consider modal differences and exploit modal correlations. This section proposes a new RGB-D target recognition method capable of learning complementary functional modes from multiple and common features between different modes. A joint loss activation function is used for cascading multimodal information features. Experimental results show that the RD-DSST model proposed in this paper, which performs better on the publicly available dataset of KITTI, compared with Unet, YOLOv5 and Faster R-CNN, proving the effectiveness of the model proposed in this paper.