A Deep Reinforcement Learning Method for Autonomous Driving Integrating Multi-Modal Fusion

Due to the input of high-dimensional data in end-to-end autonomous driving, if the feature extraction network is trained online from scratch, it will lead to difficulties in learning and decision-making. To solve this challenging problem, this paper proposes a predefined multi-modal image method to improve decision-making accuracy and a shared framework to quickly train all networks. The inputs are composed of various observations including bird’s-eye view, state, and front view, rather than traditional sensor raw data. They can be filtered out ineffectual features based on human prior knowledge to avoid over-exploration with invalid changes, such as light, trees, mountains, etc. Combined with the processed representations, predefined images are formed by embedding the trajectory, forbidden line, etc. The method can assist agents to capture the coupling relationship between the actions and states, allowing networks to positively adjust weights to quickly obtain expected rewards. Considering that embedding a feature extraction network into reinforcement learning networks will lead to repeated cumulative calculations, a shared framework is proposed to independently compress features while jointly participates in online training to reduce computing costs. This module dynamically combines the policy and critic to update its weights, which overcomes decision-making problems caused by inaccurate the latent feature sequence during pre-training and fine-tuning methods. Finally, the interactive environment is constructed on a realistic driving simulator CARLA, and the fusion of different modal states is explored. The results showed that the multi-modal fusion can explore to earn maximum rewards, and the proposed methods are effective for the training.