Autonomous Driving via Brain-Inspired Causality-Aware Contrastive Learning With Time–Frequency Prediction

Developing trustworthy reinforcement learning (RL) agents for safety-critical control tasks, such as end-to-end autonomous driving, has been a longstanding challenge due to low sample efficiency. Prior works have attempted to address this challenge by performing self-supervised auxiliary tasks like self-reconstruction or predicting long-term future states. However, there still remain unexplored sequential features and causality relationships inherent in sequential state, action, and reward signals in the frequency domain. To fully exploit the temporal and frequential features, we propose a contrastive RL framework called brain-inspired causality-aware contrastive learning (BRYANT) to achieve efficient representation learning and human-like autonomous driving. Different from existing temporal predictive methods, we transform the sequential latent representations, reward, and action signals into the frequency domain, followed by the symmetric temporal prediction pattern for real and imaginary parts of the frequential signals. To capture the temporal causality for the latent representations, we introduce a brain-inspired network structure called Closed-form Continuous-time (CfC) network to parameterize the derivative of the latent representations and establish the neural dynamic model. Experimental results conducted in the CARLA simulator demonstrate the effectiveness of BRYANT in efficient representation learning, enabling agents to concentrate on potential risks and decrease the collision rate compared to several state-of-the-art RL methods. Furthermore, through the visualization of the latent representation prediction process, we reveal the causal relationships between the critic Q values and the latent representation vectors in the frequency domain, and demonstrate the effectiveness of the frequency-domain prediction.