Exploring Highway Overtaking and Lane Changing Based on Soft Actor Critic for Discrete Algorithm

Highway is one of the important scenarios faced by autonomous driving. However, in this scenario, smart cars need to meet the requirements of high efficiency (i.e. high speed) and high safety, and provide passengers with high comfort (i.e. fewer lane changes), which poses a serious challenge to autonomous driving. This work uses highway-env as the simulation platform and proposes a discrete action based SACD (soft actor critical for discrete) algorithm that combines CNN(convolutional neural networks) and LSTM(long short term memory) to solve the safety, efficiency, and comfort issues of smart cars during highway driving. The simulation results indicate that although smart cars have high driving efficiency, they perform poorly in terms of safety and comfort. Based on this, this work adopts the idea of distributed sampling and proposes a dual buffers structure combined with safety module. The final simulation results show that the introduction of dual buffers combined with safety module architecture can significantly improve the safety and comfort of smart cars driving on highway while maintaining high efficiency.