Explainable Deep Adversarial Reinforcement Learning Approach for Robust Autonomous Driving

Deep Reinforcement Learning (DRL) has made promising progress in autonomous driving planning and guidance within dynamic urban scenarios. As the potentials for real-world applications increase, so does the demand for a safe and robust driving system. For example, an end-to-end deep reinforcement learning agent makes guidance decisions based on the observation states extracted from perception sensor inputs, which can be interfered by unpredictable adversarial attacks. The latter causes adversarial observation states, which easily leads autonomous driving agent to incorrect decisions and ultimately to unintended accidents. In this work, we propose both attack and defence approaches for robust learning-based self-driving agent. The optimal observation perturbation is realized using an efficient augmented gradient-based method. An attack detection deep network with saliency map based explainability is then proposed to flag up to the users the existing danger of the attacks on the sensor perception. Furthermore, to ensure safe driving under these perceptional perturbations, we propose a deep adversarial reinforcement learning based approach for robust autonomous driving in a roundabout passing scene. We adopt PPO (Policy Proximal Optimization) [1] as our baseline guidance algorithm and develop a theoretical supported constraint, multi-object objective function optimization to efficiently mitigate the effect on the deep guidance autonomous driving policy from strong adversarial attacks. We conduct extensive experiments and evaluate our robust model under various adversarial attack configurations in traffic scenarios. Our proposed method shows significant improvements coping with optimal adversary in dynamic environments.