Autonomous Driving in Unity Simulated Environments Using End-to-End CNN Approach

Autonomous driving has recently gained significant popularity in industry and academia. The current study of autonomous driving can be divided into two main branches: modular approaches and end-to-end approaches. In the case of end-to-end approaches, Imitation learning/Behavior cloning (IL/BC) and Reinforcement learning (RL) are two prevalent methodologies. Our paper conducted end-to-end CNN approach to autonomously driving virtual vehicle in Unity simulated environments, in which road conditions were randomly generated and weather conditions could be adjusted respectively. In order to optimize the output commands, behavior cloning was implemented to minimize the mean squared error between the generated commands and the recorded desired commands. Three prevailing CNN network were employed with weights either being initialized randomly or pretrained on ImageNet and their performance was compared. The dynamic evolution of model loss (MSE) is visualized using matplotlib. For future improvement, more obstacles or pedestrians should be introduced to better grasp the sophistication of driving in modern environments. Independent loss functions may as well be specially designed to balance the motion commands.