ANEC: Adaptive Neural Ensemble Controller for Mitigating Latency Problems in Vision-Based Autonomous Driving

Humans have latency in their visual perception system between observation and action. Any action we take is based on an earlier observation since, by the time we act, the state has already changed, and we got a new observation. In autonomous driving, this latency is also present, determined by the amount of time the control algorithm needs to process information before acting. This algorithmic perception latency can be reduced by massive computing power via GPUs and FPGAs, which is improbable in automobile platforms. Thus, it is a reasonable assumption that the algorithmic perception latency is inevitable. Many researchers have developed different neural network driving models without consideration of the algorithmic perception latency. This paper studies the latency effect on vision-based neural network autonomous driving in the lane-keeping task and proposes a vision-based novel neural network controller, the Adaptive Neural Ensemble Controller (ANEC) that is inspired by the near/far gaze distribution of human drivers during lane-keeping. ANEC was tested in Gazebo 3D simulation environment with Robot Operating System (ROS) which showed the effectiveness of ANEC in dealing with algorithmic latency. The source code is available at https://github.com/jrkwon/oscar/tree/devel_anec.