Evaluating Optimization Approaches for Deep-Reinforcement-Learning-based Navigation Agents

In recent years, Deep Reinforcement learning has made remarkable progress in various application areas such as control of robots and vehicles, simulation, and natural language processing. In recent years, various research works applied DRL to conduct different kinds of tasks for autonomous navigation of vehicles and robots such as lane changing, cruise control, or obstacle avoidance. However, DRL training is still a tedious and difficult process due to long training times, catastrophic forgetfulness and the myopic nature of DRL agents. In this paper, we integrate and explore the effect of a variety of state-of-the-art optimization approaches for DRL agents such as imitation learning, behavior cloning, frame stacking, and hindsight replay for autonomous obstacle avoidance. We evaluate the effect of each of these changes to the DRL agent in terms of training- as well as navigational performance. The resulting agents are compared against baseline approaches without those optimizations and we found an increase in navigational performance for some of these methods while other optimization approaches surprisingly resulted in decreased performing. The findings of this paper should aid in development of DRL approaches for autonomous navigation approaches.