A System Based on Deep-Learning for Dynamic Routing problems

This paper analyzes the use of reinforcement learning with the Proximal Policy Optimization algorithm (PPO) in the context of the Dynamic Travelling Repairman Problem (DTRP). DTRP are routing problems in which one or multiple agents needs to optimize the processing time of dynamically generated requests. The study case of this paper is of a Unmanned Aerial Vehicle with no motion constraints and unlimited sensing that needs to service multiple targets in bounded environments, while minimizing the waiting time of each target. We analyze the performance of two different types of neural networks architecture, a feed-forward network and convolutional neural network (CNN). The performance of each network is analyzed in terms of the number of targets serviced and the average waiting time. Two heuristic policies, ‘Nearest-first’ (NF) and ‘First Generated First Served’ (FGFS), are used as baselines to compare the performance of the neural networks. The results show that CNNs perform better than a feed forward network. Differently from the feed forward network, the CNN network is able to capture the spatial features of the environment reducing the average waiting times of the targets. The CNN architecture also shows potential to perform as well as the heuristics. Further work is necessary to extend the proposed solution to other situations.