Deep Reinforcement Learning with Fully Convolutional Neural Network to Solve an Earthwork Scheduling Problem

This paper proposes a deep reinforcement learning approach in order to optimize a sequence of tasks efficiently with the aid of image processing techniques used in computer vision. The proposed algorithm can be employed to solve the traveling salesman problem (TSP), a combinatorial optimization problem that determines the optimum trajectory of city visits so that the total traveling distance is minimized. The proposed algorithm accepts a set of images as an input, and outputs the priority over alternative tasks (or sites to visit) that should be conducted at the next time step. The proposed method applies a stacked convolutional network layer to effectively process and extract the meaningful features and uses a fully convolutional network to map the processed features to the output tasks without losing the local connectivity in the input images. The proposed algorithm has been employed to optimize the excavation schedule of a single digger for completing a 20 by 20 grid world, which is equivalent to the TSP problem with a node size of 400. The simulation results showed that the proposed method can achieve an effective schedule with optimality comparable to state of the art algorithms.