Dynamic Reservation of Edge Servers via Deep Reinforcement Learning for Connected Vehicles

Edge computing is promising for connected vehicles. As vehicles move, their resource demands for edge servers vary. Thus, it is necessary to reserve edge servers dynamically to meet variable demands. Existing schemes of edge-server reservation usually rely on statistical information of resource demands to make reservations; they are infeasible for connected vehicles, since such schemes are not adaptive to time-varying demands. To this end, a spatio-temporal reinforcement learning scheme called DeepReserve is developed to learn variable demands and then conduct edge-server reservation. Its design is based on the deep deterministic policy gradient algorithm of deep reinforcement learning (DRL), but is featured with several enhancements. First, the fully-connected neural network in DRL is replaced by a convolutional LSTM (ConvLSTM) network to extract spatio-temporal features of resource demands, which highly improves the prediction accuracy of resource demands. Thus, the actions in DRL (i.e., reservation decisions) can adapt to future demands. Second, an action amender is designed to ensure the actions selected by the neural network follow the spatio-temporal correlation. Finally, a training method called DR-Train is designed to stabilize the training procedure for different traffic patterns. DeepReserve is evaluated through extensive experiments on real-world datasets. Results show that it outperforms state-of-the-art approaches.