Energy-Efficient Dynamic Vehicle Routing for Waste Collection by Q-Learning-Based Hyperheuristic Particle Swarm Optimization

During the process of waste collection, various unpredicted disturbances might occur. Meanwhile, vehicle transport is an important source of carbon emissions. In this study, an energy-efficient multi-trip dynamic vehicle routing model is established for waste collection, which introduces two types of dynamic events: new collecting requirements of waste sites and vehicle breakdowns. A Q-learning-based hyperheuristic particle swarm optimization (QLHPSO) is proposed as a dynamic rescheduling method to solve the model. A set of low-level heuristics (LLHs) are designed by combining the learning operators in particle swarm optimization and local search operators. A Q-learning-based high-level strategy is developed to find a suitable LLH for each evolutionary state based on historical performance of LLHs. When rescheduling is triggered, a response mechanism is incorporated to construct initial population by utilizing the features of dynamic events and historical elites. Extensive experimental results on one real instance and nine synthetic instances show that QLHPSO can react to the environmental changes rapidly, and reschedule the vehicle routes with lower cost and carbon emissions compared to the state-of-the-art algorithms.