Learning-Based Temporal Sequence of Constrained Handling Selection for Constrained Multi-objective Evolutionary Optimization

Constraint-handling techniques and genetic operators are two crucial components in constrained multi-objective evolutionary algorithms (CMOEAs). Recent research in most of CMOEAs has primarily focused on adaptive designs of these components to address various constrained multi-objective optimization problems (CMOPs). However, the evolutionary process of solving a CMOP can involve various characteristics, such as continuity, discreteness, degeneracy, or some combination thereof, necessitating the tailored selection of constraint-handling techniques and genetic operators across different generations. This study conceptualizes these selections as a temporal sequence of constrained handling selection, where the time means the generation number. We argue that discovering the systematic patterns within the sequence based on the historical data of applying different selections significantly improves the performance of CMOEAs in finding Pareto optimal solutions. Based on this conceptualization, we propose a CMOEA with a deep reinforcement learning model for solving CMOPs. Specifically, the deep reinforcement learning model dynamically refines the selection of constraint-handling techniques and genetic operators for upcoming generations by learning from the performance of previous selections, thereby enhancing the predictive accuracy for subsequent selections. Experiments are conducted to validate the performance of the proposed algorithm against nine CMOEAs on thirty-seven benchmark problems and an unmanned aerial vehicle path planning problem. Experimental results show that the proposed algorithm substantially outperforms the compared algorithms regarding the obtained Pareto optimal solutions. Additionally, the results verify that discovering the systematic patterns within the sequence for CMOEAs has a positive impact on solving CMOPs in terms of objective optimization and constraint satisfaction.