Worst and Generalized Severity Evaluation for Evolutionary Dynamic Multi-objective Optimization

The No Free Lunch (NFL) Theorem states there are no universally “best” optimization algorithms, including evolutionary dynamic multi-objective optimization (EDMO) algorithms. Due to the lack of a universal optimizer at the theoretical level, the performance of EDMO algorithms rely heavily on experimental comparison of the specific problem. The evaluation of EDMO algorithms require care in designing representative test cases of the problem class, and comprehensive metrics incorporating convergence, diversity and dynamics at the same time. The literature review in EDMO reveals challenges in designing algorithms, developing efficient metrics and producing suitable test cases. In this paper, we focus on addressing the literature gap in developing more robust approaches to evaluating EDMO algorithms that have varying performance on different real-world DMOP problems. We propose a new metric to measure the worst-case performance of convergence, diversity and dynamics. We further extend this to formulate a three-layer generalized evaluation framework and two generalized metrics. Finally, we perform a case study on a transportation application with dynamic traffic changes, to evaluate our new proposed metrics and framework. Our experiments show that the proposed framework offers comprehensive evaluation and a better understanding of the compared algorithms' relative strengths at different levels.