Performance comparison of NSGA-II and NSGA-III on various many-objective test problems

Recently NSGA-III has been frequently used for performance comparison of newly proposed evolutionary many-objective optimization algorithms. That is, NSGA-III has been used as a benchmark algorithm for evolutionary many-objective optimization. However, unfortunately, its source code is not available from the authors of the NSGA-III paper. This leads to an undesirable situation where a different implementation is used in a different study. Moreover, comparison is usually performed on DTLZ and WFG test problems. As a result, the performance of NSGA-III on a wide variety of many-objective test problems is still unclear whereas it has been frequently used for performance comparison in the literature. In this paper, we evaluate the performance of NSGA-III in comparison with NSGA-II on four totally different types of many-objective test problems with 3-10 objectives: DTLZ1-4 problems, their maximization variants, distance minimization problems, and knapsack problems. We use two different implementations of NSGA-II and NSGA-III. We show through computational experiments that NSGA-III does not always outperform NSGA-II even for ten-objective problems. That is, their comparison results depend not only on the number of objectives but also on the type of test problems. The choice of test problems has a larger effect on their comparison results than the number of objectives in our computational experiments. We also demonstrate that totally different results are obtained from different implementations of NSGA-III for some test problems.