Examination of the Multimodal Nature of Multi-Objective Neural Architecture Search

Remarkable successes in deep learning have spurred significant growth in the field of neural architecture search (NAS), which is rapidly advancing as a promising technique for automating the design of network architecture. From an optimization standpoint, a NAS task for a given search space can be viewed as a multi-objective optimization problem (MOP) when considering multiple design criteria simultaneously (e.g., prediction accuracy, architecture complexity, hardware efficiency). However, whether a NAS problem is a multimodal multi-objective optimization problem or not (i.e., whether a single non-dominated solution in the objective space has multiple different neural network architectures or not) has not been examined in the literature. This presents an intriguing research question that merits further investigation. To fill this gap, we examine the multimodal nature of seven multi-objective NAS problems. By doing so, this work aims to help MOP researchers to better understand the characteristics of the multi-objective NAS problems.