Optimizing Hyperparameters for Deep Learning Models Using Evolutionary Algorithms: Solving the Four-Class Intertwined Spiral Classification Problem

This paper presents novel approaches to optimizing Deep Neural Networks (DNN) for the four-class intertwined spiral classification problem using an Evolution strategy algorithm with 1/5 success rule and a Genetic Algorithm (GA) implemented via the DEAP framework. Unlike the simpler two-class spiral, this four-class variant remains largely unexplored in machine learning fields. Evolutionary algorithms can provide a dynamic and adaptive mechanism that optimizes hyperparameters of DNNs for non-linear classification tasks and this study leverages ES(l + 1) with the 1/5 success rule and DEAP based Genetic Algorithms. Without hyperparameter optimization, the baseline model achieved an average accuracy of 34.1 % with an average loss of 1.32. GA with DEAP optimizing hyperparameters achieved the best classification accuracy of up to 96.2%. ES(l + 1) with the 1/5 success rule found a model that delivers 94.1 % accuracy. These findings establish evolutionary algorithms as powerful tools for enhancing DNN performance and provide valuable insights for future advancements in deep learning optimization.