Comparative Analysis of Automated Machine Learning for Hyperparameter Optimization and Explainable Artificial Intelligence Models

Artificial intelligence (AI) has been increasingly applied to solve complex real-world problems. One of the most significant challenges in AI lies in selecting and fine-tuning the optimal algorithm for a given task. Automated Machine Learning (AutoML) models have emerged as a promising solution to address this challenge by systematically exploring hyperparameter spaces to identify optimal configurations efficiently. This study addresses critical gaps in the current literature by conducting a comprehensive comparative analysis of AutoML frameworks for hyperparameter optimization and evaluating the effectiveness of various explainability techniques for enhancing model interpretability. For this purpose, Random forest (RF) is selected as the base model and integrated with nine different AutoML frameworks, namely Random search (RS), Grid search (GS), Hyperopt, TPOT, Optuna, GP Minimize, Forest Minimize, GBRT Minimize, and Dummy Minimize. The study focuses on predicting the ultimate moment capacity of Ultra-High-Performance Concrete (UHPC) beams and U-shaped girders. Furthermore, the insights from SHapley Additive exPlanations (SHAP) are also compared with those derived from alternative explainability methods, including Local interpretable model-agnostic explanations (LIME), partial dependence plots (PDP), and sklearn permutation importance rankings to examine the contributions of individual parameters to the ultimate moment capacity predictions of UHPC beams using the best-performing AutoML model. The findings demonstrate that Optuna consistently outperforms its counterparts, achieving the highest predictive accuracy and the lowest computational training time. The findings also highlight SHAP’s superiority in offering detailed, consistent, and actionable insights, making it the preferred method for both global feature importance and individual feature analysis in high-stakes engineering applications.