Hybrid Bayesian Optimization with Early Termination Strategies for Auto Tuning

Design Space Exploration (DSE) engines require a careful and iterative selection of hyperparameters for both model architecture and hardware design to ensure optimal performance during training and inference. However, conventional AI model tuning is computationally expensive and time-consuming, making it impractical in many scenarios. To address this challenge, we propose HBO, a hyperparameter optimization framework that extends Bayesian Optimization with multiple surrogate and acquisition functions, along with two early termination strategies to prune unpromising trials efficiently. Unlike standard Bayesian Optimization, which employs a single surrogate function, HBO leverages a diverse set of functions to enhance search efficiency. Experimental results on the CIFAR-10 dataset demonstrate that HBO achieves competitive accuracy compared to state-of-the-art tuning algorithms such as TPE and BOHB while reducing computational time by more than $50 \%$. Furthermore, the proposed approach is applicable to hardware design optimization, extending its utility beyond AI model tuning.