Speeding up Common Hyperparameter Optimization Methods by a Two-Phase-Search

Hyperparameter search concerns everybody who works with machine learning. We compare publicly available hyperparameter searches on four datasets. We develop metrics to measure the performance of hyperparameter searches across datasets of different sizes as well as machine learning algorithms. Further, we propose a method of speeding up the search by using subsets of data. Results show that random search performs well compared to Bayesian methods and that a combined search can speed up the search by a factor of 5.