Firefly Algorithm-based Hyperparameters Setting of DRNN for Weather Prediction

A Deep Recurrent Neural Networks (DRNN) is powerful to be used in sequential datasets. Quite hard tasks in DRNN is setting the optimum hyperparameters. There are known to be three types of general methods for searching the optimum DRNN hyperparameters: manual, grid, and random searches. However, these types of methods are not the right choice when a prior experience is insufficient. This paper addresses both the optimization and automation of hyperparameters to build its structure. They are carried out using a Firefly Algorithm (FA), one of the metaheuristic methods. The hyperparameters to be optimized and automated are batch size, dense, and total units in each layer. There are three things to consider in doing FA-based optimization in this test, such as designing FA, determining the initialization of fixed hyperparameters from the DRNN, and determining the range of DRNN hyperparameter values. Evaluation using the dataset of weather history recorded by the Max Planck Biogeochemical Institute, which contains 15 attributes, shows that the FA-based hyperparameters setting of DRNN gives a much lower prediction error of 0.111 than the manual tuning (0.475). Based on that result, when using FA for the optimization of DRNN hyperparameters in weather prediction, it reduces the error value, so the prediction results using DRNN are more accurate.