Evolutionary Design of Long Short Term Memory Networks and Ensembles through Genetic Algorithms

Long Short Term Memory (LSTM) network is an effective and popular deep learning model for processing complex sequential data. However, using a single LSTM network does not always achieve reliable performance with the increasing complexity of machine learning tasks. An important approach to improve the performance of LSTM networks is to learn an ensemble of these networks. This paper proposes a new multi-objective evolutionary algorithm to design ensembles of LSTMs. We use NSGA-II to evolve accurate and diverse LSTMs as the building blocks for creating effective ensembles. An integrated ensemble genetic algorithm further evolves the best-performing ensembles from the LSTMs created in each generation of NSGA-II in an efficient manner, allowing all LSTMs evolved by NSGA-II to be explicitly considered for building ensembles. The results show that the proposed approach significantly improves the efficiency and effectiveness over many state-of-the-art machine learning models on various classification and regression tasks.