Regularized Echo State Network with an Improved Log Penalty

Echo state networks (ESNs) are a kind of special recurrent neural networks, which have super performance on time series predictions. However, ESNs with over large reservoirs may lead to the output collinearity problem. In this paper, a regularized ESN with an improved log penalty is proposed to solve this problem. First, an improved log penalty is introduced to the quadratic loss function. Second, the coordinate descent algorithm is employed to optimize the loss function for a sparse solution. Finally, the proposed model is tested using two benchmark data sets. The experimental results show that our method improves the ability of ESNs to deal with the output collinearity problems.