An experiment on subspace learning for Echo State Network

Echo State Network (ESN) is a type of neural network with convex training used as nonlinear adaptive filters. It requires relative large network size and O(N2) training algorithm in order to realize its full potential. We present a subspace training scheme for ESN that reduces the training cost from O(N2) to O(NL) where L <; N is the dimension of the subspace. Experiments show that L = 20 is sufficient for networks of size N = 100 and that the performance of subspace training algorithm is actually better than current standard training algorithm while having only 20% computational cost and preserving convex training.