A Pruning Method for Echo State Network Based on Neuron Importance and Iterative Fine-Tuning

Edge intelligence provides intelligent services at the network edge and improves the performance of the applications. Deploying neural networks on edge devices can bring intelligence closer to users. As a new type of recurrent neural network, echo state network (ESN) is widely used in many fields. However, edge devices usually have very limited resources. In order to reduce the network size and the performance loss, so as to deploy ESN on resource-constrained edge devices, this paper proposes a pruning method for ESN based on neuron importance and iterative fine-tuning. First, a neuron pruning algorithm based on neuron importance is proposed. The importance of each neuron is evaluated by integrating the mutual information and the weight, and then the unimportant neurons are pruned. Second, an information compensation algorithm based on iterative fine-tuning is proposed. The reservoir weight of the high pruning rate network is fine-tuned with the reservoir weight of the low pruning rate network to compensate part of the performance loss. Finally, simulation results on theoretical and real datasets show that the proposed method has better performance compared with other pruning methods.