Comparative study on various pruning algorithms for RNN. I. Complexity analysis

Owing to the computational complexity requirement, pruning a fully connected recurrent neural network (RNN) would be ineffective for large size RNN. In this paper several non-heuristic pruning algorithms for fully connected RNN are investigated, some of them are extended from extended Kalman filter based approaches and some of them are based on weight magnitude, together with some techniques on the pruning procedures. Their effectiveness, such as on their computational complexities, network sizes and generalization abilities, is evaluated and presented. This paper presents the issue on computational complexity.