Strategies for reducing the complexity of a RNN based speech recognizer

Recurrent neural networks (RNN) provide a solution for low cost speech recognition systems (SRS) in mass products or in products with energetic constraints if their inherent parallelism could be exploited in a hardware realization. Actually, the computational complexity of SRS based on fully recurrent neural networks (FRNN), e.g. the large number of connections, prevents a hardware realization. We introduce locally recurrent neural networks (LRNN) in order to keep the properties of RNN on the one hand and to reduce the connectivity density of the network on the other hand. By simulation experiments it is shown that the recognition capability of LRNN is equivalent to that of FRNN and superior to other proposed network architectures. Furthermore, it is shown that with an appropriate representation of the network parameters and a retraining of the network 5 Bit quantization of the weights and activities is possible without significant loss in recognition performance.