Deep Learning versus High-order Recurrent Neural Network based Decoding for Convolutional Codes

In the last decade, deep neural networks (DNNs) have shown impressive results in various fields such as image classification, speech recognition, or playing the abstract strategy board game Go. Recently, also an increased interest in the application of DNNs to physical layer problems in digital communications can be observed. We use a DNN for one-shot decoding of convolutional (self-orthogonal) codes. An advantage of this use case is the unlimited amount of labeled data for training. A disadvantage is, that the number of code words to be learned increases exponentially with the dimension of the code. We compare the performance of the DNN-based decoding with iterative threshold decoding (ITD). Here, a discrete-time high-order recurrent neural network (HORNN) is used as a computational model for ITD. Unfolding the HORNN in time, we arrive at a DNN with a special structure as defined by the HORNN. With a training procedure we can optimize the performance of this unfolded HORNN (uHORNN). The advantage of this approach is that the structure of the uHORNN is determined by the structure of iterative threshold decoding. Only a few weights, which are shared within and between the layers, must be adapted to optimize the network. In this way we combine the advantages of both approaches, the structured approach of the HORNN as a computational model, on the one hand, and the training based optimization as given by a DNN on the other hand.