Diagonal rnns in symbolic music modeling

In this paper, we use Diagonal Recurrent Neural Networks on a sequence prediction task. The modification from standard RNN is simple: Diagonal recurrent matrices are used instead of full. This results in better test likelihood and faster convergence compared to regular full RNNs in most of our experiments. We show the benefits of using diagonal recurrent matrices with popularly used LSTM and GRU architectures as well as with the vanilla RNN architecture, on four standard symbolic music datasets.