Comparison of Recurrent Neural Networks for Slovak Punctuation Restoration

The paper proposes a punctuation restoration system based on a recurrent neural network that supplements a Slovak automatic speech recognition system. It compares methods based on long short-term memory, gated recurrent units, and bidirectional networks on the same training and evaluation set to discover which method is the best for this task. Experiments show that there are significant differences among recurrent neural network methods. Performance of the classification strongly depends on parameters of learning and size of training data. Each neural network inclines to overfitting. The best setup was found with bi-directional networks with gated recurrent units.