Speech recognition with Transformers - creating and testing datasets for Bulgarian language

Speech is a natural way of communicating between human beings and as such, it triggers an interest of transforming it to a way of interaction with a computer as well. Once it is converted into a sequence of words, it can easily be applied for executing commands, thus enabling people to perform tasks easily or in some cases - tasks that are even impossible for them to be performed. The automatic speech recognition becomes a task solved with various success based on the accuracy, online/offline availability, speed. However, with the constant improvement of the neural networks, it is getting easier to solve and the most prominent solution seems to be the Transformer architecture of neural networks. In this paper, we describe the design of an encoder-decoder TensorFlow architecture and the process of training the model with a specially created Bulgarian dataset.