Research on Real-Time Speech Transcription Optimization Based on Recurrent Neural Network Transducer

Real-time speech transcription has problems such as low recognition accuracy, poor generalization, and poor audio and video fusion in complex environments. We propose a real-time speech transcription optimization method based on the Recurrent neural network Transducer model to solve these problems. First, according to the principle and advantages of the Recurrent neural network Transducer model, the application scenarios and challenges in real-time speech transcription are analyzed. Second, to improve the encoder performance of the model, a model encoder optimization based on the Conformer network is proposed, which uses its convolution and self-attention layers to model local and global speech context information. Then, to improve the model's generalization ability, a model training strategy based on a sparse self-attention mechanism is proposed. By introducing sparse constraints, the self-attention layer's computational complexity and memory consumption are reduced, and the model's adaptability to speeches in various fields is enhanced. Finally, the researchers proposed an improved model predictor based on Transformer network to realize multimodal information fusion of audio and video speech transcription. The video stream is used as an additional input source to learn the interaction between audio and video features through the Transformer network, thereby improving the robustness of speech transcription. In addition, a real-time speech transcription system based on Recurrent neural network Transducer model is built. Simulation and experimental verification prove the effectiveness of the proposed method.