Simultaneous Interpretation Based on Deep Learning and Speech Synthesis

Simultaneous interpretation is a scarce skill which has only been performed by well-trained interpreters so far. The fast development and wide application of artificial intelligence stimulate us to think about its computer implementation. In this paper, we design and implement a system, in which original speech can be converted into translated speech through three steps: speech recognition, text translation and speech synthesis. We apply the newly-developed HiFi-GAN vocoder for speech synthesis, which provides a faster speed and better quality than previous vocoders. And we add the speaker's voice elements as a factor in the synthesis to make the synthesized speech much similar to the original speaker. As is demonstrated by an English to Chinese speech interpretation experiment, our purposed system outperforms a currently widely-used translation system, showing an exciting prospect for computer implementation of simultaneous interpretation.