Browser Extension for Instant Language Conversion for Video Audio

The world of multimedia content in various languages has been consistently growing, and real-time audio translation has emerged as an essential solution to overcome language barriers. We present a browser extension that can translate live audio from any video you are currently playing using machine learning. Integrating ASR, MT and TTS technologies. The extension daisy-chains audio directly from video content via either the Web Audio API or Chrome’s tab capture API to process and convert it into accurate speech-to-text. Next, the text that has been converted to speech is translated to the target language using a neural machine translation model (in our case, Marian MT). The text is eventually fed into Tacotron 2, which guarantees a high-definition reproduction of its meaning extracted from the multiple contexts it has learned to understand. The system uses real-time data over Web Sockets to reduce latency, thereby maintaining synchrony between the original video and its translated speech while maintaining buffering methods for smooth streaming. The proposed extension provides efficient and scalable real-time accessibility, thereby promoting translation to audiences from various languages. Experimental evaluations validated the strength of the given system in meeting its objective that the translations returned are accurate to content and nearly delay-free for end users.