Transformer based Lightweight Model for Punctuation Restoration and Truecasing

With the evolution of cloud, IoT, Artificial intelligent and Web 3.0 technologies, the development and usage of voice assistants are increasing and employed in aiding human communication. Voice assistants incorporated with many smart devices, part and parcel of everyone’s life, capable of understanding the requirements and generate relevant output. The voice assistants build on using cloud-based services with Automatic Speech Recognition (ASR), speech-to-text conversion and Natural Language processing techniques such as grammatical formatting of recognized text, intent identification, slot tagging, and execution. The use of computationally intensive models and techniques at each stage provides context driven accurate results. Network connectivity, high compute resources and services are necessary to achieve the results and it also raises concern in user data privacy and results delay in providing real time outcome. The proposed approach focuses on building transformer based lightweight model punctuation restoration and truecasing. It also serves as offline mode for voice assistants and ability to use at the device edge to lower the dependence on the intensive computational resource and also avoid transferring the data through network. The experiment results show that a Multi-task model performs well and exceed performance over single task model and Statistical Truecasing.