PEMS: Custom Neural Machine Translation System - Making subtitling of Portuguese TV shows and movies on the African continent work

The boom in audio-visual content such as movies and TV shows has meant that to increase global accessibility, this content needs to be localised into the target audience's language. Neural Machine Translation (NMT) presents a solution to address this at scale. Despite current NMT models having success on general-purpose text, they fail to generalise to the language nuances of media subtitles, primarily because of training data bias. In this work, we propose PEMS - Portuguese Engine for Machine Subtitling, a custom NMT model for English (en) to Portuguese (pt-pt) translation of TV show and movie subtitles. We conducted a thorough evaluation based on automatic metrics and human evaluation. The PEMS model demonstrates superior performance based on automatic evaluation metrics compared to other state of the art (SOTA) models, both open-source and cloud-based. Human evaluation by two professional translators indicated significant performance gains of post-editing machine translation compared to translating from scratch. Overall the productivity increase for PEMS, compared to translation from scratch, was 40% and 33% for each of the two translators, respectively. In addition, PEMS had fewer edits and higher adequacy and fluency scores compared to other models. The ablation study highlights sources of performance that could be applied to improve NMT in general. Specifically, the results demonstrate the importance of adapting NMT systems to the domain of interest (i.e. media). Most importantly, this work highlights the performance benefits of using NMT models such as PEMS to improving subtitling of content and, ultimately, to increase the proliferation and accessibility of media content, both on the African content and for Portuguese speakers.