Evaluating Translation Quality and Usability of Maana: A Meranaw-English Bidirectional Speech Translation App

This study presents a comprehensive evaluation of Maana, a novel Meranaw-English bidirectional speech translation application designed to bridge linguistic gaps and preserve the Meranaw language in the Philippines. Given that Meranaw is a low-resource language, we utilize a modest training dataset to fine-tune a pretrained transformer model. Maana demonstrates commendable translation accuracy, validated through both automated metrics like BLEU and METEOR and comprehensive human evaluations. Maana integrates Automatic Speech Recognition (ASR), Machine Translation (MT), and Text-to-Speech (TTS) functionalities within a user-friendly web interface. The app's usability evaluated through System Usability Scale (SUS) indicates high user satisfaction, reflecting the app's effectiveness and ease of use in facilitating cross-cultural communication. This research not only emphasizes Maana's role in language preservation but also showcases its innovative approach in the field of language translation technology.