A Fine-Grained Word-level Translation Quality Estimation Method based on Deep Learning

Translation quality estimation (QE) technology aims to evaluate the quality of machine translations without reference translations. Currently, research on word-level QE mainly focuses on discriminating the correctness of word translations in sentences. In order to further identify the types of errors in the translation, this paper proposes a fine-grained word-level QE method. A new translation error taxonomy and a method to automatically generate labelled training corpus are proposed. Two methods based on recurrent neural network BiLSTM and pre-trained model XLM-R are adopted to build a model on the automatically generated corpus, and weighted cross-entropy loss function is used to alleviate the problem of label imbalance. Experimental results show that the proposed methods can effectively identify the fine-grained errors in the translation, which provides more information for post-editors.