Evaluation of the Validity of Back-Translation as a Method of Assessing the Accuracy of Machine Translation

Inaccurate translation prevents effective communication between individuals and leads to misunderstandings. Back-translation is a process used to check the accuracy of a sentence by translating it to the checker's native language. We believe that there is a positive correlation between the accuracy of sentences translated to a target language and that of back-translated sentences. Some studies have discussed the validity of back-translation from the standpoint of the performance evaluation of machine translation systems and concluded that back-translation was unsuitable for checking translation accuracy. However, this result has been derived from the process of verification of whether there was a positive correlation between the accuracy of target-translated sentences and that of back-translated sentences. This mode of verification is not suitable to human verification of back-translation. In this paper, we verified the validity of back-translation from the standpoint of a human checker. We focused on differences that tend to occur between checkers, and determined whether there were differences between target- and back-translated sentences. Our study's contribution can be summarized as follows: (1) we examined the co-occurrence rate of each evaluated value in the same sentence and revealed possible differences in evaluated values between checkers, and (2) we defined an acceptable range of accuracy mismatch based on these differences. Moreover, we revealed how the rate of accuracy mismatch changes with the extension of the acceptable range, which can be different for different purposes. Our results enable people to judge whether they can use back-translation for their purposes.