Optimizing Machine Translation Algorithms through Empirical Study of Multi-modal Information Fusion

Machine Translation (MT) research has evolved significantly from rule-based methods to neural networks, with Neural Machine Translation (NMT) attracting considerable attention for its improvements in translation accuracy and fluency. This study investigates Multi-Modal Machine Translation (MMT), which integrates textual and visual information to improve translation quality. Despite its potential, MMT faces unique challenges, such as the different semantic spaces between text and visual data and the presence of noise in non-textual information. To address these challenges, innovative methods such as attention mechanisms and reinforcement learning have been proposed. This research study focuses on the fusion of multi-modal information in English machine translation algorithms. It uses a transformer-based model and a multi-modal gating network to dynamically assign weights to different modal information. Experimental results from an English-to-Chinese translation task demonstrate the effectiveness of the proposed approach, showing improvements in BLEU and TER scores as the complexity of the task increases. This research contributes to the advancement of MMT technology, paving way for improved cross-language communication and cultural exchange.