A Study of Automatic Solver for Math Word Problems based on RoBERTa and SW-MSA

In this paper, we encountered the major challenge of effectively solving Mathematical Word Problems (MWP) that involve complex logic and long text descriptions. To address these issues, we proposed the following methods: 1.Introduced a RoBERTa encoder optimized with a dynamic masking technique to enhance the understanding of MWP text. 2.Developed a decoder based on Shifted Window Multi-head Self-Attention (SW-MSA), which limits the self-attention computation range while effectively capturing long-range dependencies, thereby improving decoding efficiency and accuracy. By combining these methods, we proposed a new RoBERTa-SWMSA model. Experimental results show that the RoBERTa-SWMSA model performs excellently on standard datasets Math23k and Ape210K, with notable advantages in solving problems with complex logic and long text descriptions. Ablation experiments further highlight the importance of the pre-training strategy for model performance. The RoBERTa-SWMSA model provides an efficient, accurate, and generalizable solution for the automatic solving of MWP.