MEMix: Improving HMER with Diverse Formula Structure Augmentation

Handwritten Mathematical Expression Recognition (HMER) aims to transform images of mathematical expressions (MEs) into corresponding LaTeX sequences. However, the inherent complexity of 2D formula structures often misaligns with the 1D LaTeX sequences, resulting in decreased robustness in recognition models. A primary factor exacerbating this issue is the scarcity of annotated ME images with complex structures, which hinders the models to learning to good representation and adaptability for MEs. In this paper, drawing inspiration from Mixup, we introduce a data augmentation method called Mathematical Expression Mix (MEMix). This method is capable of generating typical structures in formulas, including radicals, fractions, and annotations, by employing straightforward matrix operations. Compared to alternative data augmentation methods, MEMix provides faster and more cost-effective computation, enabling online augmentation that improves training efficiency. Experiments demonstrate that MEMix significantly enhances the performance of the baseline model on the benchmark datasets.