Printed Mathematical Expression Recognition from Document Images Based on Augmentation Techniques and Transformer

Mathematical expression recognition with the aim to convert from images to Latex sequence has emerged recently thanks to its potential applications. Transformers have shown its superior performance in different computer vision tasks including mathematical expression recognition. However, one drawback of the transformer is to be lack of coverage information, useful cues for image-text alignment. Inspired from the previous work that integrated coverage information in the decoder of transformer for handwritten mathematical expression recognition, in this paper, we propose a novel framework which adopts expression augmentation step into the above mentioned model for printed mathematical expression recognition. Ablation study shows that the use of data augmentation allows to improve mathematical expression recognition performance. The proposed method obtained the state of the art methods on the subset of Marmot dataset that is prepared for ME recognition evaluation. The values of ExpRate, <=1 error, <=2 error, and WER are 70.72%, 73.21%, 81.31%, and 4.10%.