Structural String Decoder for Handwritten Mathematical Expression Recognition

Recently, recognition of handwritten mathematical expression has been greatly improved by employing sequence modeling methods such as encoder-decoder based methods. Existing encoder-decoder models use string decoders or tree decoders to generate markup of mathematical expression recognition. String decoders directly generate LaTeX strings and tree decoders decode expressions into tree structures. The generalization of string decoders is poor on mathematical expressions with complex hierarchical structures, but its language model is better. Tree decoders can deal with the complex hierarchical structures, but its language model is weakened. In order to take advantage of the above two decoders, we propose a novel structural string decoder (SSD) which not only has good generalization but also can make good use of language model. We demonstrate how the proposed SSD outperforms state-of-the-art string decoders and tree decoders through a set of experiments on CROHME database, which is currently the largest benchmark for online handwritten mathematical expression recognition.