Combined Segmentation and Recognition of Online Handwritten Diagrams with High Order Markov Random Field

The recognition of online handwritten flowcharts is studied in this work. They are considered as a 2D language with two kinds of primitives: symbols and structural relationships. Three main steps are involved to process this data. They concern i) symbol hypothesis generation, ii) symbol recognition and iii) layout interpretation using grammar description to structure information at the document level. We propose to build a high order Markov random field on stroke level to cope with segmentation and recognition simultaneously. The potential function in our Markov random field is log-linear, and we trained it using max-margin method. We tested our method on two public handwritten diagram datasets and experiments show that our symbol recognition method's performance has reached state-of-the-art.