DiagGCN: An End-to-end Framework for Online Handwritten Diagram Recognition

Diagrams are among the most common elements in documents, widely used in human-machine interfaces, education, etc. Online handwritten diagram recognition has attracted con-siderable attention due to its potential applications. However, it remains challenging because of complex 2D structures and diverse writing styles. To handle these difficulties, we formulate symbol segmentation and recognition as node clustering and node classification on graphs, solving them using a novel graph neural network (GNN) model with enhanced structure modeling ability. In detail, we introduce a stroke graph construction method to extract the local structure of diagrams and propose DiagGCN to capture their high-level features. Additionally, we design a mean-shift clustering layer to improve node clustering performance. We introduce a unified learning framework to jointly train the model and perform the diagram recognition in an end-to-end manner. Extensive experiments on three popular datasets demonstrate that our method consistently outperforms previous methods, achieving state-of-the-art performance.