Deep Frame-Point Sequence Consistent Network for Handwriting Trajectory Recovery

Intelligent Cyber-Physical Systems relies heavily on data for real-time monitoring, analysis, and control of physical systems. By converting offline handwriting into online handwriting, it provides ICPS with more diverse and abundant input data, enriching the variety and quantity of available information. Based on the acquisition approach, there are two kinds of handwriting data: online and offline data. Generally, online data which contains pen trajectory of static character, has more advantages than offline data in terms of character recognition and analysis. Due to limited means of acquiring online data, inferring from offline data is an attractive approach. In this paper we introduce a novel framework to recover handwriting trajectory from single static character image. The trajectory can be represented by two types of sequence: points sequence and frame sequence. Therefore, we design two streams: points sequence prediction stream and frame sequence prediction stream, based on the encoder-decoder structure. We combine the two streams by a novel sequence consistent module to synchronize the training process. With the two streams and their complementary advantages, our methods can predict trajectory with both high spatial and temporal accuracy. Extensive experiments demonstrate the effectiveness of our network through qualitative and quantitative comparison.