Handwriting Enhancement: Recognition-Based and Recognition-Independent Approaches for On-Device Online Handwritten Text Alignment

Legible handwriting refers to the written work that can be easily read and comprehended by both the writer and other people. Despite significant progress in the field of digital handwriting processing, enhancing the visual quality of handwritten content remains a relatively unexplored and challenging task. In this paper, we propose and analyze two approaches aimed at enhancing the legibility of handwriting by straightening the written content while maintaining the user’s original writing style. The first method is recognition-based and relies on the results of our online handwriting recognizer and heuristic rules of alignment designed for each character. The second method is recognition-independent and utilizes a Hierarchical Recurrent Neural Network (HRNN) for handwriting alignment, applying it directly to the input strokes without the need for character recognition. This neural network is trained to predict the aligned position of each handwritten stroke and uses the same training samples that we employed for building our online handwriting recognition system. We performed an evaluation and comparison of the proposed techniques, considering both their quality and speed. The results demonstrate that the suggested methods enhance the visual quality of handwritten text in over 74% of cases for both techniques. Alignment inaccuracies were observed in fewer than 4.3% of text lines in the first method and less than 1% in the second method. Our findings demonstrate that hierarchical neural networks can achieve better straightening results even without knowing the context of written letters, although it requires a diverse range of writing styles in the training data.