Online Handwriting Recognition Based on Microphone and IMU

This paper presents a method for handwritten character recognition based on the Inertial Measurement Unit (IMU) and the microphone set on the pen. It realized online handwriting recognition by integrating acceleration data and audio data of handwritten action. In this method, the audio during writing is firstly recognized and located, and the effective acceleration sensing data during writing are filtered by the audio location results. The filtered data sets of strokes and characters are used to train the model for handwriting recognition with neural network learning method. Through the training and testing of 1500 groups (2 * 15 * 50 = 1500) of 15 types of characters, this work realizes the results that the overall recognition accuracy is improved by 8% and the recognition accuracy of several typical characters is improved by 18% compared with the original model, which shows that the algorithm effectively improves the recognition accuracy of some typical characters and also enhances the overall accuracy. The method not only improves the accuracy of handwriting recognition data, but also reduces the amount of data transmitted and processed. It is suitable for the implementation of efficient handwriting recognition in embedded system.