LLM Powered HTR: Integrating Handwritten Text Recognition System with Large Language Model

At the present time, handwritten scripts are the most common form of text available. However with the gradual shift across the world towards the digital domain, it is now becoming necessary to convert handwritten texts to machine-readable format. The handwritten text recognition (HTR) system is an essential tool to convert handwritten text images into machine-readable text. This tool finds application in a wide variety of tasks ranging from interpreting handwriting and signature recognition to analysis of ancient scripts. Due to this, the accuracy of the handwritten text recognition system is of great concern. Unfortunately, because of the complexity and variability of handwritten text, the performance of HTR systems is prone to errors. In this paper, we propose to reduce the errors in the output of the HTR system by leveraging the immense power of Large Language models (LLMs). LLMs are trained on enormous amounts of language data and are proven to be very efficient in tasks related to processing of natural languages. We propose to integrate LLM with HTR as the post-processing correction unit to improve the accuracy of the text generated. By using our proposed methods, we show an 10% improvement in recognition of handwritten text.