Evaluation of Model Degradation in PaddleOCR, UltOCR, and TrOCR Across Baseline and TensorFlow Lite Environments

Optical Character Recognition (OCR) models are deployed on edge devices for many applications ranging from document scanning to real-time text recognition. For edge deployment, it is suitable to convert OCR models to TensorFlow Lite (TFLite) format. However, there is uncertainty of model's performance degradation after conversion. This study evaluates three prominent OCR models-PaddleOCR (CNN-RNN architecture), UltOCR (MASTER architecture), and TrOCR (Transformer-based architecture)-selected for their distinct architectures. Baseline, TFLite 32-bit, and quantized 16-bit versions of these models were tested on datasets containing words, numbers, and sentences to identify which architecture maintains the performance after conversion. The findings indicate that PaddleOCR and UltOCR achieved minimal degradation. However, TrOCR experienced a significant accuracy loss of 6.81% post-conversion in the sentences category. The study provides valuable insights in selecting which OCR architecture should be used for edge deployment.