Evaluation Metrics of Water Meter Reading with Optical Character Recognition

Water plays big role in our daily life. In Indonesia, to supply clean water to people, there are water providers which may owned by state through local government, private company or community. This study put attention in the water meter recording process that is done manually and prone to error due to double bookkeeping. One of the solutions is by implementing Optical Character Recognition (OCR) to read water meter. This study considers this approach offer certain flexibility since the change and cost is limited. However, most research in water meter reader evaluate their performance only with accuracy. This study proposes other evaluation metrics such as Character Error Rate (CER), Mean Absolute Percentage Error (MAPE) and Root Mean Squared Error (RMSE) to complement accuracy. By using those metrics to evaluate the prototype in this study produces these results accuracy of 84.8%, CER 15.6%, MAPE 0.2% and RMSE 122. Using the metrics, the study concludes that the prototype has good pre-processing and post-processing but not good enough recognition. The high number of RMSE also suggest that the prototype has potential to give disadvantage to both customer and provider. This study concludes that in water meter reader with OCR, accuracy metric alone is not enough to measure the performance of the system. CER and RMSE may help to give better insight to the performance of the water meter reader. The study also shows that RMSE has high sensitivity to outlier. In this study, removal of one outlier reduces the RMSE by 87%.