Region-Based Convolutional Neural Network and Tesseract Optical Character Recognition for Recognising Entity Relationship Diagram

Object detection is a common technique nowadays and is utilised in many fields such as license plate detector, obstacle detector, and many others. However, most of the efforts are found to only focus on detecting objects of a certain class and are not suitable for the detection of diagrammatic contents. Object detection for diagrammatic contents has high demand at workplaces and educational institutions. It is different from other object detection as most diagrammatic contents are in grayscale or binary. Other than that, most object detector also only classifies the object to their class and is not capable of detecting their specific name. Hence, this work proposes a novel object detector based on region-based Convolutional Neural Network (R-CNN) for Entity Relationship Diagram (ERD) object detection which will also give the relationship of the ERD as the result. The framework consists of three main processes starting by the input image being pre-processed and then object detection on the image is carried out. The result of the object detection is then pre-processed again to be passed to the name detection by optical character recognition (OCR) and then the OCR result is post-processed to obtain a more accurate outcome. The final output is expressed in the relationship of the entities and an image with the detection labelled. The evaluation measurement used is stratified cross-validation where the training-testing percentage is 80% training and 20% testing. The proposed object detector managed to obtain satisfactory average accuracy of 81.92% using a low-resolution image dataset to train.