Data Pipeline Leveraging Computer Vision and Transfer Learning for the Ingestion of Commercial Photovoltaic Module Datasheets

In this paper, we use deep learning models to detect and classify various data objects within photovoltaics (PV) module datasheets stored in pdf or other image file formats. We seek to differentiate between regular text, table, header, footer, module drawing, document header, IV curve, PV curve, warranty graph, module image, and icon in order to automate the extraction of useful metadata from datasheets. Our work investigates whether existing computer vision models such as Detectron2 and YOLOv8x can be leveraged to accurately localize and classify the various data types with and without transfer learning. The models were trained on 57 datasheets with multiple pages or 169 pages in total. Detectron2 achieved an overall mAP50-95 of 0.78 without transfer learning (600 epochs and 16 hours) and an mAP50-95 of 0.54 (60 epochs and 2 hours) with transfer learning. YOLOv8x achieved an mAP50-95 of 0.89 without transfer learning (1000 epochs and 2 hours) and an mAP50-95 of 0.9 (20 epochs and several minutes).