Protected Health Information Recognition in Medical Images Using YOLO, OCR, and LLM

The aim of this study was to develop a system capable of recognizing textual data in medical images and identifying which labels contain private information. We designed an intuitive, modular pipeline with extensibility in mind, allowing for the seamless integration or replacement of individual components as technologies evolve. This architecture ensures long-term adaptability to future advancements in OCR and machine learning. For text detection on images, we used a YOLO model fine-tuned on a specialized dataset of lung CT images. To improve text analysis, we integrated a large language model (LLM) into the workflow. Although effective, LLM introduces noticeable latency when processing numerous detected text regions, which may impact the overall image processing time. Nevertheless, with an accuracy of approximately 80% achieved on the test set, the system demonstrates robustness and scalability for the detection and interpretation of textual data in medical imaging contexts.