DefectGPT: An Automatic Retrieval-Augmented Framework for Digital Twin-Based Defect Information Management and Analytics

Effective building defect management is vital for the operational safety, longevity, and resilience of modern highrise structures. However, traditional approaches often rely on a patchwork of inspection logs, manual measurements, and inconsistent documentation, which can complicate maintenance efforts and risk assessments. In this paper, we propose DefectGPT, a framework that integrates Digital Twin (DT) modeling, Retrieval-Augmented Generation (RAG), and Large Language Models (LLMs) to streamline the lifecycle defect inspection and management. The system collects multi-modal data from Unmanned Aerial Vehicle (UAV) flights, Building Information Models, and Geographic Information Systems (GIS) to build a comprehensive digital twin of a high-rise building. On top of this unified data model, a fine-tuned LLM performs contextaware retrieval and generation, providing defect-related insights and maintenance recommendations that are grounded in up-todate building data. Periodic field studies in Hong Kong demonstrates that this approach significantly enhances defect data retrieval precision and expedites decision-making workflows by leveraging domain-specific knowledge. These findings open new avenues for AI-driven building management, particularly in large-scale urban environments where efficient, robust, and explainable maintenance systems are increasingly essential.