WeldLLM: A Vision-Language Fusion Framework for Weld Defect Detection and Report Generation

Welding defect inspection is crucial for maintaining structural integrity in industries such as aerospace and shipbuilding. Traditional manual methods are labor-intensive and prone to human error, while modern object detectors such as YOLO have achieved high accuracy in automated weld defect detection. However, these models typically output only bounding boxes and class labels, lacking the descriptive context necessary for documentation and decision-making. To address this limitation, we propose a novel feature-level fusion framework that combines YOLOv11 with the Qwen2.5-VL-3B vision-language model. A Feature Adapter is employed to align YOLO’s multi-scale features with the embedding space of the vision-language model, and a Dynamic Attention Network fuses information from both modalities. This enables simultaneous defect localization and the generation of human-readable reports that describe defect types and locations. Evaluated on a weld dataset containing diverse defect categories, our method achieves detection performance comparable to YOLOv11 alone, while additionally providing accurate and detailed textual descriptions. This work demonstrates the potential of integrating vision-language models into industrial inspection systems to improve interpretability and automation.