VLF-DETR: Integrating Vision-Language and High-Frequency Features for Transmission Line Defect Detection

Drone-based image capture with deep learning techniques has become a prevalent approach for inspecting transmission lines, allowing for efficient detection of defects and anomalies. However, most existing algorithms rely on the single-modality information, failing to fully exploit the textual modality inherent in labels or the unique characteristics of inspection images, such as sharply focused foregrounds and defocused backgrounds. To overcome these limitations, this article proposes vision-language and high-frequency features DEtection TRansformer (VLF-DETR), a detection model that leverages a multistage training strategy within the Deformable DETR framework. In the first stage, to address the absence of domain-specific knowledge in the general-purpose vision-language model foundational language and vision alignment (FLAVA), we fine-tune FLAVA to learn both textual and visual features relevant to the power industry. In the second stage, to better incorporate textual modality, we introduce conditional queries into Deformable DETR, effectively transferring knowledge from FLAVA into the defect detection model. In the third stage, leveraging the structural characteristics of inspection images, we apply a fast Fourier transform (FFT) to extract high-frequency edge features, suppress background noise, and provide spatial priors. Furthermore, an FFT-based loss function (FFT Loss) is introduced to further ensure the model converges on target regions. Experimental results demonstrate that VLF-DETR significantly outperforms baseline methods, offering a novel and effective solution for transmission line defect detection.