Transmission Line Detection Through Auxiliary Feature Registration With Knowledge Distillation

The periodic inspection of transmission lines ensures a stable power supply across various regions. Deep learning methods, particularly those based on multimodal approaches, have made significant advancements in this field. Current multimodal fusion algorithms do not prioritize the integration of supplementary information from other modal sources. Moreover, these methods frequently depend on substantial parameterization to achieve optimal performance, severely hampering deployment on mobile devices. To enhance the supportive role of the auxiliary modality and minimize model parameters, we proposed the knowledge distillation-based auxiliary feature registration network (AFRNet-S*) for RGB-T transmission line detection (TLD). The method incorporates an auxiliary feature registration module during fusion, utilizing unique complementarity between primary and auxiliary feature modalities for precise feature registration. For knowledge distillation, we devised response-mimicking distillation, semantic supplementary distillation, and cross-view feature polymerization distillation tailored for the student network (AFRNet-S, without knowledge distillation) to achieve model compression. AFRNet-S can learn comprehensive final spatial, precise semantic, and complementary feature integration information from AFRNet-T through these distillation methods. Extensive experiments conducted on an industrial TLD dataset demonstrate that the proposed AFRNet-S* (AFRNet-S with knowledge distillation) outperforms the existing state-of-the-art methods, thereby demonstrating its generality and effectiveness. Our AFRNet-S*, compared to AFRNet-T, has a reduced parameter count from 44.92M to 16.86M and a decrease in FLOPs from 21.13G to 6.41G. This further optimizes the deployment of our network on edge devices, such as unmanned aerial vehicles (drones), in the future. The code and results of our approach are available at https://github.com/WangYuSenn/AFRNet. Note to Practitioners—This study introduces the AFRNet for TLD, utilizing knowledge distillation (KD) techniques. We achieved a better balance between model performance and parameter count by integrating the KD task into the TLD task, facilitating easier deployment on mobile devices. The proposed AFRNet-S* effectively balances model performance and parameter count, ensuring efficient performance of the TLD task. Addressing RGB-T TLD-specific challenges remains a crucial area for future research. For instance, in scenarios where a target occupies a small area within an image, removing excessive background noise is essential to enhance the efficiency of target localization. Furthermore, we aim to utilize AFRNet-S* on drones for transmission line inspection in the future to address practical application challenges.