Co-InsCap: Collaborative Detection and Image Caption for Defective Insulator of High-Speed Railway Catenary

Insulator defects in high-speed railway catenary systems can impair the electrical insulation and structural integrity of the catenary. Insulator defects pose a significant threat to train safety. Detecting insulators, however, involves challenges such as complex railway backgrounds, inaccurate defect detection, and the inability to provide semantic descriptions of insulator defects. In order to address the problem, we propose a collaborative insulator caption (Co-InsCap) model to detect and describe insulator defects. First, a multiscale mask backbone (MMB) is employed to improve the backbone, enhancing the ability to extract insulators in complex backgrounds. Second, a collaborative auxiliary head (CAH) is designed to improve the Co-InsCap accuracy in the bounding boxes of insulators. Third, the BLIP model with a text filtering mechanism is introduced to generate natural language captions containing semantic information about the insulator. Additionally, an image caption dataset centered on insulators is established, which includes various images describing insulator defects to enhance the understanding and proficiency in describing insulator defects. Experimental results show that compared with the DINO, Co-InsCap improves average recall (AR) and $F1$ score by 4.81% and 16.49%, respectively. BLEU score is improved by 20.96% and 1.23% compared to LSTM and LLaVA models, validating the accuracy and applicability of the Co-InsCap.