Method for transformer oil leakage detection text generation using combined large and small models

Transformer oil leakage detection text generation presents challenges such as limited information in single image data and the inadequate capability of existing detection models to integrate local and global information effectively. To address these challenges, this paper introduces a method combining large and small models for transformer oil leakage detection text generation. First, the paper proposed the multilayer iterative depth separable segmentation algorithm (MIDSSeg). This algorithm guided the subsequent large model in generating comprehensive descriptions of transformer images. Second, leveraging MiniGPT-4, a cross-media transformer oil leakage (CMTOL) model was proposed to enhance understanding of transformer oil leakage-related knowledge by learning cross-media information. Finally, a “Small model plus large model” cascaded transformer oil leakage detection text generation (CTOLDTG) method was introduced. By integrating MIDSSeg with CMTOL, this method achieved the fusion of local and global information, thereby generating detailed textual descriptions of images. Results demonstrated that the proposed method not only accurately generated descriptive text for transformer images but also facilitated automatic expansion of the transformer oil leakage image-text pair dataset. Our codes will be available at https://github.com/letter546/CTOLDTG.