MSPL: Multimodal Statistical Prompt Learning for New Energy Equipment Defect Recognition

Monitoring renewable energy devices is crucial for the timely detection of faults and the improvement of system stability, making it a key component of the Internet of Things (IoT) ecosystem. However, existing intelligent algorithms and IoT methods rely on edge devices with limited computational capacity, which requires balancing real-time performance and accuracy. Additionally, most methods require initial training on large-scale data using cloud platforms or high-performance servers, leading to high initial costs. To address these challenges, we propose multimodal statistical prompt learning (MSPL) for new energy equipment defect recognition on IoT edge devices. This method enables rapid learning with only a few samples, avoiding the need for large-scale centralized training. Specifically, MSPL uses the pretrained contrastive language-image pretraining model as its backbone, leveraging text-based conceptual information to enhance the understanding of visual inputs. A statistical query module is implemented at the end of the backbone to extract distinctive features from the outputs, integrating these features with soft prompts to customize them for the defect recognition task. Since the learnable parameters are limited to soft prompts added at the end of the backbone, MSPL restricts gradient backpropagation to this point. This improves parameter and memory efficiency, making it more suitable for scenarios with limited computational capacity on IoT edge devices. Experimental results of MSPL on two renewable energy equipment defect datasets and edge devices indicate that it meets real-time processing requirements while maintaining high accuracy, outperforming other methods.