Hybrid Framework Combining Diffusion-Based Image Augmentation and Feature Level SMOTE for Addressing Extreme Class Imbalance

Imbalanced datasets pose significant challenges to the reliability and robustness of visual classification systems, particularly in critical applications like solar panel dust detection. Conventional data augmentation techniques, such as Synthetic Minority Over-Sampling Technique (SMOTE) or generative models like Stable Diffusion, independently demonstrate limitations, either by lacking structural diversity or failing to sufficiently address feature-space imbalance. To overcome these constraints, this paper proposes DiSMHA (Diffusion-SMOTE Hybrid Augmentation), an innovative framework integrating pixel-level image synthesis using Stable Diffusion and feature-level oversampling via SMOTE. DiSMHA systematically balances and enriches minority class representation by producing high-quality synthetic images coupled with robust feature interpolation in the latent space. Experiments conducted on four imbalanced solar panel image dataset cases demonstrate substantial improvements across multiple performance metrics, particularly accuracy, recall and F1-score, emphasizing effective minority class detection. The proposed DiSMHA approach was evaluated using several classification models, including Support Vector Machine (SVM), Multilayer Perceptron (MLP), XGBoost, Vision Transformer (ViT-Head), and TabNet. These models achieved notable accuracies ranging from 94% to 98%, significantly outperforming baseline methods. Particularly, the MLP (with accuracy 96.67%) and SVM (with accuracy 94.67%) models exhibited substantial performance gains, confirming DiSMHA’s capacity to enhance traditionally imbalance-sensitive classifiers. Comparative analyses further underscored the superiority of DiSMHA, highlighting its consistent accuracy, enhanced recall, and improved F1-scores over individual augmentation approaches. Moreover, statistical validation using the Wilcoxon signed rank test confirmed the significant superiority of our approach for minority class detection in highly imbalanced scenarios. Thus, DiSMHA represents a scalable, robust solution applicable across diverse domains facing imbalanced classification challenges, including industrial inspection, medical imaging, and rare event detection.