LoRA and ReFT Optimized Explainable Machine Learning and Deep Learning Framework for SMS Spam Detection

Short Message Service (SMS) spam detection aims to identify and filter unsolicited and potentially harmful messages, a growing problem affecting communication security and user trust. This study proposes a Low-Rank Adaptation (LoRA) and Reinforcement fine-tuning (ReFT) optimized explainable machine learning (ML) and deep learning (DL) framework for SMS spam detection (SPD). We utilize both manual feature extraction methods, including Term Frequency-Inverse Document Frequency (TF-IDF) and Count Vectorization (CV), and DL-based automatic feature extraction using Bidirectional Encoder Representations from Transformers (BERT) and Generative Pre-trained Transformer (GPT) models. LoRA and ReFT are applied for feature optimization, and ML models such as Extreme Gradient Boosting (XGB), Random Forest (RF), Gradient Boosted Decision Trees (GBDT), and Stacking are trained on both optimized and non-optimized features. Further, we fuse the best-performing manual CV and automatic GPT feature extraction outputs, applying the same set of ML models. Notably, the CV and GPT fused features optimized with LoRA and trained using XGB achieve outstanding performance, with an accuracy (ACC) of 99.82%, precision (PRE) of 71.43%, recall (REC) of 100%, F1-score (F1) of 83.33%, and specificity (SPE) of 99.82 %. Finally, explainable artificial intelligence (XAI) techniques, Shapley Additive exPlanations (SHAP) and Local Interpretable Model-agnostic Explanations (LIME), are employed to interpret model predictions and enhance the framework's transparency.