Lightweight Knowledge Distillation-Based Surrogate Model for Wheel–Rail Dynamic Contact Force Prediction Using Hybrid CNN–LSTM–Transformer Networks

This research develops a lightweight hybrid convolutional neural network (CNN)–long short-term memory (LSTM)–transformer model for accurately predicting wheel-rail dynamic contact forces, addressing the computational inefficiencies of traditional simulation methods such as the finite element method (FEM). With significant practical value for railway engineering, our approach enables rapid and real-time safety assessments and maintenance planning, making it directly applicable to the design and monitoring of modern rail systems. The proposed surrogate model leverages CNNs for local feature extraction, LSTM networks for capturing temporal dependencies, and Transformer architectures for modeling global relationships, ensuring robust performance in handling noisy and incomplete rail dynamics data. To further boost computational efficiency and support real-time deployment in industrial settings, we incorporate knowledge distillation (KD) to compress the hybrid deep learning model into a lightweight student model. By transferring knowledge from a high-performance teacher model, the distilled model retains high predictive accuracy while substantially reducing computational overhead, making it ideal for embedded system applications and on-site rail dynamics monitoring. Extensive experiments validate the model’s robustness, scalability, and adaptability across dynamic systems, demonstrating superior trade-offs between accuracy and efficiency. The integration of KD offers a paradigm shift in rail system modeling, predictive maintenance, and intelligent transportation systems, enabling highly efficient real-time safety assessments in modern rail transportation.