A Real-Time Air Quality Forecasting System Based on CTLF-Net and Kafka-Hudi Integration

Accurate forecasting of air pollutant concentrations is essential for environmental management and public health protection. This paper presents a fully integrated real-time air quality prediction system centered on CTLF-Net (Convolutional-Transformer-LSTM Fusion Network), a hybrid deep learning model tailored for 48-hour multivariate forecasting of key pollutants. The system architecture is built upon a high-throughput data pipeline where environmental sensors deployed nationwide stream hourly pollutant data to a Kafka-based backbone, ensuring low-latency and fault-tolerant transmission. A real-time prediction engine consumes these messages and invokes CTLF-Net, which combines convolutional layers for local pattern extraction, Transformer encoders for long-range temporal modeling, and LSTM (Long Short-Term Memory) layers for sequential consistency. Historical data and model outputs are persisted in Apache Hudi, enabling scalable analytics and incremental learning. Experiments on real-world data from 2019 to 2024 demonstrate that CTLF-Net consistently outperforms baseline models in RMSE and R², while exhibiting robustness to noise and sensitivity to trend fluctuations. The system and model offer a scalable and practical solution for fine-grained, real-time air quality forecasting.