Sens-BERT: A BERT-Based Approach for Enabling Transferability and Re-Calibration of Calibration Models for Low-Cost Sensors Under Reference Measurements Scarcity

Low-cost sensors (LCS) are crucial for high-resolution air quality monitoring (AQM), but their inherent noise poses challenges for wide-scale adoption. Typically, LCS are calibrated alongside reference stations to develop learning-based calibration models for accurate air quality estimation. However, existing learning-based calibration works are sensor and location-specific, requiring frequent re-calibration. The re-calibration also needs massive data, as required in the initial calibration, which is an unviable solution in practical scenarios. This work introduces Sens-BERT, a bidirectional encoder representations from transformers (BERT)-inspired deep learning approach for LCS calibration. It comprises two phases: self-supervised pretraining of Sens-BERT on LCS data alone to learn data features and embeddings and supervised fine-tuning of a neural network model for calibration. Sens-BERT offers several advantages: it requires paired data only for fine-tuning and is transferable across sensors of the same type without explicit training on them. We extensively tested our approach on diverse sensors using the community air sensor network (CAIRSENSE) and SensEURcity datasets. We demonstrated superior performance compared to traditional calibration models such as single-variable linear regression (SLR), multiple-variable linear regression (MLR), random forest (RF), long short-term memory (LSTM), and a recently developed calibration model, estimated error augmented two-phase calibration (EEATC). Further, we empirically validate Sens-BERT ’s transferability and re-calibration capabilities under reference measurement scarcity, a required scenario to establish a sensor network in remote locations.