On Predicting Sensor Readings With Sequence Modeling and Reinforcement Learning for Energy-Efficient IoT Applications

Prediction of sensor readings in event-based Internet-of-Things (IoT) applications is considered. A new approach is proposed, which allows turning off sensors in periods when their readings can be predicted, thus preserving energy that would be consumed for sensing and communications. The proposed approach uses a long short-term memory (LSTM) model that learns spatiotemporal patterns in sequences of sensorial data for future predictions. The LSTM model and the sensors collaboratively monitor the environment. They are controlled by a reinforcement learning (RL) agent that dynamically decides about using the LSTM prediction versus physical sensing in a way that maximizes energy saving while maintaining prediction accuracy. Two approaches are used for the RL: 1) the Markov decision process (MDP) model-based for low scale applications and 2) deep ${Q}$ -Network-based for larger scales. Compared to the current literature, the proposed solution is unique in predicting all sensor readings for real-time event detection and providing a model capable of learning long-term spatiotemporal correlations, enabling power conservation and detection accuracy balance. We compare the proposed solutions to the most relevant state-of-the-art approaches using a large real dataset collected in a dynamic space by measuring the accuracy, consumed energy, network lifetime, latency, and missed events’ ratio. To investigate the scalability of the solutions, these parameters are calculated for different network sizes. The results show that the system achieves 50% accuracy with 32% of activation time and 75% accuracy with 60% activation time.