Reinforcement Learning-Based Sensing Decision for Data Freshness in Blockchain-Empowered Wireless Networks

Recently, blockchain (BC)-empowered wireless sensor networks (WSN) emerged as a promising solution for secure and reliable data management. However, the integration of BC and WSN brings several challenges including long processing delay at BC, which reduces freshness of sensed data. Motivated by this, we first model the BC-empowered WSN and define the age of information (AoI), the elapsed time from the sensor’s data collection until its commitment to the BC. We then formulate the AoI violation probability minimization problem and propose the reinforcement learning-based sensing decision (RL-SD) algorithm. Using the RL-SD, the sensor intelligently makes sensing decisions, considering wireless channel conditions, BC process latency, and energy status. We further introduce the pause mechanism to save energy, where the sensor pauses sensing and transmission for a while after the successful transmission. Our experiments demonstrate that the proposed algorithm outperforms the probabilistic sensing decision algorithm that senses randomly with the optimal probability. We also verify the performance of the RL-SD for various environments with different block sizes, pause times, and AoI thresholds.