An Energy-Efficient Strategy based on Q-Learning for Energy Harvesting-based Wireless Sensor Network

In this paper, we consider a scenario in which the energy consumption transitions from sleep mode to the active mode for energy harvesting wireless sensor network (EHWSN). The sensor’s energy consumption is primarily caused by the communication module. Due to the fact that the energy consumed during sensor state transitions is not the most essential component of the energy consumed, many experimental investigations exclude the energy used during state transitions when calculating the overall energy used by the sensors. However, when the frequency of sensor state transitions rises, the energy consumption associated with wireless sensor state transitions becomes unavoidable. As a result, we present an energy-efficient algorithm based on Q-learning that train the agent to minimize sensor state transitions and prevent further energy consumption in order to accomplish the goal of energy conservation. The experiment demonstrates that our method can save 18.07% of energy when compared to the Q-learning approach in ACES (ACES-QL), while also improving the residual energy of the energy storage device by 3.40%.