Reinforcement Learning based Battery Energy Neutral Operation for EHWS

The introduction of energy harvesting technology into wireless sensor nodes has advanced the realization of autonomous wireless sensor networks. Enabling Energy harvesting wireless sensor(EHWS) nodes to maintain an Energy Neutral Operation(ENO) state by dynamically adjusting energy consumption to match the varying collected energy effectively enhances the node’s continuous survival time and energy efficiency. This paper introduces for the first time the concept of Battery Energy Neutral Operation (BENO) and proposes an Adaptive Energy Management method based on Actor-Critic for energy harvesting wireless sensor nodes (ACEM) based on this concept. BENO thoroughly considers the role of rechargeable batteries as energy buffers in EHWS energy systems in maintaining the ENO state of the node. Building upon BENO, we designs a reward function to develop the ACEM algorithm, dynamically adjusting the duty cycle to match energy consumption with collected energy, thereby sustaining the ENO state of the node. Experimental results show that under different initial energy settings, the ACEM method based on BENO achieves an average duty cycle increase of 61.11% and 43.97%, respectively compared to the AQL and FQL benchmark methods. The waste energy rates have been reduced by 0.64% and 5.53%, respectively. Achieving node BENO significantly affects maintaining the ENO state of the node.