An Improved Hierarchical Markovian Target Tracking (I-HMTT) Algorithm for Energy Efficient Wireless Sensor Networks

In this paper, we consider a target-tracking sensor network and improve its energy awareness through predicting a target trajectory and decreasing sampling rate of sensors while maintaining an acceptable tracking accuracy. The tracking problem is formulated as a hierarchical Markov decision process (MDP) and is solved through neurodynamic programming. Though this is not new, improvements in performance of the network are achieved by use of a reinforcement learning algorithm to solve the MDP that converges faster than the preceding used methods, since the energy efficiency and speed of convergence of the solution are tightly coupled. Simulation results show the effectiveness of our algorithm against other known target tracking algorithms.