Long-Term Control for Multi-agent Persistent Monitoring with Limited Detection Qualities and Packet Losses Constraints

This paper investigates a long-term control policy that enables multi-agent to persistently monitor a finite set of targets subjected to limited sensing qualities and packet losses constraints. Agents estimate the state of targets by equipped Kalman filters, and send the monitoring data to a remote control center. The objective is to control the movement of agents to minimize the average estimation errors of targets over an infinite horizon. By modeling the estimation and transmission actions as Markov chains, the persistent monitoring problem is duly formulated as a Markov decision process. On the basis of the monotonicity of value functions, a value iteration based algorithm is proposed to obtain the solutions. As a result, each agent continues to monitor the current target until the data transfer is completed. Theoretical results show that the average estimation error covariance asymptotically converges to a bounded performance. Finally, a numerical example is reported to illustrate the proposed approach.