POMDP sensor scheduling with adaptive sampling

We consider the sensor scheduling problem where a decision maker can choose from a finite set of sampling intervals to pick the next time to observe the noisy state of Markovian target. The aim is to optimize an objective comprising of false alarm, delay cost and cumulative measurement sampling cost. Taking more frequent measurements yields accurate estimates but incurs a higher measurement cost. Making an erroneous decision too soon incurs a false alarm penalty. Waiting too long to declare the target state incurs a delay penalty. The problem is an instance of a partially observed Markov decision process (POMDP). This paper reviews structural results in POMDPs. The paper then shows that under reasonable conditions, the optimal strategy has the following intuitive structure: when the posterior distribution of the Markov chain is away from the target state, look less frequently; while if the posterior is close to the target state, look more frequently.