Adaptive Sampling and Actuation for POMDPs: Application to Precision Agriculture

Given a partially observable Markov decision process (POMDP) with finite state, input and measurement spaces, and costly measurements and control, we consider the problem of when to sample and actuate. Both sampling and actuation are modeled as control actions in a framework encompassing estimation and intervention problems. The process evolves freely between two consecutive control action times. Control actions are assumed to reset the conditional distribution of the state given the measurements to one of a finite number of distributions. We tackle the problem of deciding when control actions should occur in order to minimize an average cost that penalizes states and the rate of control actions. The problem is first shown to boil down to a stopping time problem. While the latter can be solved optimally, the complexity of the optimal policy is intractable. Thus, we propose two approximate methods. The first is inspired by relaxed dynamic programming, and it is within an additive cost factor of the optimal policy. The second is inspired by consistent event-triggered control and ensures that the cost is smaller than that of periodic control for the same control rate. We conclude that the latter policy can deal with large dimensional problems, as demonstrated in the context of precision agriculture.