Kalman-like state tracking and control in POMDPS with applications to body sensing networks

In this paper, the problem of state tracking with controlled observations is considered for a system modeled by a discrete-time, finite-state Markov chain. The system state is `hidden' and observed via conditionally Gaussian measurements that are shaped by the underlying state and an exogenous control input. Following an innovations approach, a Kalman-like filter is derived to estimate the Markov chain system state. To optimize the control strategy, the associated mean-squared error is used as an optimization criterion for a partially observable Markov Decision Process (POMDP). The optimal solution is determined via stochastic dynamic programming. Numerical results are presented for the application of physical activity detection in heterogeneous, wireless body area networks.