Control of Discrete-Time Hybrid Stochastic Systems

A realistic stochastic control problem for hybrid systems with Markovian jump parameters can have the switching parameters in both the state and measurement equations. Furthermore, both the system state and the jump states are, in general, not perfectly observed. Currently there are only two existing controllers for this problem. One is based upon a heuristic multiple model partitioning (MMP) and hypothesis pruning. The other utilizes the entire future tree of models, and is called the Full-Tree (FT) controller. The performance of the latter is significantly superior to the former and their complexities are similar. In this paper we present a new stochastic control algorithm for stochastic systems with Markovian jump parameters. This control algorithm is derived through the use of stochastic dynamic programming and is designed to be used for realistic stochastic control problems, i.e., with noisy state observations. This new scheme, which is based upon the interaction of r (the number of models) model-conditioned Riccati equations, has a natural parallel to implementation. The state estimation and model identification is done via the recently developed Interacting Multiple Model algorithm. Simulation results show that a substantial reduction in cost can be obtained by this new control algorithm over the MMP scheme. Furthermore, the performance of the new algorithm is shown to be practically the same as that of the FT scheme even though the new scheme, which has a fixed amount of computations at each step of the recursion, is much simpler than both the MMP and FT algorithms.