Cart (Loading....) | Create Account
Close category search window

Optimal Control of Hidden Markov Models With Binary Observations

Sign In

Cookies must be enabled to login.After enabling cookies , please use refresh or reload or ctrl+f5 on the browser for the login options.

Formats Non-Member Member
$31 $13
Learn how you can qualify for the best price for this item!
Become an IEEE Member or Subscribe to
IEEE Xplore for exclusive pricing!
close button

puzzle piece

IEEE membership options for an individual and IEEE Xplore subscriptions for an organization offer the most affordable access to essential journal articles, conference papers, standards, eBooks, and eLearning courses.

Learn more about:

IEEE membership

IEEE Xplore subscriptions

2 Author(s)
Komaee, A. ; Dept. of Chem. Eng., Massachusetts Inst. of Technol., Cambridge, MA, USA ; Bensoussan, A.

A hidden Markov model (HMM) consisting of a controlled Markov chain and a binary output is considered. The binary output reports the occurrence of a certain state (versus any other state) in the Markov chain. The transition matrix of the Markov chain depends on a control vector which is to be chosen to minimize a discounted infinite-horizon cost function. The information provided for decision making is the entire history of the binary output which presents an incomplete knowledge of the Markov chain and, as a consequence, leads to a stochastic control problem with partial observations. The solution to such a problem is a control policy consisting of two components: an estimator to derive the posterior probability distribution of the Markov chain from the observation set, and a nonlinear control law to map this estimate into the control vector. This paper introduces a nonlinear state-space representation for the estimator, develops analytical expressions for the control law, and presents numerical methods for efficient computation of the optimal control. Application of these results to optimal management of a class of inventory systems is discussed, and the performance of optimal control for this application is examined by numerical simulations.

Published in:

Automatic Control, IEEE Transactions on  (Volume:59 ,  Issue: 1 )

Date of Publication:

Jan. 2014

Need Help?

IEEE Advancing Technology for Humanity About IEEE Xplore | Contact | Help | Terms of Use | Nondiscrimination Policy | Site Map | Privacy & Opting Out of Cookies

A not-for-profit organization, IEEE is the world's largest professional association for the advancement of technology.
© Copyright 2014 IEEE - All rights reserved. Use of this web site signifies your agreement to the terms and conditions.