By Topic

Numerical methods for the solution of the degenerate nonlinear elliptic equations arising in optimal stochastic control theory

Sign In

Cookies must be enabled to login.After enabling cookies , please use refresh or reload or ctrl+f5 on the browser for the login options.

Formats Non-Member Member
$33 $13
Learn how you can qualify for the best price for this item!
Become an IEEE Member or Subscribe to
IEEE Xplore for exclusive pricing!
close button

puzzle piece

IEEE membership options for an individual and IEEE Xplore subscriptions for an organization offer the most affordable access to essential journal articles, conference papers, standards, eBooks, and eLearning courses.

Learn more about:

IEEE membership

IEEE Xplore subscriptions

2 Author(s)
H. Kushner ; Brown University, Providence, RI, USA ; A. Kleinman

Three distinct but related results are obtained. First, an iterative method is derived for obtaining the solution of optimal control problems for Markov chains. The method usually converges much faster, and requires less computer storage space, than the methods of Howard or Eaton and Zadeh. Second, nonlinear finite difference equations, which "approximate" the nonlinear degenerate elliptic equation (2) arising out of the stochastic optimization problem (1), are found. The difference equations, and their solution, may have a meaning for the control problem even when it cannot be proved that (2) has a solution. The iterative methods for the iterative solution of these nonlinear systems are discussed and compared. Both converge to the solution, provided that the difference equations were derived using the method introduced in the paper; one, new to this paper, often much faster than the other (Theorem 2). In fact, the typical time required for the numerical solution is about the time required for a related linear problem. The method of obtaining the difference equations, and the proof of convergence of the associated iterative procedures, are illustrated by a detailed example. Finally, specific numerical results for a "minimum average time" type of optimization problem are presented and discussed.

Published in:

IEEE Transactions on Automatic Control  (Volume:13 ,  Issue: 4 )