By Topic

Practical numerical methods for stochastic optimal control of biological systems in continuous time and space

Sign In

Cookies must be enabled to login.After enabling cookies , please use refresh or reload or ctrl+f5 on the browser for the login options.

Formats Non-Member Member
$31 $13
Learn how you can qualify for the best price for this item!
Become an IEEE Member or Subscribe to
IEEE Xplore for exclusive pricing!
close button

puzzle piece

IEEE membership options for an individual and IEEE Xplore subscriptions for an organization offer the most affordable access to essential journal articles, conference papers, standards, eBooks, and eLearning courses.

Learn more about:

IEEE membership

IEEE Xplore subscriptions

2 Author(s)
Simpkins, A. ; Dept. of Mech. & Aerosp. Eng., Univ. of California, San Diego, CA ; Todorov, E.

Previous studies have suggested that optimal control is one suitable model for biological movement. In some cases, solutions to optimal control problems are known, such as the Linear Quadratic Gaussian setting. However, more general cost functionals and nonlinear stochastic systems lead to optimal control problems to which direct solutions are presently unknown but these solutions would theoretically model behavioral processes. Additionally, in active exploration-based control situations, uncertainty drives control actions and therefore the separation principle does not hold. Thus traditional approaches to control may not be applicable in many instances of biological systems. In low dimensional cases researchers would traditionally turn to discretization methods. However, biological systems tend to be high dimensional, even in simple cases. Function approximation is an approach which can yield globally optimal solutions in continuous time and space. In this paper, we first describe the problem. Then, two examples are explored demonstrating the effectiveness of this method. A higher dimensional case which involves active exploration to learn unobservable parameters and the numerical challenges which arise will be addressed. Throughout this paper, multiple pitfalls and how to avoid then are discussed. This will help researchers to avoid spending large amounts of time merely attempting to solve a problem because a parameter is mistuned.

Published in:

Adaptive Dynamic Programming and Reinforcement Learning, 2009. ADPRL '09. IEEE Symposium on

Date of Conference:

March 30 2009-April 2 2009