Cart (Loading....) | Create Account
Close category search window

Dynamic programming approach to a final-value control system with a random variable having an unknown distribution function

Sign In

Cookies must be enabled to login.After enabling cookies , please use refresh or reload or ctrl+f5 on the browser for the login options.

Formats Non-Member Member
$31 $13
Learn how you can qualify for the best price for this item!
Become an IEEE Member or Subscribe to
IEEE Xplore for exclusive pricing!
close button

puzzle piece

IEEE membership options for an individual and IEEE Xplore subscriptions for an organization offer the most affordable access to essential journal articles, conference papers, standards, eBooks, and eLearning courses.

Learn more about:

IEEE membership

IEEE Xplore subscriptions

1 Author(s)
Aoki, M. ; University of California, Los Angeles, CA, USA

As described in the introduction of this paper, the state vector xn, representing the present state of a sampling control system, is assumed to satisfy a difference equationx_{n+1} = T(x_{n}, r_{n}, upsilon_{n}), whereupsilon_{n}is the control vector of the system subject to random disturbances rn. The random variables rnare assumed to be independent of each other and to be defined in a parameter space in the following manner: Nature is assumed to be in one of a finite number,q, of possible states and each state has its own parameter value, thus specifying uniquely and unequivocally the distribution function of rn. It is further assumed that we are given the a priori probabilityz =(z_{1}, . . . , z_{q})of each possible state of nature,i= 1,2, . . . , q,summin{i=1}max{q} z_{i}=1, z_{i} geq 0. Given a criterion of performance, the duration of the process and the domain of the control variable, a sequence of control variables {upsilon_{n}} is to be determined as a function of the state vector of the system and time, so as to optimize the performance. The sequential nature of the determination ofupsilon_{n}is evident here, since the stochastic nature of the problem prevents specification of such a sequence ofupsilon's as a function of the initial state and time. By means ot the functional equation technique of dynamic programming, a recurrence relation of the criterion function of the process,k_{n}(x, z), is derived, wherexis the state variable of the system when there remainncontrol stages. Whenqis taken to be two and the criterion of performance to hex_{N}^{2}with the constraint on the control variablessummin{i=0}max{N-1} upsilon_{i}^{2} leq K, where xnis the final state vector of the system, it is shown thatk_{n}(x, z)is the formw_{n}(z)x^{2}and the optimalupsilon_{n}(x, z)is linear inx, as might be expected. The dependence ofk_{n}(x, z)andupsilon_{n}(x, z)onzis investigated further, andw_{n}(z)is found to he concave inz. Explicit quadrat- ic forms forw_{n}(z)are obtained in the neighborhood ofz = 0and 1. The optimalupsilon_{n}(x, z)is found to be monotonically decreasing inz. When the domain ofupsilon_{n}is restricted to a finite set of values, as in contactor servo systems, no explicit expressions fork_{n}(x, z)andupsilon_{n}(x, z)are available. However,k_{n}(x, z)is still concave inzand approximately given byzk_{n}(x, 1) + (1 - z)k_{n}(x, 0). By solving the recurrence relation numerically, this approximation is found to be very good for moderately large n, say 10. This means that if one has explicit solutions forp =p_{1}andp = P_{2}, then one can approximate the criterion function for the adaptive case wherePr(p =p_{1})= zby a linear function inzas shown above. This provides fairly good lower bound onk_{n}(x, z), and in turn serves to determine an initial approximate policy. A concept of suboptimal policy is introduced and numerical experiments are performed to test suboptimal policy suggested from numerical solution. The system behavior under the suboptimal policy is also discussed.

Published in:

Automatic Control, IRE Transactions on  (Volume:5 ,  Issue: 4 )

Date of Publication:

Sep 1960

Need Help?

IEEE Advancing Technology for Humanity About IEEE Xplore | Contact | Help | Terms of Use | Nondiscrimination Policy | Site Map | Privacy & Opting Out of Cookies

A not-for-profit organization, IEEE is the world's largest professional association for the advancement of technology.
© Copyright 2014 IEEE - All rights reserved. Use of this web site signifies your agreement to the terms and conditions.