By Topic

Switching models for nonstationary random environments

Sign In

Cookies must be enabled to login.After enabling cookies , please use refresh or reload or ctrl+f5 on the browser for the login options.

Formats Non-Member Member
$33 $13
Learn how you can qualify for the best price for this item!
Become an IEEE Member or Subscribe to
IEEE Xplore for exclusive pricing!
close button

puzzle piece

IEEE membership options for an individual and IEEE Xplore subscriptions for an organization offer the most affordable access to essential journal articles, conference papers, standards, eBooks, and eLearning courses.

Learn more about:

IEEE membership

IEEE Xplore subscriptions

2 Author(s)
B. J. Oommen ; Sch. of Comput. Sci., Carleton Univ., Ottawa, Ont., Canada ; H. Masum

Learning automata are stochastic finite state machines that attempt to learn the characteristic of an unknown random environment with which they interact. The fundamental problem is that of learning, through feedback, the action which has the highest probability of being rewarded by the environment. The problem of designing automata for stationary environments has been extensively studied. When the environment is nonstationary, the question of modeling the nonstationarity is, in itself, a very interesting problem. In this paper, the authors generalize the model used in Tsetlin (1971, 1973) to present three models of nonstationarity. In the first two cases, the nonstationarity is modeled by a homogeneous Markov chain governing the way in which the characteristics change. The final model considers the more general case when the transition matrix of this chain itself changes with time in a geometric manner. In each case the authors analyze the stochastic properties of the resultant switching environment. The question of analyzing the various learning machines when interacting with these environments introduces an entire new avenue of open research problems

Published in:

IEEE Transactions on Systems, Man, and Cybernetics  (Volume:25 ,  Issue: 9 )