Nonstationary models of learning automata routing in data communication networks

In a data communication network the message traffic has peak and slack periods and the network topology may change. When the learning approach is applied to routing, a learning automaton is situated at each node in the network. Each automaton selects the routing choices at its node and modifies its strategy according to network conditions. In this paper a new model of a nonstationary automaton environment is proposed whose response characteristics are dynamically related to the probabilities of the actions performed on it. As such it represents an extension of earlier models proposed. The limiting behavior of the new model is identical to that of the earlier models. Simulation studies of automata operating in simple queueing networks reinforce these analytical results and show that the parameters of the new model can be chosen to predict transient behavior.