Completely Uncoupled Utility Maximization Algorithms for State Dependent Networks

We study a completely uncoupled resource allocation algorithm for a heterogeneous network with the objective of maximizing the sum of the utilities (on the average pay-off) of users. We consider a state-dependent network, where the pay-off achieved by the users are a function of their actions as well as the state of the system. We consider four different scenarios depending on the state evolution and the users’ knowledge of the system state. In this context, we present completely uncoupled algorithms for utility maximization, where the users’ action is entirely a function of its past actions and its received pay-off. In particular, the user is oblivious to the actions of the other users in the network. Using the theory of perturbed Markov chains, we show the optimality of our algorithms under appropriate scenarios.