Dynamic routing and wavelength assignment using first policy iteration

With standard assumptions the routing and wavelength assignment problem (RWA) can be viewed as a Markov decision process (MDP). The problem, however defies an exact solution because of the huge size of the state space. Only heuristic algorithms have been presented up till now. We propose an approach where, starting from a given heuristic algorithm, one obtains a better algorithm by the first policy iteration. In order to estimate the relative costs of states, we make a simulation on the fly studying, at each decision epoch, the consequences of all the alternatives actions. Being computationally intensive, this method can be used in real time only for systems with slow dynamics. Offline it can be used to assess how close the heuristic algorithms come to the optimal policy. Numerical examples are given about the policy improvement.