Heuristic Reward Design for Deep Reinforcement Learning-Based Routing, Modulation and Spectrum Assignment of Elastic Optical Networks

In this letter, we study the deep reinforcement learning (DRL)-based routing, modulation and spectrum assignment problem in the elastic optical networks. We emphasize the importance of proper reward design of the DRL framework and we propose to include some heuristic information to the reward design. This introduction of human knowledge to the machine learning, is to reduce the exploration blindness of the latter and lead to more efficient learning of better policies. We make it clear that what kind of heuristic information should be included in the reward design is an open question. Specifically, we propose to consider the spectrum fragmentation level of each candidate path as the heuristic information. As a result, the DRL agent is more inclined to choose the candidate path that leads to lower spectrum fragmentation level, which is more friendly for future traffic requests. Simulation results show that the proposed heuristic reward design scheme outperforms both the simple-reward DRL based approaches and the heuristic rule-based approaches.