Q-Learning Based Bi-Objective Deadlock-Free Routing Optimization for Optical NoCs

Silicon photonics-based optical networks-on-chip (ONoCs) have been considered as a competitive alternative to electronic NoCs for chip multiprocessors (CMPs). However, the presence of thermooptic effects may prevent them from maintaining power efficiency advantages over electronic counterparts. To address the thermal issue, some routing schemes based on reinforcement learning have been proposed to reduce the optical power loss. Although thermal-related optical power loss could be minimized by routing packets along paths with the lowest thermal gradients, contention may occur on these paths. In this work, we first propose a contention-aware deadlock-free routing (CDR) for electronic-controlled ONoCs, using Q-learning model for single-objective optimization for contentions. Then we further propose a Q-learning based bi-objective deadlock-free routing scheme (BODR) for ONoCs under thermal variations, to comprehensively improve both the power efficiency and the network performance. Experimental results show that BODR routing can greatly reduce the delay with a small cost of optical power loss compared with the single-objective Q-routing for power optimization. Meanwhile, the proposed method can improve the optical power loss while simultaneously achieving a higher network performance as compared with other traditional adaptive routing schemes.