Reinforcement Learning Based Proactive Entanglement Swapping for Quantum Networks

Entanglement generation and swapping is a difficult process due to probabilistic nature of quantum mechanics. To overcome this issue, existing quantum routing algorithms try to create entanglement on multiple paths between source and destination. Although it is possible to save entangled qubits on unused links using quantum memories, the quantum routing algorithms discard them and try creating new entanglement in each time slot. In this work, we leverage the longevity of entanglement and introduce two enhancements to improve the performance of existing routing algorithms: (i) The generation and caching of entanglements across multiple time slots, and (ii) the proactively executing entanglement swapping for frequently utilized links, increasing the success rate of future requests. Since entanglement caching reduces available quantum memory on quantum repeaters, we apply reinforcement learning to identify the optimal set of segments that are likely to be used in the future. Through comprehensive simulations, we demonstrate that caching unused entanglements lead to 10 - 15% improvement on the performance of quantum routing algorithms REPS and SEER. Complementing the caching with reinforcement learning based proactively entanglement swapping leads to further performance improvements. Specifically, using Q-Learning to decide on which segments to proactively swap leads to 52.55% and 41.79% improvement for REPS and SEER algorithms, respectively. Deep Q-Learning outperforms Q-Learning with 61.43% and 48.27% improvements rates.