Achieving Reinforcement Learning Speedup via Quantum Kernel Estimation

Reinforcement learning is playing a crucial role in sustainable developments and enabling wide applications in our daily lives ranging from self-driving cars, robotics, automation, personalised product suggestions, user notifications and many more. Kernel-based reinforcement learning holds theoretical guarantees of convergence to a unique solution for decision function approximation. This has motivated the development of implementations that exploit the advantage of kernel methods whilst overcoming the ‘curse of dimensionality’. The theoretical properties of kernel-based methods also support the combination of learning agents in loosely coupled systems into a single combined model for composable learning. Learning speedup is achievable by introducing expert knowledge to the algorithm through the kernel, which reduces variance by increasing bias and thus learning speed. Despite the theoretical promise of kernel based reinforcement learning methods, and continued evolution of more efficient algorithms, practical implementation are not prevalent. Research in quantum machine learning has successfully applied quantum algorithms to achieve computational speed-ups for various machine-learning tasks, and has also identified potential quantum advantage through the identification of patterns not accessible to classical data mapping. To date, this has not been applied to kernel based reinforcement learning. Recent work in the field of quantum learning suggests an important duality between quantum machine learning and kernel machine learning methods. Therefore, this paper proposes a framework to complement kernel-based reinforcement learning with quantum generated feature maps for function approximation to improve learning speed. This has positive consequences for optimisation of reinforcement learning techniques already in use to support sustainability targets in applications such as industry innovation and infrastructure, power generation management, data centre cooling and self-driving cars. Our established framework enables the adoption of emerging quantum kernel methods for reinforcement learning applications. The achieved results show over 20 % improvement and positive impacts of quantum map selection on model performance for different problems.