An Empirical Comparison of Optimizers for Quantum Machine Learning with SPSA-Based Gradients

Variational quantum algorithms (VQAs) have attracted a lot of attention from the quantum computing community for the last few years. Their hybrid quantum-classical nature with relatively shallow quantum circuits makes them a promising platform for demonstrating the capabilities of noisy intermediate scale quantum (NISQ) devices. Although the classical machine learning community focuses on gradient-based parameter optimization, finding near-exact gradients for variational quantum circuits (VQCs) with the parameter-shift rule introduces a large sampling overhead. Therefore, gradient-free optimizers have gained popularity in quantum machine learning circles. Among the most promising candidates is the simultaneous perturbation stochastic approximation (SPSA) algorithm, due to its low computational cost and inherent noise resilience. We introduce a novel approach that uses the approximated gradient from SPSA in combination with state-of-the-art gradient-based classical optimizers. We demonstrate numerically that this out-performs both standard SPSA and the parameter-shift rule in terms of convergence rate and absolute error in simple regression tasks. The improvement of our novel approach over SPSA with stochastic gradient descent (SGD) is even amplified when shot- and hardware-noise are taken into account. We also demonstrate that error mitigation does not significantly affect our results.