Enhancing Adaptive Deep Brain Stimulation via Efficient Reinforcement Learning

Parkinson’s disease (PD) is a progressive neurological disease primarily impacting movement. Deep brain stimulation (DBS) stands out as a potent therapeutic treatment for addressing PD’s motor symptoms. Nevertheless, optimizing DBS parameters to enhance both effectiveness and efficiency remains a significant challenge. Traditional DBS relies on high-frequency stimulation, lacking adaptability to adequately dress the dynamic nature of PD symptoms. In this paper, we develop a novel method to enhance adaptive DBS utilizing reinforcement learning (RL), while concurrently reducing computational costs through neural network quantization. Our approach entails the discretization of state and action spaces, representing the status of neurons within PD-associated brain regions and the corresponding stimulation patterns, respectively. The primary learning objective is to maximize the accumulated treatment reward, specifically reduced power in the beta frequency band, while maintaining a low stimulation frequency. Additionally, to enhance computing and inference efficiency, we integrate two types of neural network quantization approaches, post-training quantization (PTQ) and quantization-aware training (QAT), with the RL model. Experimental evaluations conducted within a computational model of PD demonstrate the efficacy of our RL approach, even after quantization. While quantization induces some alterations in RL model outputs, the overall DBS efficacy remains unaffected. Furthermore, the animal testing of a rat with PD confirms the effects on both animal behavior and neural activity, underscoring the potential of RL-based adaptive DBS as a promising avenue for personalized and optimized treatment of PD.