Neuromechanics-based Deep Reinforcement Learning of Neurostimulation Control in FES cycling

Functional Electrical Stimulation (FES) can restore motion to a paralysed's person muscles. Yet, control stimulating many muscles to restore the practical function of entire limbs is an unsolved problem. Current neurostimulation engineering still relies on 20th Century control approaches and correspondingly shows only modest results that require daily tinkering to operate at all. Here, we present our state-of-the-art Deep Reinforcement Learning developed for real-time adaptive neurostimulation of paralysed legs for FES cycling. Core to our approach is the integration of a personalised neuromechanical component into our reinforcement learning (RL) framework that allows us to train the model efficiently-without demanding extended training sessions with the patient and working out-of-the-box. Our neuromechanical component includes merges musculoskeletal models of muscle/tendon function and a multi-state model of muscle fatigue, to render the neurostimulation responsive to a paraplegic's cyclist instantaneous muscle capacity. Our RL approach outperforms PID and Fuzzy Logic controllers in accuracy and performance. Crucially, our system learned to stimulate a cyclist's legs from ramping up speed at the start to maintaining a high cadence in steady-state racing as the muscles fatigue. A part of our RL neurostimulation system has been successfully deployed at the Cybathlon 2020 bionic Olympics in the FES discipline with our paraplegic cyclist winning the Silver medal among 9 competing teams.