Reinforcement Learning Based Motion Control Method for Electrical-hydraulic Valve-Controlled System

The automation and intelligence development of heavy-duty mobile machines and industrial equipment require the accurate control of electrical-hydraulic valve-controlled system (EHVS). This article presented a generic reinforcement learning with predictive target information control (RLPC) method for EHVS to improve the control accuracy, in which a distributional soft actor-critic (DSAC) iteration frame was established. First, a dynamic model considering nonlinearity, saturation, and uncertainty from a time-varying system perspective was established and validated through tests under typical conditions. Then the model-free RLPC method was designed for EHVS. As the baseline, the classical proportional-integral-derivative (PID) and an adaptive PID (APID) with RL-tuning were designed for comparison purposes. Finally, the corresponding iteration frame was established to improve the performance of RL-related works in the designed control methods with DSAC. The comparison results during typical operations show that the proposed RLPC method can handle dynamic response and steady errors better at different states of the system and actuator. The effects of the predictive horizon on control performance were also revealed. The stability of the method was also validated with a designed instantiating EHVS, specifically a hydraulic inverted pendulum system. The performance of the proposed methods was quantitatively analyzed with the normalizing performance indicator ρ. The mean value of the indicator was about 0.0028, which is lower than the first two methods and the existing control method for EHVS.