Integration of ECMS Control Strategies Using QLearning Algorithm In Hybrid Electric Vehicles

This research paper investigates integrating the Equivalent Consumption Minimization Strategy (ECMS) control strategies with a Q-learning algorithm. This integration was applied to estimate engine and motor torques in a P4 Hybrid Electric Vehicle (HEV). The driving cycles, crucial for the simulation, were sourced from the National Renewable Energy Laboratory website. The ECMS strategy was then used to estimate engine and motor torques. The data generated included vital parameters such as battery State of Charge (SOC), battery voltage, speed, torque requested, motor rpm, gear, motor torque, and engine torque. A Q-learning table was constructed, with columns representing the combinations of motor and engine torque classes and other parameters (environment state). The table’s rows corresponded to the scenarios resulting from the discretization of SOC, voltage, speed, torque requested, motor rpm, and gear. The Q-learning table was populated using the training data, enabling the model to learn optimal torque responses under diverse driving conditions. Finally, the developed Q-learning model was applied to simulate the HEV model under new driving cycles, estimating engine and motor torques. The results were compared with those obtained using the ECMS strategy. The findings revealed that the Q-learning model produced torques resembling the ECMS energy efficiency strategy. Although there was a slight increase in CO2 emissions, this study underscores the potential of integrating machine learning with established control strategies as a transfer learning method to enhance torque estimation in HEVs, thereby contributing to the advancement of HEV technology.