Safe, Fast and Explainable Online Reinforcement Learning for Continuous Process Control

Industrial process control using model-based technologies is well established. These technologies are typically non-adaptive and so have limitations. Reinforcement Learning (RL) provides a model-free adaptive alternative. RL is a type of machine learning (ML) where models or data sets of the environment are not necessary before learning can start. It generates data, by exploring the environment and then learn the behavior from it. Though RL has been successfully applied for learning and playing various games such as Go, Chess, Atari; its application to continuous process control problems is not trivial. There is a need for online RL implementation to be safe, fast learning and explainable when applied to industrial control problems. Rather than adding to the extensive research on augmenting existing RL algorithms, the work focuses on developing a unique systematic method of formulating the RL problem incorporating domain-specific knowledge about process constraints and objectives, reducing dimensionality and modifying the exploration process, applicable to any model free RL algorithm supporting continuous states and actions, to enhance safety, speed and explainability of online RL implementation without requiring a simulation model. The approach is successfully implemented on two multivariable processes: a simulated distillation column and a temperature control lab setup using the Deep Deterministic Policy Gradient (DDPG) algorithm. The work demonstrates that the developed method is applicable to multivariable, noisy, non-linear processes with disturbances. It will further the potential of introducing the advances in Artificial Intelligence and ML algorithms for intelligent process control capable of enabling autonomous operation in the process industry.