Approximate dynamic programming approach for process control

Many process control and scheduling problems can be formulated as Markov Decision Process (MDP), which represents a multi-stage decision problem under uncertainty. For the past two decades, the Artificial Intelligence (AI) research community has seen significant activities in Approximate Dynamic Programming (ADP), which is developed into an effective method for solving Markov Decision Process (MDP). In this paper, we explore the use of ADP for process control and scheduling applicaitons and examine different design options within ADP, such as the pre-decision state vs. post-decision state value function, parametric vs. nonparametric value function approximator, etc. We show that ADP can be tailored into an effective method to solve stochastic constrained control problems.