MDP-based outpatient scheduling for multiple examinations

Timely examinations are important for the patients to be properly diagnosed and treated. Different urgency levels of patients, different requirements of examinations and patients' behavior make the patients scheduling difficult to solve. To deal with this problem, this paper starts from two examinations, and proposes a Markov Decision Process (MDP) model for scheduling six types of outpatients by considering two different waiting time targets and no-shows. The objective is to maximize the expected revenue of accepting outpatients and minimize the expected overtime penalty. Numerical experiments are conducted to show the optimal policy for each type of patients. Based on the observation, a heuristic rule is proposed for scheduling patients. Numerical experiments show that the performance of the proposed heuristic rule is quite close to the optimal control policy and much better than first-come-first-serve rule, especially when the workload of the system is heavy.