Glucose level control using Temporal Difference methods

Control theory has been widely used in various fields; one of these areas is medical issues. Diabetes is one of the new topics of interest in control. Obtaining the rates for the injection of insulin automatically always been a concern of physicians. The purpose of the control and treatment of diabetes, is keeping blood glucose in the normal range as possible. In this paper, we used Sarsa method - which is an on-policy Temporal Difference (TD) technique - for insulin delivery rate. TD methods are the most known methods for solving reinforcement learning problem. Because TD methods don't require a precise model of environment dynamics; they have absorbed interests in medical applications during recent years. Although temporal difference methods don't require a mathematical model of the environment, but for simulating an environment, we used Palumbo mathematical model instead of real patients. Since patients' medical parameters vary from person to person, for controlling the disease we should have different drug schedules, in other word, we should have different controller for each patient. While RL methods, by interacting with their environment, automatically define suitable doses for each person. If we want less trial and error on real patients and therefore reduce the side effects of changes in dose on the patient; according to the parameters of a patient, we design a controller which estimate the appropriate insulin injection rate. Then the drug program can be applied to other real patients. At this stage controller (applies Sarsa algorithm) with less trial and error, determines the appropriate dose for real patient. The results of the simulations, represents the efficiency of the proposed method.