Experimental Results of an MPC Strategy for Total Intravenous Anesthesia

In this paper we present experimental results obtained by applying a Model Predictive Control strategy for the control of Depth of Hypnosis in general anesthesia. In particular, the Bispectral Index Scale is employed as the controlled variable and the administration of the hypnotic drug propofol is determined by applying a Generalized Predictive Control algorithm, which considers a process model that is linearized by means of an external predictor. The results of four patients who underwent elective plastic surgeries are analyzed and discussed, showing the feasibility of the approach in a practical context.


I. INTRODUCTION
General anesthesia consists in achieving a desired hypnotic and analgesic state of the patient during a surgery through the appropriate administration of drugs. In particular, in Total Intravenous Anesthesia (TIVA), the required Depth-of-Hypnosis (DoH) is generally obtained by means of the hypnotic drug propofol, while the required analgesia is achieved by means of the analgesic drug remifentanil [1].
Feedback control of DoH has attracted the attention of many researchers since sensors to measure the level of hypnosis have become available [2], [3]. In fact, the use of a control system where the anesthesiologist acts only as a supervisor might provide significant advantages in terms of patient's safety [4], [5]. Indeed, many methodologies for the design of a feedback control system for the DoH have been proposed in the last decades. They usually employ the Bispectral Index Scale (BIS) sensor, which provides a number between 0 and 100 as the process variable, while the control variable is the propofol infusion rate.
In this context, two main control strategies that have been devised are those based on Proportional-Integral-Derivative (PID) controllers and those based on Model Predictive The associate editor coordinating the review of this manuscript and approving it for publication was Xiwang Dong. Control (MPC) algorithms. Regarding PID-based controllers, different tuning methodologies have been devised [6], [7], [8] and experimental results have proven the effectiveness of such a kind of approach in the clinical practice [9], [10], [11]. However, despite the clear advantage of their simplicity, PID controllers have the drawback of being inherently unable to explicitly handle constraints and to optimize the performance.
On the other hand, MPC control is strongly appealing because of its capacity of fully exploiting the possible availability of an accurate pharmacokinetic/pharmacodynamic (PK/PD) model and to explicitly optimize the operations by taking into account the process constraints. Different MPC methodologies have been proposed in the literature and they usually linearize the Wiener PK/PD process model by inverting the static nonlinearity. In [12] and [13] a Kalman filter is used to estimate the patient state. The approaches proposed in [14], [15], [16], and [17] are based on a hybrid multi-parametric-MPC (mp-MPC) algorithm. Then, the use of the Extended Prediction Self-Adaptive Control (EPSAC) technique has been proposed in [18]. However, despite these efforts, there are only a few papers where experimental results are provided. In particular, in [19] the model considers both a continuous infusion and the administration of boluses, in [20] an adaptive fuzzy model is employed, and in [21] the EPSAC approach has been applied in a post-operative intensive care context. It appears that there is still the need of providing experimental results to demonstrate the effectiveness of MPC from a clinical perspective.
Recently, an MPC strategy based on an external predictor has been presented in [22], where the differences with respect to the approaches previously proposed in the literature have also been discussed. The devised control technique exploits a patient model to build the control architecture and design the MPC controller. This model considers the available physical information of the patient like age, weight, height and gender, thus allowing the design of a patient's individualized predictive control system. The noise compensation and the selection of an appropriate sampling period have then been further addressed in [23]. In this paper we present the first experimental results obtained by using this methodology. The aim is to show that the algorithm can be practically implemented and it is capable to successfully deal with the measurement noise, with a low signal quality index (SQI) of the BIS and with interventions of the anesthesiologist.
The paper is organized as follows. The employed MPC algorithm is briefly reviewed in Section II for the reader's convenience. The experimental setup is described in Section III, while the experimental results are presented and discussed in Section IV. Finally, conclusions are given in Section V.

II. MPC ALGORITHM
The anesthesia process consists of three main phases. In the induction phase the BIS value should be driven from its initial value (close to 100 when the patient is awake) to its set-point value 50. This transition should be done as fast as possible (less than 5 minutes) in order to reduce the patient's discomfort, but an excessive undershoot (below 30) should be avoided to reduce the risk of burst suppression [24], [25]. Then, when the BIS settles in a range between 40 and 60, the maintenance phase starts, where the BIS value should be kept in that range during the whole surgery, despite the presence of noxious stimuli. Finally, in the emergence phase, the administration of drugs is stopped and the patient regains consciousness.
The MPC technique proposed in [22] and [23] aims to achieve the control requirements by optimizing the propofol administration at each sampling interval, basing on a nominal PK/PD model. The robustness of the system with respect to intra-and inter-patient variability and the compensation of the measurement noise are achieved by the presence of suitably tuned low-pass filters.
The control scheme is shown in Figure 1. It exploits a threecompartment PK/PD model of the patient, which consists of the series of a linear part P and a static nonlinear part H [8]. The linear part is a fourth-order transfer function that describes the relationship between the propofol infusion rate u and the effect-site concentration C e : where V 1 is the volume of the primary compartment, z 1 , z 2 , p 1 , p 2 , p 3 are parameters, derived from the PK/PD state space model representation, that depend on the patient's characteristics (height, weight, age, gender), whereas k e0 = 0.456 is the frequency of the drug removal from the effect-site compartment. The nonlinear part that describes the relationship between the effect-site concentration and the BIS value is modelled by means of the Hill function H : where E 0 is the BIS value when the patient is still awake before the induction, E 0 − E max is the maximum reachable effect by means of the propofol administration, C e 50 is the concentration that is needed to obtain half of the maximal effect and γ is the steepness of the curve when C e (t) = C e 50 .
In the control scheme, the blocks P and H are used in the external predictor of the controller to obtain the feedback signalỹ that is used by a standard GPC controller. That signal contains the information about the disturbances occurring on the process and the unavoidable modelling uncertainties (note that, although a nominal model of the linear part can be obtained from the patient demographics, the Hill function parameters are unknown and only average values can be used [26]). Due to the characteristics of the experimental study (where the effect-site concentration cannot be measured) and of the closed-loop system dynamics, the detailed information on the plant-model mismatch cannot be easily obtained. However, the GPC internal predictions are performed under receding horizon strategy, thus limiting the influence of the prediction errors over large horizon. Additionally, the use of the external predictor and of the filters improves the overall robustness of the system to modelling uncertainties. As already mentioned, the block with the inverse of the Hill function implies a linearization of the process that allows the use of a standard GPC controller. The filters F n , F d and F r are first-order low-pass filters, whose time constants are, respectively, T n , T d and T r . The purpose of F n is to filter high peaks of the measurement noise, while F d provides the necessary robustness to the system and F r provides the second degree-of-freedom (set-point following) to the control architecture.
The GPC controller determines, at each sampling time, the infusion rate that minimizes a cost function that considers the sum of the error between the reference and the predicted BIS values (over the prediction horizon) and the future control effort (over the control horizon) weighted by the coefficient λ. In this context, constraints related to the saturation values and maximum slew-rate of the infusion pumps are also taken into account.
It is worth highlighting that simple tuning rules for the MPC controller cannot be used, since the overall performance depends also on the external predictor parameters (the time constants of filters) that need to be adjusted simultaneously. For this reason, the prediction and control horizons N and N u of the GPC controller, the control effort weighting/scaling  parameter λ and the three time constants T n , T d and T r , have been obtained by minimizing the worst-case integrated absolute error (where the error is the difference between the reference value r and the BIS value y) for a dataset of 13 patients that are representative of a wide population [18]. The optimization has been performed by means of a genetic algorithm with a population size of 40 elements [22]. The initial population has been generated by using a uniform distribution and the mutation has been performed by means of a Gaussian mutation function. The optimization has been stopped when the relative change in the cost function value over the last 60 iterations was less than 0.001. The obtained parameters for the implementation with a sampling period of T m = 1 s are shown in Table 1.

III. EXPERIMENTAL SETUP
The experimental setup consists of a Dräger Infinity Delta monitor (Drägerwerk, Lübeck, D) that provides the BIS value and two Alaris GH syringe pumps (one for propofol and one for remifentanil). They are connected to a PC, where the control algorithm is implemented, through three USB-RS232 converter cables. Given that the BIS values are sent to the controller with a frequency of 1 Hz, the sampling period of the control algorithm has been selected as T m = 1 s, which provides the best performance achievable by the MPC controller [23]. However, the control signal has then been downsampled by sending a new infusion rate to the propofol syringe pump every 5 s to avoid causing excessive mechanical stress. This value is calculated as the average of the last five samples of the controller output.
The anesthesiologist can supervise the behaviour of the controller through a suitably designed graphical user interface (GUI), shown in Figure 2. On the left side of the GUI there is a box dedicated to system initialization, where it is required to insert the patient's demographic data and the COM ports where the medical devices are connected. Then, there is a box to select the BIS target value (whose default value is 50), a box to insert notes and a box where the status of the connected medical devices is shown by means of red and green indicators. In the central part of the GUI there is a box where it is possible to interact with the control system algorithm. In particular, on the top part there are the buttons to start and stop the control algorithm and to temporarily suspend it. The blue button ''Switch Mode'' can be used to switch the control system mode between induction and maintenance. In fact, even if the controller parameters remain the same in both phases, the constraints posed in the optimization problem of the GPC controller change between the induction and the maintenance phases. The selected mode is indicated with a green light. The two ''Change'' buttons can be used to safely replace an empty drug syringe from the syringe pumps. Additional boluses can be performed by the anesthesiologist, if deemed to be necessary, by pressing the orange ''Bolus'' buttons for the required amount of time. After the button is released, the boluses administration is immediately stopped for safety reasons. The amount of drug that are infused are displayed in real time in two boxes on the right of the ''Bolus'' buttons. The yellow button in the ''Manual Control'' box can be used by the anesthesiologist to switch the system from automatic control to manual control and vice-versa. When manual control is activated a blinking yellow light indicates this situation to the user. The propofol and remifentanil infusion rates are decided by the anesthesiologist by means of the two text boxes placed in the ''Manual Control'' box.
Manual control is also automatically forced, for safety reasons, when the BIS value is not received by the controller or its SQI falls below the safety threshold of 40. In particular, if one of these conditions happens for more than 5 s (for example, because the BIS sensor is removed from the forehead of the patient, or because the use of an  electrocautery device introduces electromagnetic noise that impairs the SQI), an alarm appears in the GUI and manual control is applied. The situation is reversed, that is, closedloop control is reactivated, when the BIS value with a SQI more than 40 is received again by the controller for more than 5 s. On the right side of the GUI there are plots and indicators that allow the anesthesiologist to easily supervise the system during its functioning. The control software also records on a file patient's demographic data, controller logs, pumps infusion rates and the patient's physiological data read from the monitor. These data are then used to evaluate the performance of the control system.

IV. EXPERIMENTAL RESULTS
The MPC controller for the propofol administration has been implemented in the setup described in Section III. The remifentanil infusion rate is manually regulated by the anesthesiologist by means of the specific text box placed in the ''Manual Control'' box on the GUI. In fact, it can be freely chosen by the anesthesiologist and it can be changed at any time during the experiment. Anesthesia is induced automatically by the single-input-single-output (SISO) MPC controller as regards propofol infusion, while for the analgesic component a 1-2 µg/kg bolus of fentanyl is manually administered.
Four patients undergoing general anesthesia for elective plastic surgery have been enrolled in the experiment, their demographic data and the type of surgical procedure are shown in Table 2. The individual records of the clinical variables of interest are shown in Figures 3-6. In order to better evaluate the results, some typical performance indexes have been computed. Regarding the induction phase, they are:    BIS is the measured BIS value, j is the sample number and N is the number of PE values.
• MDAPE (median absolute performance error): it provides a measure of the inaccuracy of the control system. It is determined as: • WOBBLE: it provides a measure of the intra-patient variability. It is calculated as: • Propofol: it is the propofol average infusion rate during the maintenance phase.
• Remifentanil: it is the remifentanil average infusion rate during the maintenance phase. infusion rates in ml/h; heart rate; systolic (red), diastolic (black) and mean (blue) blood pressure.
• T awakening: it is the time-to-extubation. It is determined as the time interval between the time instant when the drug administration is stopped and the time instant when the laryngeal mask or the endotracheal tube is removed. In other words, it is the time interval taken by the patient to breathe autonomously after the end of the anesthesia. The values of the performance indexes related to the induction phase are shown in Table 3. The controller provided a fast induction of anesthesia. For all the patients anesthesia was induced in less than 2 minutes, which is a time interval comparable with that obtained by using a bolus. This was paid at the cost of a slight undershoot of the BIS, as highlighted from the value of the Lowest BIS. However, this undershoot did not cause any clinically relevant consequence on the hemodynamic variables. Indeed, blood pressure (BP) infusion rates in ml/h; heart rate; systolic (red), diastolic (black) and mean (blue) blood pressure. The vertical dash-dotted black line indicates a manual bolus of propofol. The constant values in the propofol infusion rate are due to system automatically switching to manual control due to low SQI. Missing data in the BP were due to temporary issues with the sensor that however did not interfere with the operation of the control system. infusion rates in ml/h; heart rate; systolic (red), diastolic (black) and mean (blue) blood pressure. The constant values in the propofol infusion rate are due to system automatically switching to manual control due to temporary issues with the BIS sensor reading. The sharp rise in the values of BP around minute 45 is due to an ephedrine bolus. Missing HR and BP data were due to temporary issues with the sensors that however did not interfere with the operation of the control system. and heart rate (HR) remained within clinically recommended ranges. Moreover, the administered propofol dose in induction varies from 1.91 mg/kg to 2.52 mg/kg and they are   Table 4. The controller was able to keep the BIS inside the recommended range from a minimum of 58% to a maximum of 75% of the maintenance time. The MDAPE values indicate that the control inaccuracy remains below the threshold of 20% of the BIS target value. This means that the median absolute BIS value is inside the recommended range from 40 to 60. The MDPE values indicate that the obtained BIS has a negative bias with respect to its target value. This means that the controller tends to keep the BIS below 50. However, MDPE remains above the threshold of -20% of the BIS target value. This means that the median BIS value is above 40. By analyzing the values  ; propofol (black) and remifentanil (red) infusion rates in ml/h. Missing BIS data were due to temporary issues with sensor reading forcing the system to automatically switch to manual control.
of BIS < 40, it appears that, when the BIS is not between 40 and 60, it is mainly below 40 rather than over 60. This is appreciable from a clinical point of view as it implies that the risk of intra-operative awareness is reduced. It has to be remarked that this is not obtained by overdosing the patients as indicated by the propofol maintenance dose that is, in each patient, below that normally used in the clinical practice of 6 mg/kg/h. The remifentanil maintenance dose is shown in Table 4 but it is not managed by the controller, indeed it is manually regulated by the anesthesiologist. As regards the emergence phase, the awakening times were short for all the patients enrolled and typical of the clinical practice. The automatic control system was able to manage the whole anesthesia procedure and there was no need for the anesthesiologist to apply corrective actions. Only in patient 3 the anesthesiologist performed an additional bolus of propofol around minute 62 as a preventive measure for a particularly painful part of surgery on burned tissues. It is worth noting that the controller performance was not affected by this manual intervention. The hemodynamic variables remained stable for patient 1 and patient 3. In Patient 2 there was a rise of BP around minute 62 in response to painful stimulation. In patient 4 the sharp rise in the value of BP around minute 45 was caused by the administration of ephedrine to treat low BP.
In patient 1 an oscillatory behavior of the BIS is observed. However, oscillations remained bounded as indicated by the WOBBLE, which is 12% of the BIS target value, and they did not cause any significant consequences from a clinical point of view. In patient 2 the prolonged undershoot of the BIS that occurred after induction was due to the rejection of a rise in the BIS signal that occurred around minute 3 due to intubation. Then, around minute 62 another rise of the BIS signal due to painful stimulation was promptly rejected by the controller. In patient 3 the BIS remained stable with the exception of a sharp rise around minute 62 due to a particularly painful stimulation. In patient 4 the BIS remained stable throughout the whole surgical procedure although there were issues with the BIS sensor reading and the system automatically switched to manual control as explained in Section III.
It is worth stressing that the purpose of this experiment is not to show that the MPC-based control system outperforms the manual control provided by the anesthesiologist. Conversely, it aims to prove the feasibility of this control technique as a supporting tool for the anesthesiologist. In this context, the SISO MPC controller showed a satisfactory performance and a behavior that is compatible with the clinical practice. Note again that, in the design of the SISO MPC controller, the infusion of remifentanil is not taken into account, while in the clinical practice it is always present. Hence, from the controller point of view this is seen as a disturbance. In fact, the oscillatory behavior of the BIS seen in patient 1 and the sharp rises in patient 2 and patient 3 can be due to a not optimal management of the remifentanil that is, indeed, manually regulated. Also the undershoot of the BIS seen in the induction phase can be due to the manual administration of fentanyl that is unknown to the controller.

V. CONCLUSION
In this paper we have presented experimental results obtained with an MPC control strategy for general anesthesia. The required DoH has been achieved for all the enrolled patients in both the induction and maintenance phases despite the unavoidable modelling uncertainties. Thus, with these promising results, we have demonstrated the applicability of the proposed control strategy in the clinical practice.
Future studies will investigate on how the overall performance can be improved by developing more accurate models, by using for example an online identification and/or adaptive MPC controller for such a kind of application. Additionally, future work will focus on an MPC controller that explicitly takes into account also the infusion of remifentanil, that is, for a two-inputs-one-output process.