Physiological Closed-Loop Control (PCLC) Systems: Review of a Modern Frontier in Automation

Over the past decade, there has been an unprecedented international focus on improved quality and availability of medical care, which has reignited interest in clinical automation and drawn researchers toward novel solutions in the field of physiological closed-loop control systems (PCLCs). Today, multidisciplinary groups of expert scientists, engineers, clinicians, mathematicians, and policy-makers are combining their knowledge and experience to develop both the next generation of PCLC-based medical equipment and a collaborative commercial/academic infrastructure to support this rapidly expanding frontier. In the following article, we provide a robust introduction to the various aspects of this growing field motivated by the recent and ongoing work supporting two leading technologies: the artificial pancreas (AP) and automated anesthesia. Following a brief high-level overview of the main concepts in automated therapy and some relevant tools from systems and control theory, we explore -- separately -- the developments, challenges, state-of-the-art, and probable directions for AP and automated anesthesia systems. We then close the review with a consideration of the common lessons gleaned from these ventures and the implications they present for future investigations and adjacent research.


I. INTRODUCTION
The modern advance of automated and intelligent systems toward more-and-more visible roles in society (e.g., commercially available automated vehicles, humanoid and biomimetic robots, household vacuums and thermostats, etc.) has accelerated the development of many new technologies and restored interest in several challenging fields. Principal among these recent works, are the broad ongoing efforts to deploy collaborative human-machine systems, such as those designed to provide advanced support in medical diagnostics and therapy. In this review, we focus on a special class of these systems that apply concepts from feedback control to attain automatic regulation of physiological variablessuch as blood pressure, depth of anesthesia, blood glucose concentrations, and others (see examples in Table I) -known as physiological closed-loop control systems (PCLCs). While the pioneering works toward many kinds of PCLCs were performed over a half-century ago, we are now only nascent in an era of suitable computational/technological availability and physiological/pharmaceutical scientific understanding to begin realizing these devices on a broad commercial scale.
From the outset, it is important to understand that these devices do not and are not intended to replace expert and supporting clinical staff. In fact, automated devices rely on the knowledge of experts and simply put, promise to do a much better job at following clinical instruction than present technology. By introducing sensors capable of continuous monitoring (i.e., generating measurements every few seconds or minutes depending on the treatment) these devices stand to recognize and eliminate unintentional variations in therapies, including missed doses and faults like sensor detachment or blocked fluid pathways. Furthermore, post analysis of the measurements obtained by these devices and the treatment profiles they generate can provide important insights into the effectiveness of treatment strategies (e.g., the clinical setpoint is maintained to no effect), possible heterogeneity in disease pathologies, and goals for future pharmaceuticals (e.g., short or longer acting doses). The development and adoption of these new systems is currently providing research directions in control engineering, artificial intelligence, human-machine interactions, social science, medical practice, therapeutic technology, pharmaceutical development, and biological/physiological systems modeling. Further, since poor controller performance in physiological systems can lead to immediate adverse, potentially irreversible, and even fatal physiological responses, PCLCs are a rapidly developing field of safety critical control systems and additional research on safe PCLC operation will be of particular value in the transition to common medical practice (esp. as networked and ambulatory systems are realized).
As a primer for researchers in these and other disciplines, we have sought in this review to introduce key information from various aspects of the ongoing research to provide both a background and up-to-date perception of the field for unfa- Fig. 1. Comparison of (a) manual therapy applied by a clinician according to observations and set guidelines for making therapy adjustments and (b) automated therapy applied by a physiological closed-loop control (PCLC) device according to a control algorithm with the possibility of a clinician acting in a supervisory capacity (e.g. monitoring Bispectral Index (BIS) for additional safety in anesthesia). miliar readers. This includes a brief introduction to modeling and control, within the scope of those methods common to physiological systems, in Section II; the use of case studies in Sections III-IV to frame and discuss important topics in current research; and a minimalist presentation of the detailed mathematical models, which are all very well documented in the cited references. Following the case studies, in Section V we will discuss several aspects from these investigations that are of great importance in the broader scope of physiological control as well as possible directions for the future.

II. PRELIMINARIES
Regardless of the controlled variables and outputs, the general process implemented in PCLCs can be represented conceptually by a diagram resembling Fig. I. Here, a control algorithm -normally implemented on a computer/processor -provides an interface between (i) target values for physiological variables that have been specified by clinicians, (ii) feedback signals containing measurements and estimates of system variables, and (iii) the infusion pumps or other actuators that apply treatment to the patient. Notice, from a high-level perspective, there is little difference between the "automated" approach and the classical manual approach aside from the simplification of tasks required from the clinician/patient, who no longer has to interact with the infusion pump (or other actuator) directly and instead can focus on specifying treatment goals and assessing therapy performance.
Recognizing the generality of this concept, many agencies and organizations are in the process of developing more rigorous guidelines for commercial PCLCs that will help to establish the scope of this up-and-coming class of devices. As noted in a recent paper from researchers at the Center for Devices and Radiological Health (CDRH) at the US Food and Drug Administration (FDA), the CDRH maintains a working definition of PCLC medical devices (or PCLC devices) as, "a medical device that incorporates physiological sensor(s) for automatic manipulation of a physiological variable through actuation of therapy that is conventionally made by a clinician" [9]. This definition -demonstrated in Fig. Ib -recognizes the role of automation in PCLC while allowing a great deal of flexibility in the disorder being treated, the types of sensors and therapies used (many are certainly yet to be discovered), and the extent to which the PCLC operates independent of clinician intervention -known as the level of automation (LOA) [9]. To help guide the development of commercial PCLC devices, the United States [10], Canada [11], and the European Union [12] have formally recognized versions of the recently developed IEC 60601-1-10 consensus standard [13], which is the first standard to provide, "requirements for the development (analysis, design, verification and validation) of a physiologic closed-loop controller (PCLC) as part of a physiologic closed-loop control system (PCLCS) in medical electrical equipment and medical electrical systems to control a physiologic variable" [13]. Beyond regulatory concerns, such guidance may help the broader research and development communities establish a common language (possibly even one that is shared among medical and engineering professionals); a set of safe design practices, testing procedures, and quality management objectives; and a platform to qualify and address issues raised by designers, clinicians, and patients. Treating this feedback system similar to those in more traditional engineering disciplines, the design and implementation of PCLCs requires accurate models of the patient, sensor, and actuator dynamics, including any fundamental limitations and safety constraints. While it may be obvious that the sensor and actuator dynamics can be described using traditional differential equations, similar mathematical modeling techniques can also be applied to develop equations representing patient physiology and pharmaceutical action. In fact, the human body is generally modeled using compartmental techniques (see Section II-A) from mathematical systems physiology, which can describe the behavior of healthy and perturbed organ/tissue interactions as well as the distribution/transportation of pharmaceuticals in the body (pharmacokinetics) and the action of these pharmaceuticals on certain organ systems (pharmacodynamics) in terms of nonlinear differential equations.

A. Compartmental Models
When working with physiological or biological systems, the dynamic interactions of system states/variables are often depicted in the form of compartmental models. This representation is slightly different from the standard block diagram models of systems theory but conveys similar information. Under this modeling paradigm, a system is comprised of one or more compartments, each of which has a (normally) fixed volume V i and contains a variable state quantity q i (t). The representation of this quantity q i (t) as a fraction of the compartment volume gives the concentration c i (t) = q i (t)/V i . When the change in these quantities or concentrations takes the form of ordinary differential equations, a compartmental modeling diagram can be drawn to depict the contributions of each state quantity to the dynamics (similar to state-variable representation). In the case of a linear time-invariant system, every compartment is associated with a linear first order differential equation described by the signals entering and leaving the respective compartment. Considering the general two-compartment model shown in Fig. 2, the change in q 1 (t) is given bẏ q 1 (t) = u 1 (t) − k 21 q 1 (t) + k 12 q 2 (t) − k 01 q 1 (t) (1) where u 1 (t) is the input to compartment 1; k 01 q 1 (t) is the leakage from this compartment to the environment; k 21 q 1 (t) is the exchange from compartment 1 to compartment 2; and k 12 q 2 (t) is the exchange from compartment 2 to compartment 1. Similarly, based on the arrows pointing toward and away from compartment 2, the change in q 2 (t) can be written aṡ q 2 (t) = u 2 (t) + k 21 q 1 (t) − k 12 q 2 (t) − k 02 q 2 (t) where u 2 (t) is the input to compartment 2; k 02 q 2 (t) is the leakage/loss from compartment 2 to the environment; and all other terms are the same as described previously. Together, these equations describe the exchange between compartments and the environment. In general, there may be n compartments, each of which may have a constant or changing volume V i and can be linked to any or all other compartments by linear or nonlinear interactions. For more detailed information on compartmental modeling, readers are referred to the extensive coverage in [14].

B. Control Methodologies
In this section, we briefly review the fundamentals of the 3 most common control methods currently used in physiological systems: (i) proportional-integral-derivative (PID) control, (ii) model predictive control (MPC), and (iii) fuzzy-logic (F-L) control. The application of these control schemes to specific physiological systems, including the advantages, variations, and recent developments to each strategy, are described in later sections. Of course, many other control methods are also being pursued in the literature and will be discussed as needed.
1) Proportional-Integral-Derivative (PID) Control: PID control is one of the most frequently used control approaches in industrial systems due to its simple mathematics, wellknown gain tuning methods, and broad application to both linear and nonlinear systems. The Laplace domain transfer function of the ideal PID controller can be given by where s is the complex-valued Laplace variable; E(s) and U (s) are, respectively, the Laplace domain input and output of the PID controller; k p is the proportional gain; T i is the integral time-constant; and T d is the derivative time-constant. In most cases, the coefficients k p , T i , and T d are freeparameters that take constant values and are designed to obtain desirable system properties. The values of these parameters may be obtained by either theory-based design strategies or numerical/experimental tuning methods.
2) Model Predictive Control (MPC): Model predictive control is a robust and optimal control technique used to handle systems with constraints while optimizing the controller action and predicting system outputs for robustness against noise and q (x t+k , u t+k ) subject to x t = x(t) (Measurement or estimation) x (t+k+1) = Ax (t+k) + Bu (t+k) (System model) x (t+k) ∈ χ (State constraints) u (t+k) ∈ u (Input constraints) U * t (x(t)) = {u * t , u * (t+1) , , u * (t+N −1) }(Optimization variables) where U * t (x(t)) is the optimal input sequence for the entire planning window N ; q is the cost function; and χ and u are, respectively, sets of state and input constraints. At any given time t, this controller finds the set of optimal control actions U * t based on the predicted dynamics over the entire window N and implements only the first control action u * t in this sequence.
3) Fuzzy-Logic (F-L) Control: Fuzzy systems are knowledge-based or rule-based systems consisting of fuzzy "if-then" rules that represent heuristic verbal conditions as continuous membership functions Fig. 3 [15]. This method can be used to model and control complex dynamics, nonlinear systems, and systems that are difficult to express mathematically. In general, fuzzy-logic systems can be classified as Mamdani and Sugeno systems [16]- [18]. Fuzzylogic systems have been successfully implemented in many industries.

C. Case Studies
In order to demonstrate and contextualize many important principles in PCLCs, we will proceed over Sections III and IV with detailed case studies of the two most active areas in physiological control: (i) automated anesthesia and (ii) automated insulin therapy for diabetes. For each system, a detailed background on therapeutic goals, physiology, and mathematical representations is given prior to discussions on current trends in the control methodologies and specific challenges encountered in each area.

III. DIABETES AND THE ARTIFICIAL PANCREAS
To understand the problem of blood glucose regulation, let us begin with a brief review of the essential physiology of a properly functioning glucoregulatory system and build towards a fundamental understanding of the diabetic disorders (pathophysiology) as well as their underlying causes (etiology) and associated health risks (complications and comorbidities). By way of this introduction, we hope to provide clarity for the coming discussions on system models and insights when broaching the subjects of treatment and closed-loop therapy.

A. Physiology of the Healthy Blood Glucose Regulatory System
The natural regulation of blood glucose in a healthy individual is the result of a variety of interacting physiological processes involving hormones, neurotransmitters, and our own consumption of food. Historically, the accepted metric for judging the quality of this regulation is the amount of glucose present in the blood stream, measured as a molar (mmol/L) or mass (mg/dL) concentration. This blood glucose concentration is commonly referred to as an individual's blood glucose level and is generally well-maintained between about 4-6 mmol/L (72-108 mg/dL) during healthy glucose homeostasis. However, even in healthy subjects, there are occasional excursions above this range (e.g., after recent meals) or slightly below it (e.g., during or after periods of high activity). The interactions that govern glucose concentrations are highly diverse and most, if not all, of these interactions are reciprocal -both the glucose concentration and the acting processes are mutually affected by the regulatory procedure. The systems involved in this regulation include the brain, liver, kidney, gastrointestinal tract, adipose and muscle tissues, and pancreas. Of these systems, the pancreas is arguably the most critical for maintaining proper blood glucose and will be our focus in the sequel.
The pancreas, located behind the stomach, is a glandular organ primarily responsible for secreting digestive enzymes through ducts to the small intestines and releasing pancreatic hormones into the blood stream. While both enzymes and hormones play important roles in proper metabolic homeostasis, it is the latter (i.e., hormones) that are principally responsible for regulating blood glucose levels [19].
These hormones, known as the pancreatic hormones, are produced at small clusters of endocrine cells (i.e., cells that secrete directly into the blood stream), known as islets, that are distributed sparsely throughout the pancreatic tissue (comprising <2%). These pancreatic islets (or islets of Langerhans) are constructed from a combination of 5 kinds of endocrine cells called α-cells, β-cells, γ-cells, δ-cells, and -cells. Each of these cell types secrete a specific set of hormones that are known to be involved in at least some aspects of normal glucoregulatory function. Of these cells, the α and β types makeup >80% of all pancreatic islet cells and are considered the 2 most essential endocrine cells for blood glucose regulation [19]. The first of these, α-cells, produce a single hormone called glucagon, which drives the production of endogenous glucose by catabolizing (breaking-down) stored glycogen in the liver (known as hepatic glycogenolysis) and to a lesser extent by promoting the generation of glucose from noncarbohydrate compounds in the liver and kidneys through processes termed, respectively, hepatic and renal gluconeogenesis. The latter type of cells, β-cells, produce 3 hormones called C-peptide, amylin, and insulin, all of which have important roles in glucose regulation and are believed to correspond with various symptoms of blood glucose pathophysiology. Of these 3 hormones, insulin is the most well-known, and is responsible for lowering blood glucose concentrations through promotion of glucose utilization by adipose and muscle tissues, as well as by prompting the creation of glycogen and fatty acid energy stores from glucose through glycogenesis and lipogenesis, respectively. Thus, in contrast to glucagon, which raises blood glucose levels, insulin provides the primary means to reduce this concentration [19].
As one might expect, based on their roles in the blood glucose regulatory function, glucagon is primarily secreted during periods of low blood glucose such as sleep or fasting, while insulin secretion is stimulated by elevated blood glucose concentrations and the incretion effect promoted by ingestion of glucose, fructose, amino acids, and long-chain free fatty acids [19]. Of course, insulin secretion is highest following meals (i.e., when blood glucose levels are at their highest). Together, glucagon and insulin provide the basis for many simple and intuitive bi-hormonal models of the pancreatic endocrine system. While many of the physio-chemical mechanisms for these secretions and uptakes are quite well known, it is important to understand that these hormones are released directly into blood stream and interact with many different organ systems. Thus, the specific mechanisms and outcomes of these interactions remain highly active research topics.
Of course, the 2 other β-cell hormones and those secreted from the γ, δ, and -cells are also intrinsic to proper physiology. Unfortunately, the functions of these hormones are not yet as well understood as those of insulin and glucagon and are often (perhaps necessarily) disregarded in the controloriented literature. However, there is reason to believe study of these pancreatic hormones, and their roles in both physiology and pathophysiology, will likely lead to better multi-hormonal models and treatment approaches in the coming years [19], [20]. For example, amylin -produced by β-cells along with insulin -is believed to perform complementary functions to the insulin-based removal of glucose, instead inhibiting the appearance of absorbed (exogenous) and glycogenolytic glucose by, respectively, reducing the flow of food from the stomach to the small intestines (gastric emptying) and blocking the release of postprandial glucagon (i.e., inhibiting unnecessary endogenous glucose secretion following meals) [20]. While an FDA-approved amylin analog (i.e., pramlintide) is available, the cost of treatment with such complimentary hormone analogs may be prohibitively expensive and perhaps even detrimental if not properly (e.g., physiologically or pseudo-physiologically) controlled.
1) Pathophysiology and Etiology: Type 1 Diabetes Mellitus (T1DM): The term type 1 diabetes mellitus describes blood glucose regulatory disorders resulting from the complete (or near complete) destruction of all pancreatic β-cells, leading to hyperglycemia and life-threatening complications. T1DM is generally caused by an auto-immune response, but does occur infrequently without any known or identified cause (idiopathic T1DM) [21], [22]. The characteristic β-cell destruction of T1DM leads to a dependence on external sources of insulin (i.e., insulin therapy) for survival; hence, this form (esp. the auto-immune form) was previously identified by the term insulin-dependent diabetes mellitus (IDDM) [21], [23]. Type 1 diabetes accounts for 5-10% of all cases [21].
Considering the preceding discussions on healthy glucoregulatory function, it is clear that destruction of β-cells in T1DM suppresses the production of the insulin, amylin, and C-peptide hormones. Without these hormones (particularly insulin), cells throughout the body cannot remove glucose from the blood stream for use in metabolic processes (e.g., storage as glycogen). As a result, the body turns toward the metabolism of fats, which leads to increased levels of free fatty acids and ultimately ketones in the blood stream [23]. Overproduction of the latter leads to life-threatening diabetic ketoacidosis. Furthermore, complications due to elevated blood glucose levels (hyperglycemia) appear across numerous organ systems and include microvascular diseases such as neuropathy, retinopathy, and nephropathy [23], [24]. In T1DM, the absence of interactions between pancreatic α-cells and β-cells removes an important pathway for natural glucagon regulation, leading to continued glucagon production during periods of elevated blood glucose, resulting in more severe hyperglycemia and even greater risk of complications [23]. Unfortunately, while many of the risks associated with T1DM can be reduced by tight regulation of blood glucose levels [23], the destruction of β-cells affects multiple hormones and many systems, some of which are not fully restored by external blood glucose regulation. For instance, increased risk of cardiovascular disease -the leading cause of reduced life-expectancy in type 1 diabetics -remains prevalent in type 1 diabetics with good blood glucose control [23]. Type 2 Diabetes Mellitus (T2DM): The term type 2 diabetes mellitus describes a family of blood glucose regulatory disorders that are marked by relative insulin deficiency, due to a combination of poor insulin production and poor insulin utilization, leading to episodes of hyperglycemia [21], [22]. Previously identified as non-insulin-dependent diabetes mellitus (NIDDM), current opinion is trending towards a more graduated classification system that may improve differentiation of T2DM by severity and pathophysiology of impaired insulinsecretion (production) and insulin-sensitivity (utilization) [21], [25], [26]. Type 2 diabetes accounts for about 85% of all cases [21], [23], [26].
Unlike T1DM, T2DM is not an autoimmune disorder and has neither a single underlying cause (etiology) nor a unique physical origin. Instead, the term T2DM applies to various non-specific disorders (i.e., not specifically meeting the criteria for definition as any distinct disorder such as monogenic, gestational, or type 1 diabetes mellitus) that result in chronic excess blood glucose [21], [22]. While in some cases, T2DM is marked by a reduced pancreatic insulin secretion that results in a relative insulin deficiency, in others, insulin production is unaffected, but cells exhibit an inability to utilize available insulin (i.e., insulin resistance). However, most of the type 2 diabetic population exhibit both insulin deficiency and insulin resistance [21], [22], [25], [26]. Thus, it is becoming more common to characterize each instance of T2DM along gradients of insulin-sensitivity and insulin secretion (among other variables including coexisting conditions), to better identify root-causes and optimal treatment regimens [25], [26]. While T2DM is not normally life-threatening by itself, it is associated with reduced life-expectancy due to macrovascular complications, as well as numerous microvascular complications that are believed to be linked with episodes of hyperglycemia. Like T2DM itself, the accompanying complications are diverse and, some research suggests, potentially well-correlated with pathophysiologically distinct forms of the disorder [25]- [27].
Gestational Diabetes Mellitus (GDM): Glucose intolerance that develops in women during pregnancy, indicated by the emergence of persistent hyperglycemia, is known as gestational diabetes mellitus [21], [22], [28]. Like T2DM, GDM is comprised of a highly heterogeneous set of disorders, which may present predominantly insulin resistant or insulin deficient characteristics [28]. While standards for diagnosing GDM vary, the critical feature of GDM is that it is developed during pregnancy and was not present prior [21]. The pathophysiology of GDM is not yet well-understood [28], but ongoing research may benefit from more accurate and detailed sensing technologies. GDM is a disorder of international interest and can result in complications for both mother and child and can progress into a form of T2DM after pregnancy [21].
Other Specific Types of Diabetes: In addition to T1DM, T2DM, and GDM, there are numerous, less common diabetic disorders (together comprising ¡5% of cases) with more specific association with distinct etiological factors such as genetics, drug or chemical exposure, and pancreatic diseases [21], [22]. Differentiation of these specific forms from T1DM and T2DM is critical, as the treatment requirements for many of these disorders differ from general guidelines and mischaracterization can lead to poor or detrimental outcomes and unnecessary expenses [21], [25].

B. Traditional Monitoring and Treatment
Considering the variety of disorders that may exist, or coexist, in diabetic individuals, it is not surprising that there are a plethora of diabetic diagnostic tools and interventions, some of which have been shown to be most effective in monitoring and treating specific disorders, while simultaneously potentially detrimental for treating others. Likewise, treatment guidelines are often adjusted for improved effectiveness in specific demographics often based on age, gender, and overall health, with the final recommendation for individuals tailored by their physician [29], [30]. 1) Laboratory Screening: For the past half century, blood glucose regulatory function and concentration targets for diabetics and non-diabetics have primarily been specified according to 2 standard measurements: (i) static blood glucose concentrations and (ii) percent glycated hemoglobin (HbA1c, or less formally A1C). While a static measurement of blood glucose provides insight into the current glycemic state (i.e., the current mmol/L concentration of glucose in the blood stream), it does not provide any insight into recent concentrations or an indication of whether this value is increasing or decreasing [31]. Conversely, A1C provides an approximate indication of the average blood glucose concentration over a period of about the preceding 2-3 months [29], [31]. While A1C is less sensitive to daily blood glucose variations and a primary predictor of diabetes related complications, acceptable A1C values have been shown to vary significantly across populations and with various environmental factors, and further do not provide any specific information regarding occurrences of hypoglycemia, hyperglycemia, or their severity [21], [29].
In clinical settings, diabetic screening is often performed using plasma blood glucose concentrations that are measured by accredited laboratories using standardized tests [32] to determine levels under the moderately controlled circumstances either following a period of fasting to obtain the so-called fasting plasma glucose (FPG) level or 1-2 hours into an oral glucose tolerance test (OGTT) administered after a period of fasting. If the FPG or OGTT results fall below certain thresholds (see characteristic thresholds from the ADA [21] in Table II), these tests results, respectively, indicate impaired fasting glucose (IFG) or impaired glucose tolerance (IGT) associated with metabolic disfunction. The accuracy of these tests depends heavily on patient adherence to the fasting guidelines and clinical sample handling. Thus, for clinical A1C (also performed in laboratories [32]), FPG, and OGTT testing, confirmation of an initial diagnosis of diabetes often requires that 2 of these values fall outside the acceptable thresholds [21].
2) Self-Monitoring of Blood Glucose (i.e., "Finger-Stick" Monitoring): Once diagnosed, most diabetics (especially those with T1DM or T2DM requiring insulin therapy) will require portable diagnostic equipment so that blood glucose concentrations can be monitored according to the severity of their disorders or in the event of symptoms that indicate excursions from normal glycemic control. For many years, the predominant method for this daily testing was self-monitoring of blood glucose (SMBG) by so-called "finger-stick" devices [31], [33], which require drawing a small sample of blood during every test. Initially accepted for their marked improvement over urine testing equipment, use of these finger-stick devices causes pain, and measurements can be uncomfortable or unsafe (due to blood spill) to perform in public [31]. While significantly less accurate than accredited laboratory testing [32], these portable diagnostic tools have enabled diabetics to maintain tighter glycemic control by regular assessment of their blood glucose levels, especially prior to meals, sleep, and insulin dosing. However, like static laboratory measurements,
3) Continuous and Flash Glucose Monitors: Starting in the late 1990s, diagnostic tools for continuous home monitoring of blood glucose began to emerge as a potential remedy to the shortcomings of traditional SMBG [31]. Originally designed to record, but not display, blood glucose measurements over several days -to be analyzed retrospectively by a physician [32]- [34]-more recent real-time CGM (rtCGM) devices that log data and provide immediate access to measurements and analytical trends are now available on a broad commercial scale (Table III) [31]. Unlike SMBG, which requires taking blood samples at each measurement time, most rtCGM devices are minimally invasive and measure glucose concentrations subcutaneously through the interstitial fluid rather than whole blood or plasma [32], [34]. These sensors can be worn continuously for up to a week, and provide updated measurements every 5 minutes, requiring finger-stick measurements for calibration (up to twice per day) [32]. While only one such device is FDA approved for non-adjunctive use in insulin therapy (i.e., for use in determining insulin dosage without the need to confirm blood glucose levels by traditional SMBG), this is a recent development and more devices may be expected in the future [29].
This consistent availability of current data has led to several important features that are impossible with traditional SMBG. First, many rtCGM systems allow users to set alarms that indicate or even predict the onset of hypoglycemia, allowing diabetics to increase their carbohydrate/glucose intake and avoid hypoglycemia, which is especially useful for those suffering from hypoglycemia unawareness [31]. Second, many rtCGM systems allow users to automatically share data/trends with caregivers and physicians, having important implications for young children and their parents [31], as well as elderly and disabled individuals who are living alone. Finally, rtCGM systems not only provide improved predictive measures for users through real-time measurements, measurement histories, and predicted glycemic trajectories, many of these systems can be interfaced with portable insulin infusion systems to automatically suspend insulin dosing when near or approaching hypoglycemia [31], [34]. Combinations of rtCGM and insulin infusion systems, known as sensor augmented pump (SAP) therapy, are further enabling improved overnight glycemic control, and represent a step toward wide-spread acceptance of closed-loop insulin therapy systems.
More recently, a new class of glucose monitor known as the "flash" or "intermittently sensed" glucose monitor (isCGM) have become available that provide immediate, on demand glucose measurements (lagging true blood glucose concentrations by about 5 minutes) in a minimally invasive package worn continuously for 14 days without calibration (the sensing element is replaced after 14 days without requiring any SMBG calibrations) [33], [34]. While this device may not be suitable for those with hypoglycemic unawareness (developed by repeated hypoglycemic excursions) as it does not currently provide an autonomous alert, various studies have demonstrated improved time-in-range and greatly reduced time in hypoglycemia in both type 1 and type 2 hypoglycemic aware populations previously using finger-stick SMBG. Small observational studies indicate that these results may follow from the rapid and pain-free nature of blood glucose selfmonitoring with these devices, which allows users to follow even stringent (¿10 times per day) testing guidelines without experiencing discomfort [33]. This research has also reported behavioral changes in patients, including better adherence to monitoring timelines, which may produce additional societal benefits, as unchecked low blood sugar is associated with otherwise avoidable injuries and a increased risk of car accidents [42]. Due to time lag and measurement inaccuracy -possibly exceeding 15% mean absolute relative difference (MARD) from matched reference measurements -users of these devices must still perform traditional SMBG tests when hypoglycemia is expected, measurements vary rapidly or do not corroborate symptoms, and when required by law [33]. While these flash monitors are available and gaining significant interest, there remain significant concerns in areas including sensor placement, failure modes, user insulin stacking due to time lags, and measurement variations over the duration of the 14-day sensor life-span.
Studies on CGM devices have consistently indicated that, compared to control groups using SMBG, insulin treated diabetics using CGM systems show improved HbA1c as well as significantly decreased frequency and severity of hypoglycemia and hyperglycemia, with rtCGM demonstrating superior outcomes to isCGM [29], [31], [33], [34]. Despite their benefits, modern CGM sensors suffer from delays due to glucose transport from plasma to the interstitial fluid, may be subject to sensor interference from mechanical [43] and pharmaceutical (e.g., acetaminophen [44]) sources, and are known to demonstrate reduced accuracy during the first day of use FreeStyle Libre 11.4 14 days none *Sensing elements may limit the duration CGM systems and affect their accuracy but do not alone give the mean absolute relative difference (MARD) or calibration schedule. and at low glucose concentrations (i.e., near hypoglycemia) [31], [43]. With further research, these issues may be resolved in the near future.

4) Non-Invasive, Wearable, and Implantable Monitoring:
During the past decade, there has been increased academic and commercial interest in the development of non-invasive and continuous monitoring of blood glucose levels through wearable technologies. Unlike traditional monitors, many of these sensors do not measure blood glucose directly, but attempt to leverage alternative fluids such as urine, sweat, saliva, breath, ocular fluid, and interstitial fluid [45]. While some of these devices are promising, with many designed to be discrete and comfortable, measurements taken from these alternative physiological fluids tend to be less reliable than traditional methods, and any such technology will surely require rigorous testing and clinical studies before any statements can be made regarding safety and accuracy [46].

5) Non-Insulin Therapy:
While not generally suitable for T1DM, non-insulin oral antihyperglycemic agents are common in treating T2DM. In fact, the first-line pharmaceutical therapy for T2DM is a non-insulin antihyperglycemic called metformin, which reduces hepatic glucose production (i.e., in the liver) and is often highly effective at reducing HbA1c and associated cardiovascular risks, without raising significant concerns of hypoglycemia [30]. According to ADA guidelines, metformin is frequently prescribed as part of a dual or triple pharmaceutical regimen (combination therapy) when metformin alone does not provide sufficient reductions in HbA1c -used in this case as a primary indicator of glycemic control. These additional treatments may be oral agents such as sulfonylureas (increases insulin secretion), DPP-4 inhibitors (increases glucose dependent insulin secretion and decreases glucose dependent glucagon secretion), and Thiazolidinediones (increases insulin sensitivity), but may also include subcutaneous injections of insulin or GLP-1 receptor agonists, which act on multiple levels [30].
Considering the underlying lifestyle factors (e.g., sedentary behavior, obesity, and diets that are high in fats and sugars [47]) that are known to promote the development of T2DM, prevention and treatment through lifestyle modification is a significant area of study [47]- [49]. Intensive lifestyle modifications have been shown to improve HbA1c, and are believed to be associated with reduced microcardiovascular and macrocardiovascular complications as well as lower rates of diabetes related mortality in T2DM. Similarly, there is evidence to support the benefits of nutritional balance in T1DM; however, the direct evidence for reduction of diabetes related ailments (beyond general health and fitness) associated with physical activity and obesity are much better supported for T2DM. In fact, due to the increased glycemic variability of individuals with T1DM, the ADA recommends physicians take care in prescribing specific exercises and durations for individuals when using physical activity as a means of glycemic control [49].
While these non-insulin therapies are often effective at slowing the progression of T2DM, in some cases leading to remission of the disorder, most individuals with T2DM will often require insulin therapy at later stages of treatment, even with careful adherence to therapy guidelines and healthy lifestyle choices [30].
6) Multiple Daily Insulin Injections: Recalling that severe insulin deficiency is the hallmark of T1DM related β-cell destruction, it is not surprising that primary treatment for T1DM is insulin therapy [30]. The most common method of insulin therapy consists of the regular periodic injection of basal (baseline) insulin multiple times a day -known as multiple daily insulin injections (MDI)-supported by additional insulin doses (boluses) and oral glucose or glucagon as needed to maintain normoglycemic conditions (e.g., at meal times). While several different practical guidelines exist for calculating the required basal and bolus insulin doses, dosage is always described in terms of units of insulin or units/day (based on weight as units/kg/day).
Recent pharmaceutical advances have produced a variety of injectable insulins that may be categorized as either synthetic human insulin or modified insulin analogs. Referring to Table  IV [30], [50], variations in these insulin formulations have led to markedly different properties, and so it is common to classify insulins according to their concentration in units/mL (e.g., an insulin specified as U-# contains # units/mL), the time it takes for the insulin to begin reducing blood glucose levels (onset time), the time it takes for the insulin action to reach its peak effectiveness (peak time), and the length of time that the formulation will remain effective after injection (duration) [50]. When prescribing or taking insulin, each of these factors comes into play and (in an MDI scheme) make some forms better suited as basal insulin and others as mealtime boluses [30]. Thus, even within insulin therapy, there is now an array of options that allow for personalization and situational evaluation of treatments.
Note, the use of insulin is not limited to T1DM, and there are, of course, many factors affecting the selection, efficacy, and safety of different insulin formulations and treatment regimens. In addition to single insulin formulations, some injections are developed as a mix of rapid acting (i.e., quick onset and peak times with short duration) and long acting (i.e., delayed onset time, low or no peak, and long duration) insulins to provide both basal and bolus action from a single injection, thereby reducing the number of injections required per day [30], [50].
In some cases, an analog of the pancreatic β-cell hormone amylin may be used to augment insulin therapy [30]. However, addition of such agents may increase treatment costs significantly. Further, in certain cases, it may be beneficial to perform pancreas or islet transplantation, in place of insulin therapy. 7) Continuous Subcutaneous Insulin Infusion: Initially developed over 40 years ago [51], continuous subcutaneous insulin infusion (CSII) systems are portable pump therapy devices that are generally constructed as a combination of an onboard insulin reservoir, an electromechanical infusion pump, and an infusion apparatus (tubing and cannula) [51], [52]. According to various studies, these systems can be operated using synthetic human insulin or rapid-acting insulin analogs (RAIA), with RAIA providing superior performance to synthetic human insulin [52]. In most cases, CSII uses the same basal dosage as MDI, with the basal insulin dosage applied more consistently over the day in CSII [52].
While pragmatic evidence supporting substantial improvements to patient outcomes achieved by switch from MDI to CSII is minimal or frequently contradicted [30], [52], there is evidence that switching to CSII is beneficial for diabetics with poor glycemic control under MDI [52], and CSII is beginning to see adoption as a recommended first-line therapy for some type 1 populations (e.g., adolescents) [30], [52]. However, when CSII is used in conjunction with rtCGM for sensor augmented pump therapy, the benefits are more pronounced.
Connected systems of monitoring and treatment devices, like SAP, provide therapeutic opportunities that cannot be obtained using MDI, such as overnight glycemic regulation. As an extension of this concept, the ability to continuously monitor and control insulin concentrations (whether by CSII or other novel future technology) is paramount to the development of closed-loop insulin therapies. Like SAP, closed-loop therapy systems are designed to utilize the features of continuous infusion that cannot be matched by MDI, with or without CGM, to achieve tighter glycemic control than is possible by any current treatment.
As with all therapy devices, CSII must be appropriately used and maintained, as device performance depends on proper operation (e.g., timely replacement of consumables) to avoid failure modes such as clogged or impeded infusion pathways, which can lead to insulin deficiency and hyperglycemia [52].

C. Modeling for Analysis and Control of Diabetes
Building on nearly 2 centuries of research and modeling studies focused on the pancreatic endocrine system, the recent worldwide interest in diabetes has instigated a surge in the development of glucoregulatory models and simulation platforms, primarily aimed at T1DM [53]- [57] and, to a lesser extent, T2DM [54], [56], [57]. In fact, according to several reviews [53], [57], there are well over 100 different mathematical models describing, reproducing, and/or predicting various aspects of the blood glucose regulatory dynamics from cellular mechanisms and neurological signaling to longterm outcomes, disease progression, and risks of complications [57]. The mathematical descriptions used in these dynamic models are diverse and include ordinary, partial, and stochastic differential equations, which often include time-delays, and are frequently treated with a degree of modularity -as new models may be established as composites of subsystems from multiple predecessors [57].
Among these models, some, such as the mechanistic models that describe the Ca 2+ and K + ion channel mediated release of insulin from pancreatic β-cells, have solid footing in the first principles of electrochemistry and physics. However, the inclusion off such models tends to add unwarranted complexity as the pathophysiological and etiological relevance of these states are not well-understood, while producing negligible variations in the observable input-output behaviors that are presently more relevant to healthcare outcomes. Thus, in the development and analysis of systems for the closed-loop treatment of diabetes, models are generally categorized according to their level of detail and may be considered as either (i) reduced complexity control-oriented models -for controller synthesis methods [53], or (ii) high-fidelity models for analysis and validation [55], [56]. In either case, these models attempt to replicate the glucoregulatory dynamics of diabetic patients. However, while the former is concerned with representing accurate input-output relationships, in the latter category it is also important that models are structured such that variables and parameters maintain physiological significance and produce physiologically viable results at internal states. Thus, models of this second kind, can provide additional insights into substance and organ interactions, and have become known as virtual patient models -as they are generally used as simulated analogs of real patients. While early models of virtual patients utilized nominal or average parameter values, obtained from population studies, their more recent use in developing closed-loop therapy systems has led to a need for individualized model behaviors that closely match the severity and mode of dysfunction in specific patients. Accordingly, while some parameters may still be estimated by nominal values from population data [58], others have been identified from measurements on specific patients. For several models, this method has been used to generate arrays of virtual patients -called cohorts -that can be used to represent the interpatient-variability that may exist within a particular pathophysiology. When used for internal model type control schemes, the ability or inability to identify these individualized parameters from readily available data frequently dictates whether a simulation model is useful for real-time control. The availability of such data may also dictate the environments in which it is possible to implement certain control schemes. This is of particular concern when adaptation/tuning or online identification is required, as some diagnostic tools may only be accessible, or practical, in a clinical setting.
While many physiological models have been studied, several, such as the Bergman minimal model [59], [60], UVa/Padova simulator [61], and Cambridge model [62] represent the overwhelming majority of simulation studies using virtual patients.

1) Minimal Glucoregulatory Models:
While any glucoregulatory or subsystem model that depicts the essential dynamics while using the fewest possible states may be considered minimal [56], [63], the Bergman minimal model (BMM) is by-far the most well-known and frequently adapted. Despite simplifications and neglected dynamics, the parameters and variables in minimal models must both convey clear physiological significance and be readily identified from measurable clinical data [63]. Notice, this requisite simplicity and identifiability are often shared with the kinds of models that are sought in controller synthesis. Hence, in addition to its primary analytical purpose, the BMM has also been adapted for use in the implementation of closed-loop blood glucose control strategies.
To understand the importance of this model, it is interesting to consider its origin. Initially developed as a means to quantify the insulin sensitivity of individual patients [59], the BMM was the result of early experiments to identify the structure of blood glucose-insulin interactions by obtaining frequent measurements of both plasma glucose and plasma insulin concentrations through the duration of a 1 hour intravenous glucose tolerance test (IVGTT) applied following a single bolus injection of glucose. Blood samples were taken first before injection, to obtain basal levels, and following injection at specific intervals (i.e., rapidly at first but reducing in frequency over the hour). The results of these blood tests were fitted by nonlinear least squares to various pre-established nonlinear ODE model candidates, which were evaluated according to the number and physiological meaning of model parameters, as well as the accuracy of the parameter estimates and each model's overall goodness-of-fit. Two of 7 possible models (one linear, the other nonlinear) passed this initial testing, both of which included insulin-dependent glucose utilization -3 of the models were insulin-independent and were rejected for their poor prediction of the glucose kinetics. Ultimately, the nonlinear model was found to reproduce known physiological responses with higher fidelity. Thus, the BMM was selected as the 3-compartment nonlinear system of ODEs with 5 parameters shown schematically in Fig. 4 [59]. Notice, this model has 1 glucose compartment and 2 insulin compartments (one plasma compartment and one remote compartment).
Since the time of this original model, many adaptations have been made to account for additional dynamics, such as the connection of a second glucose compartment, the addition of glucagon feedback, and the replacement of IVGTT by oral glucose tolerance testing (OGTT) to obtain a more physiological measure of insulin sensitivity [64]. However, even with these augmentations, the BMM must be further adjusted to be useful for closed-loop control. For example, one prevalent control-oriented model intended for T1DM control that utilizes the BMM is detailed in ref. [65]. This model augments the original BMM with insulin-dependent glucose dynamics, renal glucose clearance, meal absorption, external insulin infusion and subcutaneous transport, circadian variations, and an adjusted hepatic balance.
While many of the equations in this model are linear firstorder ODEs (e.g., all of the insulin and the meal absorption dynamics), there are several nonlinearities including circa- dian variations in insulin sensitivity in the interstitial glucose compartment and hard nonlinearities in the endogenous blood glucose production dynamics. Further, any effects caused by glucagon or physical activity have been excluded. Note, the original model also includes intravenous routes for insulin and glucose, which have been omitted above. Due to its questionable accuracy and remaining complexity, the BMM is not always the first choice for model-based closed-loop control. However, it is frequently adapted for this purpose, and has provided an important clinical tool for establishing an individual's level of β-cell function through the insulin sensitivity index [60], [64], [66]. Like the BMM, other models used in controller design for closed-loop insulin therapy generally trade physiological complexity for model structure, identifiable parameters, and appropriate input-output behaviors -typically by linearization. This trade-off with traditional physiological models is -at least partially -responsible for the increasing role of data-driven models for personalized control of diabetes [67].
2) UVa/Padova Models and Simulations: Since the time of the BMM, several detailed models have been developed, or modified, to analyze the glucoregulatory behavior of diabetic patients. However, to date, there is only one family of models that has been approved by the United States Food and Drug Administration (FDA) as a replacement for pre-clinical trials in the validation of a closed-loop control design. These are the socalled UVa/Padova models [68] developed primarily between collaborators at the University of Virginia in the USA and the University of Padova in Italy.
In contrast to the simplicity of minimal models, the UVa/Padova model is a complex and hierarchically organized model that integrates high and low-level details into submodels of organ systems and metabolic pathways, to obtain a comprehensive, physiological simulation of human patients [69]. This model, sometimes referred to as a maximal model due to its maximal level of detail [56], contains over 30 parameters [58] that must be identified to match a specific patient and is generally too complex to produce desirable controllers without simplification. Conversely, this complex model can accurately reproduce many aspects of the glucoregulatory dynamics of healthy and diabetic individuals, which has led to the development of individualized patient simulation models known as in silico subjects. Using high quality clinical data, researchers have developed a large collection of patient models representing numerous individuals from each of various demographics, enabling studies on the entire population as well as particular cohorts of interest [61], [68], [69]. Such detailed high-fidelity modeling is the foundation for an emerging symbiotic relationship, wherein simulations are not only useful for analyzing the effects of treatments on known (patho)physiology, but also for testing hypotheses regarding the (patho)physiology itself and models thereof [56].
Presently, a distributed version of the UVa/Padova type 1 diabetes metabolic simulator (T1DMS) for use in MAT-LAB/Simulink can be obtained by academic and commercial entities through The Epsilon Group [70]. In addition to its early capabilities, the current version of the UVa/Padova simulator includes dynamic models of the bi-hormonal (i.e., insulin and glucagon) glucoregulatory dynamics, an improved patient cohort, and updated depiction of regulatory feedback during hypoglycemia [61], [70]. As of 2018, researchers from UVa/Padova have announced further updates to their simulation platform that introduce, among various other extensions and improvements, a set of time-varying patient models to account for diurnal variations in patient dynamics associated with overnight rise in blood glucose (i.e., the dawn phenomena) and patterns of varying insulin sensitivity at morning, afternoon, and evening meals [71]. While the previous updates [61] are FDA approved, approval for the more recent timevarying simulator [71] is still pending at the time of writing this article.
The development of the UVa/Padova model is well documented, and interested readers are referred to the relevant literature for extensive details on the underlying model, in silico population, recent updates, and application to the validation of control algorithms. Note, despite its complexity, several closed-loop control algorithms have been designed using reduced order approximations of these models [55], [72], [73].
3) Cambridge (Hovorka) Models: Similar to the UVa/Padova model, the Cambridge model [62], [74], [75] has frequently been used for in silico validation studies of closedloop insulin systems and has its own MATLAB/Simulink based simulation environment and cohort of 18 clinically validated virtual subjects [55], [62]. However, unlike the UVa/Padova simulator, results from this software are not FDA approved substitutes for pre-clinical trials. While this may lead to several additional steps prior to clinical studies, the Cambridge model has significantly fewer mathematical states and parameters than the UVa/Padova model [58], while maintaining physiological significance and modeling diurnal variations, which has allowed some researchers to modify and adapt it more readily for their specific needs [76].
Despite its complexity, the Cambridge model is frequently used in internal model or model predictive control schemes [53], [74] and has been the basis for several clinically tested investigatory closed-loop devices [75].
4) Alternative and Data-Driven Models: In addition to the more analytical and physiologically guided virtual patient models, numerous simplified control-oriented models have been proposed that attempt to leverage various modeling paradigms. Of particular interest, the use of linear parameter varying (LPV) type models [77] has been shown to provide a simplified means to accurately reproduce the complex timevarying and nonlinear dynamics of maximal diabetes models. Using appropriate techniques, reduction of maximal models to LPV models may provide a superior model-based and controloriented approach to blood glucose controller synthesis.
Noting that even the best physiological blood-glucose models are, at present, phenomenological and difficult to individualize for optimal real-time control, many researchers have instead adopted data-driven modeling paradigms such as autoregressive exogenous (ARX) and autoregressive moving average exogenous (ARMAX) models. Provided with a predefined model structure, these techniques utilize online system identification to obtain and, potentially, update models to achieve a best approximation of the input-output behaviors of the glucose-insulin system based on a given performance index [67]. Like LPV models, it is unlikely that data-driven models will maintain any detailed physiological information about a given patient, but instead provide a useful foundation for implementing established controller synthesis approaches. 5) Models of Therapeutic Agents and CGM Sensors: Following previous discussions on sensors and treatment approaches, it is natural that these components will contribute dynamics to the overall system. Models for CGM sensor accuracy [78] and insulin formulations and kinetics [79], [80] have been developed by various investigators. These models enable simulations that provide a complete picture of blood glucose control under specific therapy strategies and allow the investigation of closed-loop systems while taking sensor and actuator performance limitations into account.

D. Automated Therapy for Diabetes
Attempts to develop fully closed-loop insulin delivery systems date back more than 50 years [81], and many companies and academic researchers have contributed significantly in these developments [82]. While early work focused on sugar and insulin infusion by intravenous methods, the success of these systems along with the development of both CSII pumps and CGM sensors has shifted the modern focus toward minimally invasive therapies applied via subcutaneous routes [83]. Within the last 5 years, there has been -and continues to be -rapid progress toward the commercialization of fully automated insulin therapy systems [84], [85], frequently referred to as artificial β-cell, artificial pancreas (AP), or automated insulin delivery (AID) systems. The primary goal of these automated therapy devices is to alleviate the burden and risks associated with multiple daily injections from T1DM patients and caregivers by reducing diabetes treatment to a combination of proper lifestyle modifications (perhaps even less stringent ones once on AP therapy) and device maintenance (to ensure proper AP function and patient safety). Although most current work is focused on T1DM, there are parallel efforts to investigate the use of these systems for specific cases such as insulin dependent T2DM [86], [87], T1DM during pregnancy [88], [89] (but not yet GDM as far as we are aware), and blood glucose regulation for patients in intensive care units (ICUs) [90].
1) Challenges to Automatic Control: While the virtual patient models discussed previously can reproduce many aspects of normal and diabetic blood glucose regulation, there remain several challenges that prevent these models from fully predicting the dynamics of real patients in real-time and limit the accuracy of designs for real-world control performance. The most notable among these challenges are the blood glucose disturbances caused by meals and exercise and the intrapersonal variations -particularly in insulin sensitivity and glucose production -that may occur within a single patient over their course of treatment.
While many meal absorption and gastric emptying models have been proposed to describe how meal contents -generally carbohydrates -act as sources of exogenous glucose [91], the timing, duration, and nutritional content of meals and snacks are generally unknown for patients in an uncontrolled, or free living, environment. As a substitute for such detailed information, many current algorithms resort to manual meal announcement strategies, through which patients or caregivers can provide nutrition and schedule information to the controller regarding an upcoming meal. However, even with meal announcement and approximate nutritional content, there may still be unpredicted digestive variations. Thus, some schemes -with or without meal announcement -implement meal detection algorithms that identify when a meal has occurred and may also estimate carbohydrate content [91]- [95]. At the current time, very few controllers can provide tight glycemic control when subjected to unannounced meals as it generally takes around 70 min after administration for peak insulin action to occur [95]. This delay in insulin action can provide a window for hyperglycemia to occur following even a well identified meal and may result in either hyperglycemia or hypoglycemia if meal properties are estimated incorrectly.
Like meals, periods of physical activity are known to disturb the nominal behavior of the blood glucose regulatory system. However, while meals primarily act by regulating exogenous glucose, physical activity can simultaneously affect numerous aspects of a closed-loop therapy system including physiological requirements [96]- [98], internal bio-mechanical pressure differentials [98], and CGM accuracy [98]. In fact, it is well known that different types, intensities, and durations of physical activity may provoke contrasting physiological responses (e.g., light aerobic exercise can cause hypoglycemia and brief stints of intense combined aerobic-anaerobic exercise can cause prolonged hyperglycemia [85], [96]). Thus, it is not only important to detect physical activity, but also to obtain enough data to classify -and perhaps continuously reclassify, track, or predict -the kinds of activity or exercises a patient performs. Further complicating this problem, certain kinds of physical activity can increase the rate of insulin absorption from the subcutaneous compartment to the blood stream [96], [98] and the corresponding increase in insulin action can cause hypoglycemia without any infusion of additional insulin. As noted previously, insulin does not act immediately and will not reach peak effectiveness for about an hour; thus, even if insulin has been regulated correctly for the current level of physical activity, variations in physical activity can change the internal utilization of the existing insulin dosage and affect hypoglycemic excursions that cannot be corrected by a single-hormone insulin therapy system [96]. To counteract these effects, many researchers are considering more advanced control systems that may utilize dual-hormone (i.e., glucagon and insulin) infusion sets [98], [99] or multiple sensors and algorithms to detect and characterize physical activity [92], [96], [100]. Despite these promising developments, there are currently no accurate models for simulations of exercise in diabetics [92], [97], and physical activity remains a difficult challenge for fully automated free-living diabetes therapy.
Additionally, the effects of many other environmental factors and lifestyle behaviors on localized and body-wide dynamics including glucose production, insulin secretion, and insulin sensitivity are still poorly understood. For example, recent studies have suggested that insulin sensitivity decreases significantly in both healthy [101] and diabetic individuals [102], [103] with reduced duration and quality of sleep. Further, the effects of circadian phasing, misalignment, and behaviors (i.e., timing of meals as hours after waking up) were recently shown to produce 15-20% changes in insulin sensitivity and β-cell function in healthy adults [104]. To the authors' knowledge, similar results have yet to be reported in diabetics with T1DM or T2DM. Combined with other influential factors (e.g., sensor and physiological interference from outside substances, user error in attachment and maintenance, and electromechanical faults [105]), any model or controller that has been individualized for a specific patient will almost certainly be subject to significant time-varying uncertainty, depending on patient behaviors and operating environment.
While these challenges apply to most diabetics, the extension of AP systems to T2DM may benefit from updated models that include compartments and inputs reflecting the action of noninsulin medications such as metformin, sulfonylureas, and GLP-1 agonists or additional hormones such as amylin (e.g., by the amylin analog pramlintide), which could also benefit T1DM patients and researchers [106]. Considering the apparent importance of glucagon for avoidance of hypoglycemia during exercise and the potential benefits of amylin for mealtime blood glucose regulation, it is not surprising that there has been recent interest in the development of closed-loop insulin-glucagon-amylin therapy [107], [108]. However, to the authors' knowledge, such complex hormone and combination therapy models are not yet common in closed-loop diabetes therapy devices but may provide significant avenues for system augmentation and a more physiological replication of β-cell function as the relevant technologies mature.
2) Proportional-Integral-Derivative Control Methods: Proportional-Integral-Derivative (PID) control is one of the most versatile and commonly applied methods of control for both linear and nonlinear systems and has been likened to the physiological behavior of the pancreatic β-cells [109], [110]. Numerous variations on PID control have been studied in simulations of T1DM [111], [112] and clinical trials. Most of the recent clinical studies using PID for closed-loop insulin delivery are related to the development of the PID-based external physiological insulin delivery (ePID) strategy [1], [110] including the introduction [1], [113] and refinement [114] of insulin-feedback (IFB) to a form near the one currently implemented in the commercially available Medtronic 670G hybrid closed-loop (HCL) therapy system [115]. In addition to the PID-based closed-loop control of basal insulin, these systems provide feedforward control action by meal-time boluses in response to manually entered announced carbohydrate loads [115]. Systems that implement this kind of manual, feedforward control are considered hybrid rather than fully closed-loop.
Another recent algorithm, known as the safety auxiliary feedback element (SAFE) has been developed that implements a two-step approach to augment therapy controllers with safety constraints in order to decouple the problems of achieving nominal control performance and abiding system limitations [116]. This auxiliary algorithm runs in a rapidly sampled (i.e., much faster than the 1 sample/min in the current core control algorithms) outer-loop that uses sliding mode reference conditioning to shape the desired insulin infusion profile to avoid violating safety constraints without affecting the nominal performance of the main control scheme. The SAFE algorithm can be used with a variety of controllers and constraints of known, constant relative degree, and has been used to enhance the meal-time performance of standard PID [116] and modified ePID algorithms [117].
In addition to these single-hormone systems, PID has also been implemented in several dual-hormone (i.e., glucagon and insulin) systems where it has been used as the control algorithm for glucagon micro-bolusing (in conjunction with a separate insulin control algorithm) as well as the control of both insulin and glucagon [118].
3) Model Predictive Control Methods: For many, model predictive control (MPC) has become the standard formulation for insulin delivery algorithms because of its intuitive treatment of constraints and ease of modification. Like the many implementations of PID variants, numerous MPC algorithms have been reported that attempt to leverage new or existing MPC techniques to improve the compatibility of these algorithms with the requirements and constraints of diabetes treatment [82]. Among the more recent developments in artificial pancreas systems are the extended [119], multiple model [82], [94], [120], run-to-run tuning [82], [121], zone [122], [123] and periodic-zone [124], event-triggered [125], and velocity-weighted & velocity-penalized [126] MPC algorithms. Each method of MPC attempts to overcome one or more of the challenges discussed previously including practical/safety restrictions [127], interpatient variability, intrapatient variability, and unannounced disturbances such as meals or exercise. Some of these methods have already become quite successful and provide important guides for researchers and engineers seeking to refine diabetes control strategies.
Safety Considerations: Among the first challenges overcome by AP algorithms were the restrictions imposed by limitations on insulin infusion. In principle there are limitations on both (i) the rate of insulin infusion, for instance due to saturations in pump displacement and the inability to remove insulin from the bloodstream once injected, and (ii) the amount of insulin in a patient's bloodstream at any given time. Notice, however, that while these problems are analogous to issues present in other industries, and general tools like anti-windup are readily available, the management of insulin is safety critical and MPC provides a means to design control systems that not only acknowledge these limitations but anticipate and avoid them while providing optimal control action. In fact, even predictive infusion shut-off systems [128], which temporarily halt basal insulin therapy when continued treatment is predicted to cause hypoglycemia, without any further feedback capabilities have been shown to drastically improve system safety by decreasing the risk of overnight hypoglycemia. However, the effectiveness of these simple predictive shut-off algorithms can be diminished if the patient already has significant insulin in their subcutaneous or plasma compartments. This point is of particular concern when insulin is infused according to feedback control laws, which may frequently exceed basal rates. Thus, many recent systems use insulin action curves to implement estimates of the patient's "insulin-on-board" (IOB) [129] to account for delayed insulin effects as part of their predictive algorithm and apply infusion rate limits to avoid dangerously aggressive profiles when glucose concentrations are low.
Similar to IOB estimates, another recent and promising MPC algorithm uses a more accurate estimate of the patient's plasma insulin concentration (PIC) to perform online adaptation of dosage constraints for individual patients. This method has performed well in tests on the UVa/Padova simulator without using any meal announcement but has not yet gone to clinical trials [130].
While IOB and PIC estimates can be useful for minimizing unnecessary control actions, the zone MPC approach attempts to resolve this issue by replacing the usual desired blood glucose set-point with a larger region, or zone, of acceptable concentrations [123]. Thus, the objective of the controller is better aligned with the normal physiological regulation of blood glucose to the euglycemic range, rather than a single specific concentration, and control action is only applied inside this range if the predictive algorithm anticipates an excursion.
When no excursion is predicted, the basal infusion rate is maintained, greatly reducing unwanted blood glucose variations and the risk of insulin induced hypoglycemia. Variations of the zone MPC approach include periodic zone-MPC control [124], which accounts for diurnal variations in activity and reduced hyperglycemic risks overnight, and zone MPC with adaptive cost functions [126], [131].
Considering the non-negative nature of blood glucose/insulin concentrations and infusion rates, the performance limitations of the diabetes control problem, without glucagon, has been likened to recent results regarding fundamental trade-offs limiting the achievable response in linear positive systems [132]. Despite nonlinearities in the UVa/Padova simulator, this approach may provide a means to obtain (in the ideal numerical case) the control signal that would give a near ideal combination of safety and performance, which has been suggested as a potential benchmarking tool for simulation studies on AP systems. One recent MPC variant [126] seeks to achieve this ideal response by applying (i) a "velocity-weighted" optimization wherein the cost of blood glucose deviations is a function of the rate of change of the measured blood glucose and (ii) a "velocity-penalization" term in the cost function that penalizes actions that would allow blood glucose to increase (i.e., have a positive rate of change) when blood glucose is tending toward hyperglycemia. In combination, the velocity-weighting and velocity-penalty MPC scheme acts aggressively against disturbances by switching on the velocity-penalty at the first signs of impending hyperglycemia, as evidenced by elevated blood glucose levels (above some threshold) with a positive rate of change, while avoiding hypoglycemia due to over corrections by effectively tapering the control action as the blood glucose rate of change decreases. Notice, this combined action is reminiscent of a nonlinear PD control with predictive capabilities and appears to provide some benefits of both MPC and PD control strategies.
Model Individualization: Recalling that every patient affected by type 1 diabetes may respond to treatment differently, there is no single model or set of model parameters that can capture every patient's physiology with acceptable accuracy. One method to eliminate the uncertainty (and conservatism) associated with the presence of such interpatient variability is to generate patient specific models prior to system deployment or during an initial training period. To this end, research into individualization of models [133]- [136] has been an active subject for many years, and remains an open problem in diabetes control. Despite this, individualized models alone may not be sufficient to account for the intrapatient variations discussed previously, and adaptive control schemes may provide a more robust answer to both kinds of variability.
Adaptive Control: Unlike the model individualization techniques discussed previously, adaptive online identification techniques can be used to overcome both interpatient and intrapatient variability. Clinical uses of online model identification with MPC date back to the transformative Advanced Insulin Infusion using a Control Loop (ADICOL) trials of 2004 [74], where model parameters were updated at every control interval using Bayesian methods over a predetermined learning window. Other implementations of adaptive control in MPC include the run-to-run tuning methods in [121], the cost function updating schemes in [126], [131], and several of the adaptive schemes discussed in [137].
Furthermore, generalized predictive control (GPC) algorithms [122], [138] have been used successfully in both clinical and simulation studies. Like MPC, GPC strategies utilize an internal model to predict future states and determine the next immediate optimal control signal. However, in GPC, a simplified model is used for prediction, and this model is updated (i.e., adapted) at every control interval to minimize the difference between the predicted and actual responses. Recent implementations of GPC in diabetes are reported in [137], [139] using recursive time-series models and in [130] using recursive predictor-based subspace identification.
4) The Control to Range Method: Similar to the two-step approach taken by the SAFE algorithm [116], the control-torange (CTR) method was proposed as a modular two-element strategy consisting of (i) a range correction module (RCM) that operates at a longer (e.g., 1-15 minutes) sampling interval to maintain blood glucose within a specified target range and (ii) a safety supervisor module (SSM) that operates at a much higher sampling rate to predict and prevent dangerous hypoglycemic excursions [140], [141]. Although this concept may be applied without regard for the control method used, the CTR strategy has developed around the use of MPC-based RCM algorithms [141]. 5) Fuzzy-Logic Control Methods: While PID and MPC have historically dominated the field of diabetes control, more recently, fuzzy logic controllers [138], [142]- [144] have been quite successful in generating AP systems that closely mimic expert clinical decision making in the face of complex environmental factors and incomplete information.
Among the most prominent fuzzy AP systems used in clinical trials is the MD-Logic system [142], [145], which includes information on the patient's clinical physiology and therapy guidelines obtained from a set of training data to provide fully automated individualized care (without meal announcement) using a combination of control to range and control to target (standard set-point control) methodologies. Treatment decisions are made using past, present, and projected future data (based on trends), forced to abide safety constraints similar to those discussed previously, and applied according to a variable basal infusion plus bolus strategy.
Alternatively, another clinically tested fully closed-loop fuzzy-logic controller (without meal announcement) known as the dose safety controller (DSC) [143] eliminates the basal insulin infusion entirely and instead applies a micro-bolus strategy based on quantized sets of blood glucose level, rate (i.e., first time derivative of blood glucose), and acceleration (i.e., second time derivative of blood glucose). To individualize treatment, each patient is given an initial "personalization factor" (PF), which scales the dosage associated with each level-rate-accelerate combination, and the PF is adjusted to provide more or less aggressive treatment depending on the occurrence of hypoglycemic events. More information on recent updates to the DSC is available online at [146].
6) Run-to-Run Control: Apart from the more well-known PID, MPC, and fuzzy-logic controllers, a somewhat unique class of industrial process control techniques known as runto-run algorithms [147] have been used for a variety of tasks in diabetes care. Run-to-run methods utilize a measurementbased algorithm to improve the effectiveness of a controller between distinct iterations of a repeated process. For diabetes, this process may be the 24-hour/3-meal cycle [147]- [149], in which case the control law is updated before the start of each day to account for some intrapatient variability and improve the rejection of blood glucose disturbances due to meals. Alternatively, similar run-to-run methods have been used to adjust basal insulin infusion rates in simulated trials [150] and to improve the accuracy of CGM measurements based on week-to-week recalibration [151]. 7) Alternative Control Methods: While clinical studies have been dominated by PID, MPC, and Fuzzy-Logic controllers (with several using CTR), a variety of additional paradigms have been studied in academic simulations. These methods include artificial intelligence methods [152], H∞ control [55], linear parameter varying (LPV) control [77], and sliding mode control [153]. However, recent clinical studies implementing these techniques are sparse, if available at all.

E. Outlook for Automated Diabetes Treatment
The insulin-based treatment of type 1 diabetes is currently in the midst of an international transformation from manual to automated care. While more complete evidence of the benefits and burdens of closed-loop therapy will become available as these systems are adopted, the momentum behind this transition is bolstered by the significant well-known improvements in safety and accuracy of closed-loop control over sensor augmented pump (SAP) therapy and manual multiple daily insulin injection (MDI) approaches during overnight blood glucose regulation. Furthermore, both hybrid closed-loop (e.g., requiring meal announcement) and fully closed-loop treatments have demonstrated promising additional improvements in more general simulated and clinical scenarios including ambulatory and free-living conditions.
Since the initiation of modern closed-loop insulin therapy clinical trials with the 2004 ADICOL studies [74], there have been over 100 publications concerning new clinical trials. A collection of available data regarding recent (2004-2016) clinical trials obtained from several studies [85], [105] is shown in Fig. 5. This data was cross-referenced against the AP clinical trial publication database [154] and shows a trend toward the use of MPC in clinical studies. Among the attractive features of MPC is its predictive adherence to complicated constraints, which cannot be offered by PID control or controllers that implement reference conditioning such as the SAFE algorithm.
Despite interest in MPC, the first and only commercially available hybrid closed-loop system to-date (MiniMed 670G by Medtronic), released in 2016, uses a PID controller with insulin feedback (IFB) [155], [156]. While recent studies regarding the effectiveness and patient reception of this device are promising [155], [157], many patients expected the device to have a higher degree of autonomy and found the system difficult to wear. However, as a larger variety of commercial systems and fully automated AP systems become available over the next few years [158], patients will be able to select between systems to find one that best meets their needs and expectations. With the widespread adoption of these technologies will come new safety concerns and hazards [159] associated with the unfavorable conditions that may be encountered during continued, long-term, free-living use. Early action to preemptively address these obstacles may provide substantial benefits to patient safety, ergonomics, and perception of closed-loop artificial pancreas systems. Such actions might include adoption and development of standards relevant to commercial artificial pancreas systems, such as IEC 60601-1-10 [13], and familiarization with relevant FDA guidance (e.g., [160], [161]) and regulatory considerations arising from public discussions (e.g. [9]).
In addition to commercial systems, opensource solutions to the treatment of type 1 diabetes, such as the open artificial pancreas system (OpenAPS) [162], have been gaining popularity in recent years. These systems are not regulated by the FDA and must be designed and constructed by diabetic patients themselves. While there is a growing repository of source code and community feedback for interested patients, and the results of the higher-end patient-driven studies [163], [164] are impressive, the safety and performance of opensource medical systems raise serious concerns. Implemented across a broad and inconsistent scale, these systems are vulnerable to algorithmic, communication, and electromechanical errors, which may arise when devices are combined and used in a manner inconsistent with their design. While some projects, such as OpenAPS provide a thorough reference guide and code that incorporates safety limitations, the essence of opensource implies that there is inevitably little preventing the enduser from tampering with these constraints and accidentally creating dangerous devices. This becomes an issue of further concern when considering the potential for proliferation of improper designs and the estimated 1/3 of OpenAPS users who are children operating systems designed by their parents [165] (however this may not be representative of the larger opensource community). The availability of simple and reliable AP systems may help to significantly reduce the number of patients who choose to develop their own AP systems, and in fact the desire for such systems to come-to-market appears to underlie the goals of OpenAPS. Even with the availability of name-brand, FDA certified HCL and fully closed-loop systems, curiosity surrounding their operation -as well as the desire to replicate and improve it -will undoubtedly drive continued interest in opensource solutions.
Finally, it is important to keep in mind that poor glycemic control is a consequence of type 1 diabetes and not the underlying cause. As medical practice and technology progress, the artificial pancreas may be able to grow beyond insulin infusion and implement more comprehensive and physiologically appropriate therapies. Steps in this direction include the ongoing work to facilitate bi-and tri-hormonal systems augmented with glucagon and amylin. Furthermore, there may be benefits to the investigation of closed-loop therapies that are capable of affecting both glucose/insulin concentrations and insulin action (i.e., insulin sensitivity) -perhaps through combination therapy with non-insulin treatments -to provide personalized treatment options for the large and diverse population of diabetics with forms of diabetes other than T1DM. Of course, economic viability, education for clinicians and new users of AP devices, and research into a cure for diabetes may ultimately affect the impact and longevity of automated AP technology.

IV. AUTOMATED ANESTHESIA
The word anesthesia means the loss of sensation and can be defined as the lack of response to and recall of noxious stimuli. The first application of anesthesia was performed by Crawford Williamson Lang in 1842 using inhaled ether [166]. In clinical settings, anesthesia is induced by the administration of anesthetic drugs into a patient's body to achieve localized or general anesthetic effects. In the latter case, the term "general anesthesia" refers to a lack of movement across the whole body while also maintaining an unconscious state. In general anesthesia, patients may be given two types of anesthetic drugs: (i) inhaled anesthetics (gases or vapor drugs) and (ii) intravenous anesthetics (injection drugs). A concise comparison between these two methods is described well by Nascu: "Intravenous medications are given directly into a vein and are commonly used to induce anesthesia, as induction is usually smoother and more rapid than that associated with most of the inhalational agents. Intravenous anesthetics administered as repeated bolus doses or by continuous infusion may also be used for maintenance of anesthesia, either alone or in combination with inhalational agents. An advantage of inhaled anesthetics is that measuring the difference between inhaled and exhaled concentrations allows an accurate estimation of plasma or brain drug uptake· · · " [167].
Generally, intravenous anesthesia drugs can be categorized into three different classes based on their physiological effects: (i) hypnotic drugs, (ii) analgesic drugs, and (iii) neuromuscular blocking (NMB) drugs. The first kind of anesthetic, called hypnotics, are used to numb the brain, keeping the patient unconscious during surgery. Propofol is currently the most common type of intravenous hypnotic drug due to its fast redistribution and metabolism inside the body, its lack of accumulation in tissues [167], and its lower chance of negative side effects compared to other hypnotic drugs [168]. The second class of anesthetics, called analgesics, attenuate the sensation of pain. Currently, opioid analgesics (e.g. remifentanil) are the most common type of analgesic drug. Finally, the third class of anesthetics, known as NMB drugs, interrupt the transmission of nerve impulses in the neuromuscular junction and cause paralysis of the affected skeletal muscles [166]. NMB can be used to facilitate the processes of endotracheal intubation and mechanical ventilation [169], [170]. As shown in Fig. 6, each of the 3 classes of drugs used in anesthesia contribute, respectively, to achieving the 3 main targets of general anesthesia: (i) hypnosis, or loss of consciousness (due to hypnotic drugs), (ii) analgesia, or loss of sensitivity to pain (due to analgesic drugs), and (iii) immobility (due to NMB drugs).
General anesthesia is performed in 3 phases: (i) induction, (ii) maintenance, and (iii) emergence. Induction is most critical phase in anesthesia because all the three types of drugs -hypnotic, analgesic, and NMB -have to be administered at high enough doses to induce all 3 main aspects of anesthesia and allow the anesthesiologist to insert an endotracheal tube for mechanical ventilation to help the anesthetized patient breathe (i.e., endotracheal intubation).
Maintenance follows the induction phase. In this phase, the anesthesiologist balances anesthesia with the use of anesthetics agents, manages the infusion of intravenous fluids, and regulates the administration of different drugs to maintain hemodynamic stability (i.e., stable blood flow), normal function of body organs, and adequate depth of hypnosis [171].
Emergence is the last stage of anesthesia and occurs at the end of surgery when the patient is removed from anesthesia and brought back to normal conditions as drug administration is tapered off. The goal of emergence is the full reversal of neuromuscular blockade and the smooth return of spontaneous ventilation and reflexes while maintaining the stability of hemodynamics and other physiological functions. Thus, the return of consciousness can occur in a safe and comfortable environment [171].

A. Monitoring
To achieve satisfactory anesthesia, the amount and timing of drug infusions must be estimated in each phase during surgery. Several clinical indicators and physiological parameters effect the anesthesiologist's estimation. Clinical indicators such as heart rate, blood pressure, lacrimation, sweating, and papillary dilatation are monitored during surgery and provide anesthesiologists valuable information about the anesthetic adequacy [172]. However, physiological monitors (i.e., electromyography (EMG), electrocardiography (ECG), blood pressure (BP), electroencephalography (EEG), and oxyhemoglobin saturation (SpO2)), and quantified indexes such as bispectral index (BIS), entropy, auditory evoked potential (AEP) and surgical stress index (SSI) can provide anesthesiologists more accurate information about a patient's status [173]. The monitors and quantified indexes corresponding to several important clinical signs and physiological parameters are shown in Table V for  reference. Analyzing the electroencephalogram (EEG) is one of the most common approaches for precise evaluation of anesthetic delivery. EEG signals depict the electrical activity in the cerebral cortex and the characteristics of the EEG waveforms (i.e., frequency and amplitude) vary by the type of drug that has been used and the amount of its dosage [174]. There are a number of signal monitoring devices such as the Bispectral Index (BIS) monitor, Narcotrend monitor, Cerebral state monitor (CSM), and AEP monitor [175] that measure and quantify the EEG in order to derive a surrogate measurement of anesthetic adequacy [176], [177].
Bispectral index (BIS) is the most notable index and is closely related to the level of consciousness [178]. Measurements of the BIS provide anesthesiologists important information during anesthesia, and clinical trials have shown its potential to increase the patient safety. [179], [180]. The BIS index is scaled between 0 and 100, where a value of 100 represents the fully conscious state and value of zero depicts an isoelectric EEG. With increasing concentrations of anesthetic drugs, the BIS index decreases [181]. Fig. 7 shows the patient state for ranges of BIS indices along with the corresponding effects on memory of surgical events. Acceptable BIS scores for general anesthesia range from 40-60 [178]. Despite its wide-spread use, recent experimental results have shown that the BIS index is not reliable when patients have been infused with NMB drugs. In one pivotal study [183], fully awake subjects were shown to achieve BIS scores below 50 (i.e., corresponding with BIS during general anesthesia) when dosed with NMB drugs alone. Furthermore, additional research has suggested that BIS values should be interpreted with caution when patients are children [184], elderly [185], have known neurological disorders, or are taking psychoactive medications [186]. Beyond BIS, it has been noted that, "the use of DoA monitors remains a major controversy in anesthesiology. Indeed, many factors, such as age, race, gender, low core body temperature, acid-base imbalances, low blood glucose, drugs administered to the patient (e.g. neuromuscular blocking agents), and brain ischaemia, have a significant effect on raw EEG data. Additionally, DoA monitors are limited by their calibration range and inter patient variability in dose response curves..." [186]. These limitations and interactions can degrade the performance of manual infusion systems and will nodoubt impose challenges to high-quality closed-loop control of anesthesia. Recently, patient state index (PSI) algorithm-based devices have been considered as alternatives to BIS monitors for their high sensitivity to variations in sedation/hypnosis and uniform response to various anesthetic agents [186], [187]. Although the PSI devices have some advantages to classical EEG monitors, including reduced sensitivity to noise, the PSI index is more sensitive to interpatient variability than BIS [188] and may be affected by EMG signals [186], [189]. In general, there remains a need for studies investigating the effects of drug combinations and EMG signals on sedation indices. For more detailed information on current challenges and limitations of monitoring during anesthesia, readers are referred to the extensive coverage in the literature (e.g., [186]).

B. Modeling
An accurate infusion of anesthesia drugs and continuous monitoring of brain states during each phase can help decrease the frequency and severity of dosage errors, improve the efficiency of anesthesia, and reduce surgical costs. As mentioned in Section IV-A, the BIS index is the most common monitoring metric available for displaying a patient's anesthesia level. However, to fully utilize the BIS and other measures for optimal drug dosing, a mathematical model is needed that represents all aspects of anesthesia including the effective physiological parameters and various dynamics (i.e., corresponding to the hypnotic, analgesic, and paralytic states). Various such models exist and can be categorized as singleinput single-output (SISO), multi-input single-output (MISO), or multi-input multi-output (MIMO) models. In what follows, anesthesia models from each of these three classes will be briefly discussed.
1) SISO Model: The pharmacokinetic/pharmacodynamic (PK/PD) model is a compartmental model that illustrates both the drug distribution in the human body and its effect on certain physiological systems. For anesthesia, the pharmacokinetics are represented by a linear compartmental model that defines the concentration of the administered drug in different parts of human body, while the pharmacodynamics are represented by a static nonlinear function (i.e., Hill curve) relating the resulting drug concentration to the measured effect (e.g., BIS index) [174], [190].
The current PK/PD model is a SISO model with either hypnotic drugs (e.g., propofol) or analgesic drugs (e.g., remifentanil) as the input and either patient consciousness or degree of pain as the output. Such SISO models are used mainly in anesthesia modeling by control engineers and clinicians. As shown in Fig. 8, the PK/PD model of anesthesia is a compartmental model with three main compartments and one effectsite compartment where V 1 describes the central compartment including arterial blood, brain, and liver volumes; V 2 denotes the fast peripheral compartment, which has fast dynamics and contains the well perfused body tissues such as muscles and viscera; V 3 represents the slow peripheral dynamics, which include the dynamics of poorly perfused body tissues like fat and bone; and the effect-site compartment is used to account for the time lag between drug concentration in blood plasma and its effect [191].
Pharmacokinetic Model: A pharmacokinetic model of a drug is a mathematical expression relating the concentration C p (t) (mg/ml) of the drug in the blood plasma to the infusion rate of the drug into the central compartment I(t) (mg/min) [178]. Pharmacokinetic compartmental models are derived by establishing balance equations for the amount of drug (x i in mg) distributed in each compartment. Accordingly, the governing equations for the PK model in Fig. 8 are given byẋ Fig. 8. Patient states corresponding to BIS indices (adapted from [181], [182]. where the constants k ij (min −1 ) indicate the transfer rate of the drug from the i-th compartment to the j-th compartment; the constant k 10 is the rate at which the drug is metabolized; and V 1 is the volume of the first compartment. These equations can be written in state space as Then, the transfer function for the PK model from infusion rate I(t) to concentration C p (t) = x 1 (t) is where every p i is a function of the constants k ij as given by the conditions    p 1 + p 2 + p 3 = k 10 + k 12 + k 13 + k 21 + k 31 p 1 p 2 + p 1 p 3 + p 2 p 3 = (k 10 + k 12 ) (k 21 k 31 ) + k 13 k 21 p 1 p 2 p 3 = k 10 k 21 k 31 (11) The parameters k ij are different for each person and can be calculated as [191]       where V i and Cl i denote, respectively, the volume and clearance rate (i.e., the rate at which the drug is cleared from the body or removed from the central compartment) of the i-th compartment. For propofol, the clearance rate and volume parameters can be obtained as  It should be noted that although the above expressions are the most commonly used relations for calculating the values of V i and Cl i , there are alternative, generally more complex, relations used to calculate these parameters for the infusion of propofol, such as those developed by [192].
For the infusion of remifentanil, the PK parameters can be calculated as [193], [194]  Note, in (13)- (16), the patient's height, weight, and age are given in units of cm, kg, and years, respectively. Furthermore, the lean body mass (lbm) is calculated differently for male and female patients and can be obtained as [195] lbm m lbm f = m 0 0 f where m and f are set equal to one and zero, respectively, for male patients and vice versa for female patients.
Pharmacodynamic Model: Pharmacodynamic models are used to describe the relationship between the concentration of a drug and its effect on the body. As shown in Fig.  8, the general PD model has two parts. The first part is a compartment called the effect-site, introduced by [196], that represents the lag between drug concentration and its effect. In the case of anesthesia, since the effect site compartment is small in comparison with the central compartment, its effect is assumed to be negligible [197]. In steady-state, the concentration of the effect-site can be related to the plasma concentration byĊ where k e0 and k 1e are constants and x e is the amount of the drug in the effect compartment. The rate at which drugs enter and exit this compartment is assumed to be constant and equal to k e0 (e.g., k 1e = k e0 = 0.456 for propofol) [191]. For the infusion of remifentanil, k e0 can be calculated as Consequently, (19) is often used aṡ where C e is the concentration within the effect-site compartment. Thus, for propofol, the effect site works like a low-pass filter that relates the plasma propofol concentration (i.e., in the blood) to the effect-site propofol concentration. The transfer function describing the relation from the plasma concentration to the effect-site concentration is given by The second part of the PD model, shown as the "nonlinear drug/effect relation" in Fig. 8, is often comprised of a sigmoid Hill equation where E 0 indicates the baseline value (obtained from the awake state without propofol), which is typically set to 100; E max is the maximum effect achievable by the drug infusion; EC 50 denotes the drug concentration at half maximal effect, which represents the patient's sensitivity to the drug and should be measured experimentally; and γ determines slope of the sigmoid curve (i.e., the receptiveness of the patient to the drug) [198]. In (22), the value of the effect E is obtained in terms of the BIS scale. In Table VI, the values of these parameters are shown for a cohort of 12 representative patients. These values are among the most frequently used for designing and simulating anesthesia control systems. Additional data sets are available in [166]. Although the aforementioned models are used quite frequently, they are in need of some refinements. Among other issues, the most critical weaknesses of the models presented thus far include a lack of standard parameters for other kinds of hypnotic drugs, important neglected factors such as cardiac output and mode of drug administration (e.g., bolus or infusion), and unmodeled drug interactions [174]. It is also worth noting that the commonly used values in Table II are not representative for a variety of important demographics. In fact, in the case of infants and adolescents, different PK/PD models are required, which have been described completely in [199]- [202].
It should be noted that the SISO models discussed above incorporate the infusion of hypnotic and analgesic drugs but do not consider neuromuscular blocking drugs because neuromuscular blockade has no effective interactions with anesthetic drugs and opioids. In some studies, separate (different) SISO models are used to represent the dynamics of NMB [203], [204]. Furthermore, a different PK/PD model structure, comprised of one central compartment (representing both  [198] blood and organs heavily supplied with blood), one peripheral compartment (representing the parts of the body with a lower blood flow), and the effect compartment (representing the PD model) is presented in the literature [205], [206]. Alternatively, a second order linear dynamic model is presented for NMB in [207].
2) MISO Model: While the SISO model in Section IV-B1 can be used separately for both hypnotic and analgesic drugs, it is not capable of fully representing the combined administration and effects of these drugs. As a work around, some studies have used parallel SISO models of each drug to obtain a simple MISO model. In this arrangement, the WAVCNS index used by the NeuroSense NS-701 (NeuroWave Systems, Cleveland Heights, USA) monitor appears to be a more suitable measurement index than BIS for measuring the anesthesia level because it is obtained through a linear timeinvariant relation that does not cause an additional delay in the closed-loop system [208], [209]. The interaction of propofol and remifentanil is described in [210] as where ν p = (c p e )/(EC p 50 ), ν r = (c r e )/(EC r 50 ), and the parameter α (identified from clinical data by [210]) determines the degree of interaction between propofol and remifentanil. This interaction is clasified as synergistic if α > 0; additive if α = 0; and antagonistic if α < 0. The parameters EC p 50 and EC r 50 are the propofol and remifentanil concentrations at half maximal effect.
As an alternative to using the PK/PD models of propofol and remifentanil in parallel, a new MISO Weiner model called the parameter parsimonious model (PPM) has recently been introduced by [211]. In this model, the effect-site concentrations of propofol and remifentanil (i.e., c p e and c r e , respectively) due to their infusion rates (i.e., u p and u r , respectively) are given in the Lapalce domain as C p e (s) = C r e (s) = l 1 l 2 l 3 η 3 (l 1 η + s) (l 2 η + s) (l 3 η + s) u r (s) where α and η are dependent on patient parameters and do not have any physiological meaning. The parameters k i and l j are dimensionless constants that were obtained from clinical data by [212] as k 1 = 10 , k 2 = 9, k 3 = 1, l 1 = 3, l 2 = 2, and l 3 = 1. The corresponding BIS value z(t) is approximated by [211] as where In these equations, the parameters µ and γ do not have any physiological meaning and are dependent on each patient. This model can be expressed in state space as [213] x (t) = Ax (t) + Bu (t) C p e (t) C r e (t) where The common values for α, γ, µ and η are identified by [214] and are given in Table VII.
Although the PPM model is not entirely physiological, it is better suited to model-based control than the general PK/PD model because it has fewer parameters that need to be  [213] identified [213]. However, both the PK/PD and PPM models have been used in many controller design studies. Some important factors for choosing between these models include the availability of measurement devices, the type of desired control structure, the types of drugs being administered, and the costs associated with the overall design and implementation. In the following sections, some recent studies that utilize these models will be discussed.
3) MIMO Model: While the previous models have a single measurable output (e.g., BIS or WAVCNS), there are several other variables that can be considered as additional system outputs. Two MIMO models have been reported prevalently in the literature, both of which take the dosage of hypnotic and analgesic drugs as inputs but differ in the quantities used as outputs. When considering MIMO systems with m inputs and n outputs, we can define m × n transfer functions that describe the effects of each input on each output.
The first of these MIMO models [215], which is used less frequently in the literature, includes some clinical metrics like heart rate and blood pressure as system outputs. By assuming the analgesic drugs have negligible effects on the BIS value [216] and considering depth of hypnosis (DoH) and blood pressure as the system outputs, this model is given by x P = e −τ P s K P T P s + 1 U P (s) where indexes B and P denote hypnotic and analgesic drugs, respectively; τ B and τ P are initial time delays; K B and K P are defined as drug sensitivities; T B and T P are time-constants representing the response speed of the patient; f B and f P are defined in [217] as nonlinear sensitivity functions; d B and d P are external disturbances; U B (s) and U P (s) are the Laplace transforms of drug infusion rates; x B and x P are the effectsite concentrations of the anesthetic drugs; and y B and y P are the DoH and mean blood pressure, respectively. According to (27), this representation consists of four (i.e., 2 × 2) different transfer functions, which are reported in [215].
The second common MIMO model, depicted in Fig. 9, defines the infusion rates of propofol (i.e., the hypnotic drug) and remifentanil (i.e., the analgesic drug) as the inputs, and the patient's depth of hypnosis and degree of pain are used as the system outputs. Thus, the depth of hypnosis and degree of pain are interpreted from BIS and state entropy (SE) measurements, which consequently may be considered alternative model outputs. The state-space of this model is similar to the MISO model of (24)-(33); the main difference between these models is the output matrix, which is described as [218] where y 1 denotes the BIS value; y 2 denotes the SE value; y 01 and y 02 are equal to 97.7 and 91, respectively; and ρ 1 , ρ 2 , γ 1 and γ 2 are parameters that are estimated by clinicians.

C. Automation in Anesthesia
In 2010, 48.3 million surgical operations were performed in the U.S. [219]. Many of these surgeries require general anesthesia, yet ensuring optimal and on-time infusion of anesthetic drugs remains an important challenge that is complicated by the unique physiology of each patient and numerous known and unknown environmental factors. Further complicating this problem, patients' anesthetic requirements are known to change during surgery in both predictable and unpredictable ways, and once administered, anesthetics cannot be removed. Poor regulation of anesthetics, such as too little hypnotic or too much analgesic, can result in complications including anesthesia awareness (i.e., unintended consciousness during surgery), pain during surgery, prolonged emergence period, and increased risks of other side effects [220]. As an example of the regularity of these complications, Table VIII shows data collected on the incidence of anesthesia awareness over the last half-century.
Although the statistics above show a near steady improvement, there are still many faults occurring during anesthesia administration which may be avoidable by the development of an automated anesthesia system that improves the administration of anesthetics and mitigates the hazards of system faults. In fact, besides a reduced risk of surgical complications, the use of an automatic control system may provide significant additional benefits such as a reduction in the cognitive workload of the anesthesiologist (who must be present to supervise any automated system), a decreased volume of drugs administered (which implies a faster and better recovery time of the patient in the post-operative phase), and a more robust performance with fewer over-and under-dosing episodes and better rejection of surgical disturbances. Of course, the main goal is increased safety and comfort for the patient [222]. The advantages of automatic anesthesia systems over manual anesthesia have recently been validated by experimental results [223]. Any automated anesthesia system will likely include two main phases: first, the desired depth of anesthesia (DoA) (i.e. desired BIS index) must be reached, and second the DoA needs to be maintained by the application of various anesthetic drugs. During the first phase, the system should calculate the sufficient drug dose and regulate its infusion to move from the initial BIS value E 0 of the patient during the induction phase to a final desired BIS value. Note, the determination of a desired DoA means setting final values for all the three types of anesthesia (hypnotic, analgesic, and neuromuscular blocking drugs). However, to date, most closed-loop studies have been focused only on achieving the depth of hypnosis (DoH), which is the most important type of anesthesia, while the analgesic and neuromuscular blocking drugs must still be infused manually by clinicians.
During the second phase, the system should infuse the drugs continuously while monitoring the aforementioned physiological parameters and clinical signs and also rejecting disturbances that occur during the maintenance phase. The main goal during this phase is to maintain the patients at the final BIS value throughout the surgery until anesthesia is stopped during the emergence phase.
Control applications in general anesthesia began receiving attention several decades ago [224], [225] from both control engineers and clinicians. Throughout this period, a variety of methods have been proposed with varying degrees of automation. As illustrated in Fig. 10, most systems can be categorized as manual, open-loop feed-forward, or closed-loop controllers based on the approach used to calculate the drug infusion rates in general anesthesia.
Manual systems (i.e., without automation), shown in Fig.  10a, are the current standard practice. In these systems, an anesthesiologist first sets the final value of the DoA or DoH and then monitors the anesthetic state of the patient and regulates the drug dosages accordingly [226]. The success of this method is highly dependent on the expertise of the individual anesthesiologist and is not reliable in some critical situations (e.g., irregular events during surgery).
In 2003, a large step toward achieving fully automated anesthesia was taken with the introduction of target controlled infusion (TCI) systems [227]. The Diprifusor system was the earliest commercial TCI pump for propofol [228] and since its introduction several other TCI systems have become available from various manufacturers. In fact, developments and modifications to TCI systems has made them a standard among infusion systems for the administration of opioids, propofol, and other anesthetics in clinical practice [228]. Considering our categorization, TCI systems operate as openloop feed-forward controllers, shown in Fig. 10b, which rely on both an anesthesiologist to assess the patient's state and the output of a PK/PD model to estimate an exact drug dosage and infusion time. In practice, the anesthesiologist sets a target drug concentration, and the TCI system determines an infusion rate and sends this value to the infusion pump. Although these systems are used in many countries and have some advantages over manual anesthesia systems, the success of TCI still depends heavily on the expertise of the anesthesiologist and does not have its own means of identifying the patient's current needs when tuning the infusion rate. Since TCI lacks real-time feedback, its performance depends on the accuracy of the patient model and is susceptible to disturbances caused by surgical stimulation and hypnotic-opioid synergy (i.e., the combined effects of these drugs) [226].
Closed-loop controllers are the most advanced of the 3 categories and consider the individual patient's physiological parameters in addition to the clinical metrics (used by anesthesiologists in manual and TCI approaches) in the process of calculating infusion rates. As mentioned in Section IV-A, such physiological parameters may already be available as the measured output of advanced monitors that record and display values including the patient's EMG, EEG, ECG, and BIS. In this case, feedback control is used instead of an anesthesiologist to frequently adjust the infusion profile or target concentration in closed-loop controllers; however, anesthesiologists are still needed to set the desired DoA or DoH and supervise the patient's state via clinical metrics. The result is a reduction in the anesthesiologist's workload, potentially helping to avoid problems associated with distractions and fatigue, as well as increased safety for the patient due to continuous monitoring, the possibility of lower administered drug dosages with a faster postoperative recovery, and a reduction in the occurrence of drug-induced side effects [229]. For these reasons, fully automated systems may have the potential to one-day outperform manual infusion dosing [230], [231].
Two different approaches for implementing closed-loop controllers in anesthesia are shown in Fig. 11. The first, in Fig.  11a, is based on a scheme that directly identifies the anesthetic Fig. 11. Two commonly used closed-loop control architectures in anesthesia drug delivery with (a) direct control of the infusion rate, and (b) cascaded control of TCI system (adapted from [226]) infusion rates and sends control signals directly to the infusion pump (e.g. [232]). The second, in Fig. 11b, continuously adjusts the target value for a downstream TCI system, which in turn continuously sets the rate of the infusion pump (e.g. [233], [234]). The second method can be considered a special case of the first, and so the first (direct) approach is less restrictive and may be preferable for control design as the TCI dynamics can be replaced with any effective algorithm.
Generally, automation in anesthesia consists of three main components: a patient model, a measurement system (i.e., sensors or monitors), and a controller. Each of these components can have significant effects on the performance of the other components and the overall system. For example, as mathematical models become more accurate and include more detail on variables and effective parameters, controllers can be designed to provide higher quality drug infusion. Furthermore, as measurement devices become more accurate, individualized physiological model parameters can be identified more accurately and controllers will have access to a higher quality and quantity of relevant information via feedback and observation -ultimately resulting in improved drug infusion. Note, the quality of monitoring directly effects the controller performance in implementation.
While Sections IV-A and IV-B have focused on modeling and monitoring, controller design is among the largest remaining challenges to automated anesthesia, and we will now review several prominent strategies. Many scientists and control engineers have explored various control approaches such as PID, MPC, fuzzy-logic, adaptive, and neural networks as well as hybridizations of these controllers to design and produce automatic anesthesia infusion systems for clinical use following the general structure in Fig. 12. Some comprehensive reviews on controller designs can be found in [174], [235]- [238].
While the majority of recent publications in the physiological closed-loop control of anesthesia are related to the use of intravenous anesthesia, several closed-loop studies have specifically considered the application of inhalational anesthesia [239]- [241]. One benefit of the inhalational approach to anesthesia is that it may be considered safer than intravenous anesthesia because end tidal gas measurements are available Fig. 12. Closed-loop control for drug administration in anesthesia (taken from [178]) with permission pending.
to monitor the drug concentration within the brain and the vessel rich group (VRG) at steady state. However, these sensors provide a poor approximation of the VRG concentration during common dynamic situations such as initial uptake and emergence [242]. Thus, some researchers have developed mathematical models to represent these dynamics (e.g., for purposes of concentration estimation) with the aim of investigating, by simulation and clinical studies, the closedloop control of inhalational anesthesia using model predictive control [243]- [245]. For detailed information on modeling and closed-loop control of inhalational anesthesia, interested readers are referred to the extensive coverage in the literature (e.g., [246]).
1) Challenges to Automated Anesthesia: During the design and simulation phases of the controller development, there are several important constraints that must be taken into account. First, each drug has a maximum effective dosage, which means higher doses will not have any greater effect on the patient. Second, because the control signal defines the infusion rate, it cannot have a negative value (i.e., the drug cannot be extracted once infused) and must further be within the operating range of available infusion pumps. Third, excessive doses infused over a short time can shock the human body and may harm organ systems [247]; this can be avoided by maintaining lower infusion rates. The selection of proper drug infusion rates is determined as a trade-off between the side-effects of rapid infusion and the desire to reach the required BIS value within a certain time-frame. In current surgical operations, the optimal time-frame has been reported as around 15 minutes [167], while shorter periods are frequently obtained in simulation studies found in the literature (e.g., 4-8 minutes in [222]). Lastly, the designed controller must work in the presence of both interpatient and intrapatient variability.
In addition to these systematic constraints, different sources of disturbances and noise -such as poor signal quality and surgical stimulations -should be considered in the system modeling and controller design activities. Of these disturbances, surgical stimulations are among the most challenging to manage and are a subject of current investigations by both clinicians and control engineers. As an example of how these stimulations may arise, Fig. 13 shows a typical timeline of the variations in BIS index, caused by surgical stimulations, during the time-course of an archetypal surgical procedure. In this figure, stimulus A shows the arousal due to laryngoscopy/intubation; B represents surgical incision followed by Fig. 13. A standard surgical stimulation profile (adapted from [179]). a period of no surgical stimulation (e.g., waiting for laboratory results); C represents an abrupt stimulus after a period of low level stimulation; D represents the onset of continuous normal surgical stimulation; E, F, and G simulate high amplitude short-term stimulations within the surgical period; and H simulates the withdrawal of stimulation during the closing period [179].
As discussed in Section IV-A, there is currently a need for more accurate, reliable, and robust sensing techniques that provide real-time monitoring of a patient's depth of anesthesia (DoA) without reliance on demographic characteristics or undesirable interference from EMG signals, NMB drugs, or interactions with other substances (e.g., opioids affect AEP measurements [248]). The present limitations to these sensing technologies may represent a fundamental barrier to accurate closed-loop control. Consider, for instance, the recent results regarding NMB drugs and BIS [183], which have a direct consequence on closed-loop control. According to this study, an anesthetic PCLC device can -under the right circumstances -achieve the desired BIS value without actually reaching the desired DoA (e.g., potentially resulting in anesthesia awareness). Even in the case where DoA is generally well represented by the sensing paradigm, automated anesthesia remains challenged by sensor noise, measurement error, and sensing delays.
In the next sections, the most common controllers used in closed-loop control of anesthesia will be discussed, and the recent developments and remaining challenges to each approach are presented.
2) Proportional-Integral-Derivative Control: Proportionalintegral-derivative (PID) control is one of the most used control approaches in industrial applications due to its simple mathematics, well-known gain tuning methods, and broad capabilities. Several successful clinical and simulation studies of automated anesthesia have been performed using PID based controllers over the past two decades such as [227], [249]- [254]. The simulated and experimental results of these studies have generally indicated that PID control may have some utility in the closed-loop control of anesthesia. One of the most common PID controllers used anesthesia is given by the Laplace domain transfer function [222] U (s) where U (s) and E(s) are, respectively, the Laplace domain control and error signals; k p is the proportional gain; T i is the integral time-constant; T d is the derivative time-constant; and T f is the time-constant of a second-order filter used to attenuate measurement noise.
In the relevant works, the proposed PID controllers have been shown to track desired anesthesia levels (e.g., desired BIS values); however, anesthetists and control engineers agree that this simple PID algorithm suffers from major deficiencies because it is not a robust and adaptive controller and does not perform well in the presence of noise and physiological variability. As a result, simple PID gives poor disturbance rejection, unwanted oscillations in the BIS response, and can suffer from windup of the integral control term during the induction phase. Therefore, recent studies are trying to address these problems to achieve comparable performance to other more advanced controllers. For instance, in [255] it is shown that the low-pass filter, included in (40), reduces the sensitivity of the derivative term to random changes. Furthermore, introducing zero reference weight for the proportional and derivative control terms (i.e. no reference input for proportional and derivative parts) results in better disturbance rejection and avoids unwanted oscillation in the BIS response [256]. Other investigations have shown benefits from applying a tracking time-constant as the geometric mean of the integral and derivative time-constants of the PID controller [257] and introducing reference shaping (i.e. the desired BIS index is changed from a step input to a specialized profile) [170] and integrator anti-windup [258] to prevent integrator windup during the induction phase. Some alternative approaches that have also been shown to improve the PID controller performance include using an event-based control scheme, which decreases the variations of controller signal and optimally cancels the noise and disturbances, and using an inversionbased methodology, which increases patient safety by reducing BIS overshoot and producing a smoother drug infusion rate [259]. From the clinical point of view, event-based control is similar to manual infusion and is more transparent to anesthesiologists because they can easily monitor the drug infusion profile [229].
Additional challenges to controller design stem from the presence of nonlinearities (i.e., a Hill equation) in the anesthesia dynamics. One common approach to handle this nonlinearity is to linearize (22) around the nominal operating point (e.g., a BIS value of 50) in the maintenance phase [260], [261]. This linearization converts (22) into a constant coefficient k m given by where BIS 0 is the nominal operating point in the maintenance phase. However, this approach is not feasible in the induction phase because the lack of a well-defined nominal operating point causes a large overshoot in the response due to the integral term of the controller. Recently, a new approach has been proposed, which is applicable to both induction and maintenance phases. This approach uses a linearizing filter in series with the inverse of the nonlinear part of the Hill function [256], [262].
Since the structure of PID controllers is generally welldefined, the selection or tuning of controller gains is very important. This task is challenging in closed-loop control of anesthesia, since the gains should be tuned according to the physiological parameters of each patient [263], [264]. While trial and error is common in tuning PID controllers [261], optimization methods, such as genetic algorithms [257], [265], [266] aimed at minimizing the integrated error, can increase the performance significantly [258], [267]. Nevertheless, the physiological parameters of patients vary based on age, weight, disease, and type of surgery being performed, and presently the available patient data is limited and does not adequately depict the physiological parameters of all patients. Therefore, online identification of patient parameters can be useful for improving the controller performance [268]. Additional studies have shown that the use of gain scheduling techniques may also be beneficial. In this case, the gains should be tuned different for different phases of surgery. During the induction phase, the gains are tuned to follow the set-point with a reasonable setting-time and low overshoot. Then, during the maintenance phase, the gains should be tuned for disturbance rejection. Therefore, when DoH or BIS levels reach the target value and have stabilized around the set-point for an adequate time interval (i.e., signaling the end of induction phase) the controller parameters should be switched to those tuned for the maintenance phase [222].
Another important point is that PID controllers are frequently used in different configurations (e.g., PD or PI only) for different dynamics. However, recent studies have proved that all three parts are necessary in anesthesia control. For example, in [269] the derivative part is eliminated to prevent noise amplification, but the performance in the induction phase and disturbance rejection in maintenance phase were less satisfactory for this PI controller than for the PID case, which had a shorter settling-time and less undershoot in the BIS response.
3) Model Predictive Control: Model predictive control (MPC) is a robust and optimal control technique used to handle systems with constraints while optimizing the controller action and predicting system outputs for robustness against noise and disturbances. The applicability of MPC and related strategies (e.g., generalized predictive control (GPC)) to medical systems -especially in closed-loop control of anesthesia -has been verified by both simulation and clinical results [270]- [273]. The outstanding features of this controller have recently encouraged control engineers to further develop MPC controllers to address the complex and nonlinear behavior of closed-loop anesthesia control.
One caveat with standard MPC is the complexity of the computations used to solve the online optimization problem, which prevents its use in real-time anesthesia control. This limitation can be overcome by using a multi-parametric MPC (mp-MPC) technique, which reduces the complexity of the computations by solving an offline optimization problem on an analytical function instead of numerical solutions [274], [275]. Further, to improve rejection of BIS noise and surgical disturbances, MPC has been implemented in combination with advanced state estimation techniques such as Kalman filtering and moving horizon estimation (MHE). Simulation results show that the MHE method has better accuracy and reduced overshoot compared to Kalman filtering [276], [277]. Likewise, studies have also shown that the addition of eventbased input and state output correction features to MPC can increase its robustness against noise and decrease the amount of anesthetic drugs administered [278], [279].
Another strong point of MPC is its ability to handle intrapatient and interpatient variability through online estimation of pharmacodynamic parameters [245], [280], estimation of the slope of the linearized Hill equation at each time-step [281], [282], and the use of offset-free and state output correction strategies [278], [283].
Piecewise linear PK/PD models can be used to address the problem of nonlinearities in the PD model, which represent one of the main challenges in closed-loop control of anesthesia. This technique defines several linear parts in the hill function to achieve a more accurate linear approximation of this function [198], [283]. Likewise, multiplying the inverse of the Hill function into the input command and the feedback signal is a common approach to cancel the nonlinear part of the anesthesia system [284]. Other techniques such as linear model predictive control (LMPC) [285]. switching the control strategy of mp-MPC to apply bolus treatments in critical BIS indexes [277], extended prediction self-adapting control (EPSAC) [198], and online identification of time delays [280] can improve the performance of closed-loop anesthesia as evidenced by reduced settling-times (i.e., obtaining a faster response), decreased overshoot, and the elimination of unwanted oscillations from the BIS response. 4) Adaptive Control: Adaptive controllers are frequently applied to systems with variable or unknown parameters, and implement algorithms that retune or restructure the effective controller during operation (i.e., online) [286], [287]. As mentioned, the PK/PD model of anesthesia is subjected to large (possibly time varying) uncertainties and its coefficients are different from patient to patient. Thus, adaptive controllers show promise for estimating parameters of the PK/PD model and improving the regulation of anesthesia.
Adaptive controllers that adjust their control action by directly changing controller gains (rather than an internal model) are known as direct adaptive controllers. One such method of direct adaptive control is to implement an adaptive proportional-integral-derivative control algorithm, which allows adaptive control of anesthesia without directly using the PK/PD model [288]. However, in practice, this controller produced undesirable oscillations and its performance was considered unacceptable. Some studies have also shown improvements to the performance of these controllers by considering the nonlinear uncertainties in the PK model and designing nonlinear adaptive controllers [289]. Additionally, neural networks have been used to improve the performance of nonlinear adaptive controllers for dynamic systems with complicated uncertainties [290]. In these controllers, states of the physical system remain in nonnegative orthant of the state space [291]. The effectiveness of this type of controller has been investigated by using several experiments and clinical evaluation [292].
As an alternative to direct adaptive control, indirect adaptive control uses parameter updating laws to identify unknown parameters in an internal reference model and thereby retune the effective controller gains. Several methods are used to obtain such control laws [234], [293], [294], and studies have shown that variations between simulated and real-world control performance can be attenuated by using a minimally parameterized model with an online identification strategy (e.g., an Extended Kalman Filter) to design adaptive nonlinear controllers [212].
Other types of adaptive feedback controllers have also been designed for anesthesia, including model reference adaptive controller (MRAC). However, MRAC controllers based on standard models do not show significant improvements in performance over non-adoptive techniques. Alternatively, MRACs may also be implemented using fractional order models [295], and the results of studies using fractional order models have confirmed the effectiveness and robustness of fractional order model reference adaptive control (FOMRAC), which can also compensate for the time delays in the anesthesia system [296]. Similarly, L1-adaptive methods can achieve faster adaptation than MRAC [297] and have demonstrated good performance and interpatient robustness [298]. In addition, system identification methods can reduce the order of patient models while also improving performance over standard L1-adaptive control [299]. Furthermore, L1-adaptive controllers can be used to ensure the safe switching between human-in-the-loop and automated close-loop modes of operation [300]. 5) Fuzzy-Logic Control: Since the compartmental models of anesthesia are not exact representations of the human body and are subject to variability in system parameters, closedloop control of anesthesia can benefit from fuzzy modeling and control. In fact, fuzzy-logic has been applied successfully to all three compartments of anesthesia (i.e., hypnosis [301]- [304], analgesia [305]- [307] and immobility [308], [309]). Some comprehensive reviews can be found in [173], [310].
The main benefit of using fuzzy-logic for anesthesia is its ability to express patient models by clustering real patient data without any knowledge of the underlying physiology. Most fuzzy anesthesia systems use regular (i.e., type-1) fuzzy sets, which have members that take crisp values in [0 1]; however, some systems use type-2 fuzzy sets where the membership values of each element are in another fuzzy set in [0 1] [311]. Finding the best membership function (MF) for such fuzzy sets in [0 1] is one of the main problems with using fuzzy models. To this end, genetic and neural network algorithms are commonly used to improve performance and optimize the type-2 fuzzy sets' parameters (e.g., footprint of uncertainty (FOU), centroid, and scaling factors) [312]- [315].
To develop fuzzy control laws, sets of rules are collected according to the expert opinions of anesthesiologists regarding the best corrective actions to take under sets of circumstances. Then, these sets of rules are implemented with an aim to create a closed-loop system that mimics their expertise at manual infusion [316]. It should be mentioned that even with these rules, fuzzy-logic controllers have not demonstrated suitable performance without also being combined with genetic and neural network algorithms. Using these algorithms improves the fuzzy controller by allowing adaptation in the presence of interpatient and intrapatient variability [317].
Frequently, fuzzy-logic controllers are changed to so-called self-organized fuzzy-logic controllers that are capable of adapting to changes in the system [318]. However, these controllers are known to cause steady-state error when used with bolus type therapy. To overcome this, simple fuzzylogic can be used during bolus treatment and the controller can be switched to the self-organized fuzzy-logic type once the system is operating near the desired set-point [302]. As mentioned before, genetic algorithm can then be used for optimizing the fuzzy-logic model [310].
The majority of the fuzzy-logic controllers are based on the type-1 set which cannot handle model uncertainties, and controllers that are based on type-1 sets suffer from steadystate error in the fuzzy estimation of the desired set-points. However, these problems can be handled by changing to type-2 fuzzy sets [319]. Moreover, fuzzy neural network controllers can be used to handle uncertainties in anesthesia [320]- [323], but this approach has limited effects on the type-1 fuzzy sets. So, type-2 fuzzy neural network controllers are used to solve this problem [324]. Furthermore, controllers based on type-2 fuzzy-logic sets can be improved by switching to a self-organized strategy, resulting in self-organized type-2 fuzzy-logic controllers, which can compensate for control uncertainties [2]. By adding genetic algorithms to find the best membership function for the self-organized type-2 fuzzy-logic controller, better results can be obtained in noisy (i.e., real) environments at the cost of increased error in the noise-free environment [311].
As mentioned, type-2 self-organizing fuzzy-logic controllers are effective. However, signal noise and dynamic uncertainties -such as changes in the PD and PK systems -can degrade controller performance [325]. In most cases, merging fuzzylogic controllers with other control schemes can ameliorate the final results. For instance, hybridization of fuzzy-logic with MPC can help to predict the effects of uncertainties in system [326], [327].

D. Outlook for Automated Anesthesia
As mentioned previously, there are four different types of controllers that are commonly used in the closed-loop control of anesthesia: proportional-integral-derivative, model predictive, adaptive, and fuzzy-logic controllers. Besides these controllers, strategies such as observer-controllers [213], robust and robust deadbeat controllers [220], [263], [328], nonlinear H-infinity controllers [329], non-overshooting tracking controller [330], sliding mode controllers [331], [332], and other types of nonlinear controllers [333] have been studied for use in anesthesia. Some of these studies have employed performance metrics such as settling-time, overshoot, undershoot, lowest observed BIS value (BIS-NADIR), performance error (PE), median performance error (MDPE), median absolute performance error (MDAPE), total variation (TV), integrated absolute error (IAE), and WOBBLE (an index of response variations over time) to compare their results and controller effectiveness to other control strategies in attempts to demonstrate advantages over other schemes and find the most suitable class of controllers for anesthesia [272], [334]- [336].
From a control systems perspective, most of the controllers discussed have shown adequate performance. Additionally, some important studies recently have been done regarding safety systems for closed-loop of anesthesia [337], [338]. However, due to the safety critical nature of anesthesia and consequently restrictive laws, from the clinical point of view, no controller has been completely acceptable for use in a fully closed-loop anesthesia system yet. In fact, none of the four controllers discussed are by themselves able to overcome the complex problem of anesthesia due to the presence of intraand inter-patient variability, surgical disturbances, and nonlinear dynamics. As recent papers indicate [326], [327], [339], combinations of these controllers -optimized to leverage their individual strengths -show promise for improving the performance of closed-loop anesthesia and achieving acceptable simulation results.
The development of automated anesthesia systems capable of achieving broad international regulatory approval may benefit from improved physiological models designed to include the interactions of more organ systems, physiological parameters, and clinical signs for an appropriately diverse cohort of subjects and surgical scenarios. If such a model further included all three modes of anesthesia, simulation results might provide a higher degree of confidence in the safety and performance of control algorithms at an early stage. Concurrently, it is also apparent that there is a need for control relevant anesthesia models with reduced model complexity while maintaining the ability to capture intra-and inter-patient variability. The development of such models will require collaboration between clinicians, anesthesiologists, mathematicians, and control engineers.
Furthermore, automated anesthesia would also benefit from developments in clinical monitoring aimed at achieving significant reductions in measurement noise and time-delays as well as the realization of methods to quantify specific aspects of anesthesia. To this end, quantification of the depth of anesthesia may provide interesting opportunities to investigate the use of sensor fusion to develop reliable metrics based on multimodal (and perhaps redundant) monitoring. Since the nonlinear pharmacodynamics of anesthesia are difficult to model, the development of methods to measure the drug concentration in the effect compartment may also help to remove the need for the corresponding nonlinear model equations and introduce a new set-point to complement or replace BIS values. Again considering the shortcomings of current monitoring technologies, it appears that fully autonomous (unsupervised) anesthesia should not be employed, until a reliable real-time measure of DoA is discovered. Under further consideration of the rapidly developing pharmacological and medical fields, the challenge presented by possible as-of-yet unknown drug interactions and patient variability (with respect to DoA metrics) may, in general, provide sufficient cause to opt for moderated levels of automation (LOA) in PCLC anesthesia devices (i.e., always incorporating the anesthesiologist in a supervisory role).
Moving forward, research into the optimal combination of control strategies has the potential to introduce controllers that are feasible for real-world use. Of course, any commercial controller for anesthesia should be able to perform properly in the presence of infusion limitations (e.g., the amount, rate, and frequency of drug infusion), intra-and inter-patient variability, noise, disturbances, and nonlinearities, while also meeting standards for safety and regulatory approval.

V. DISCUSSION AND FUTURE OF PHYSIOLOGICAL CLOSED-LOOP CONTROL
Considering the historical and current investigations in PCLCs, including the artificial pancreas and automated anesthesia, there are several common issues that must be handled when implementing control on physiological systems. Chief among these are the ever-present intra-and inter-patient variations that come with age, health, genetics, hormones, medications, physical activity, and a variety of other inherent and environmental factors. The combination of this variability, its many pathways, and the complex nature of physiology leads to challenges in both controller design and system validation. The presence of interpatient variability, or differences in the responses observed between different patients, introduces the need for controller robustness and a large collection of high quality clinical data for validation. However, even with a large collection of data, there remains a question of patient dynamics that are not represented by the data or have not yet been encountered. Furthermore, the presence of intrapatient variability, or changes to the responses observed within a single patient, introduce time-varying dynamics which may further introduce either parametric or structural uncertainties. The overwhelming potential for plant-model mismatch has led to a wide acceptance of adaptive techniques in PCLC formulations, used in combination with robust or predictive control strategies.
Another key issue in PCLC is the presence of strict constraints on therapy, which often allow only positive valued control actions (e.g., substances like propofol and insulinanalogs cannot be removed once injected) and limit both the rates and amplitudes of applied treatments and estimated physiological states. Of course, one of the most important methods in designing systems that combine these requirements with adaptation and robustness are those based on predictive control approaches. Then, the application of prediction to physiological control places a significant burden on the observers and estimators to provide reliable approximations of constrained physiological quantities. While there are arguments for other control methodologies, predictive control methods have been gaining momentum in PCLC and are likely to continue as mathematical models of physiological and pharmaceutical systems develop.
Changing our focus to a different portion of the feedback loop, the ability to obtain accurate and on-time measurements is of utmost importance for both safety and performance. Thus, the development and availability of improved sensing technologies will, in part, dictate the success of PCLCs. Currently, direct access to many measurable physiological states requires highly invasive approaches, which often cannot be justified for long-term or ambulatory use. Thus, less accurate, and possibly delayed, measurements are being used as surrogates, for instance in diabetes, which may impose significant performance limitations. While weighing the risks of trade-offs in measurement quality vs. invasiveness provides a short-term solution in some fields, there is an immediate need for improved and miniaturized measurement techniques, perhaps by translation from other fields, that provide rapid and accurate results without jeopardizing patient comfort or health. If we consider the case of long-term PCLC solutions, worn by patients for months or years or in free-living conditions, other issues arise as well. For instance, many electrochemical sensors limited lifespans. This case is evident in the CGM systems used for artificial pancreas systems, which last at most around 2 weeks prior to replacement and may provide degraded performance at the early and late stages of use. Additional concerns with long-term use include irritation or infections at attachment sites. Further, the unrestricted environments presented by free-living conditions may lead to cases of unforeseen sensor failures or errors, similar to the effects of physical activity on subcutaneous glucose measurements, wherein physiological redistribution of glucose during exercise may bias subcutaneous measurements. On the contrary, the FDA recently approved the first fully implantable glucose sensor, which has a 90-day lifespan and operates on fluorescent sensing technology [340]; new technologies such as this may provide solutions to some of the classical issues with GCM.
Finally, the potential for interactions between multiple drugs or between drugs and sensing elements, as in the case of Tylenol and some CGM systems, should be considered, to the extent possible, in future investigations. As PCLCs mature and data becomes available, it may be possible to implement such interactions within the in-silico models used for validation.
These models have already vastly improved the rate of controller development in the field of diabetes control and been successful in many other areas as well [341]. The addition of PK-PD models for multiple drugs and known drug interactions within these models may help identify risks or possible failure modes in ambulatory use.
Relaxing, slightly, the definition of physiological control to include systems with a primary goal of achieving a target drug concentration (rather than physiological effect), there is additional growing interest in applying control theory to regulate the concentration of pharmaceuticals within certain organs/tissues in the human body [285], [342]. While dosage control alone may reduce the effects of variability in pharmacokinetics and indirectly improve regulation of physiological variables, it is also possible that such dosage control could be incorporated as a target in future MIMO physiological control systems. Of particular note in this adjacent field are the recent success of feedback control using electrochemical-aptamerbased (E-AB) sensors [343]) and the development of longacting bioresponsive pharmaceuticals [344], which implement a form of analog closed-loop control whereby tailored drugs remain inactive in the body until stimulated by biological feedback conditions.
Modern interest in physiological and pharmaceutical control have helped bring about the development of a new field distinguished as mathematical pharmacology [345], which focuses specifically on modeling the pharmacokinetic and pharmacodynamic properties of substances. Future work in this field is likely to provide direct support to advanced automation in medical practice in the form of new and refined pharmacological models as well as insights into the classes of models [346] (and therefore control paradigms) that are well matched to a given drug-dosage or physiological control problem. Furthermore, the integration of mathematical pharmacology with systems physiology has the potential to entrust modeling and control design activities with the respective experts in each area and alleviate the present bottleneck wherein control design and validation is faced with significant uncertainty in part due to the complexity of physiological and pharmaceutical interactions.
Clearly, the development of mathematical models for control design [347] and evaluation remain important areas of research in physiological closed-loop control. Further, while we have mentioned that some methods of control do not require an explicit mathematical model, these methods may be difficult to evaluate in a computational setting, and considering the risks involved with physiological systems the immediate future of such methods in PCLC -especially with regard to regulatory approval -is unclear [9]. Of course, the effectiveness of model-based control is itself subject to the appropriateness of the model(s) used for controller design [347], and a recent review [348] has shown that many published methods fall short of completely demonstrating their suitability to the proposed use (e.g., controller design, closed-loop performance evaluation, hardware-in-the-loop testing) in PCLC devices. This problem is exacerbated by the limited data available to many researchers as well as the extraordinary range of potential disturbances, interactions, and variability that affect physiological systems. Ongoing work in the area of credibility assessment for computational patient models [348], [349] and the recently published V&V 40 standard for "assessing credibility of computational modeling" for medical devices [350] from the American Society of Mechanical Engineers (ASME) seek to provide guidance for establishing model credibility based on the specific context of use (COU), model influence in decision making (e.g., selection of infusion rate), and consequences of suboptimal decisions [348]. Within this framework, researchers are able to qualify their modeling and control results with appropriate rigor and can establish the credibility of models and techniques as new areas of PCLC emerge. This framework may also have significant value for justifying comparisons between control and safety strategies.
With all of these ongoing efforts and the promise of future developments, it seems that the broad introduction of commercially available PCLCs is now imminent, and the coming decades will likely see many breakthroughs in this regard. Thus, pertinent discussions on appropriate regulatory policy are underway [9], and similar conversations are being held with regard to ethical implementation of automation in medicine [351], [352]. The continuation of these discussions and refinement of regulatory and ethical guidelines is essential to the future of PCLCs, for establishing both methods of safe practice and guidelines for design, and also for well-informed selection and use of novel medical instruments by hospitals and physicians.

VI. CONCLUSION
In the years to come, closed-loop medical devices, artificial intelligence, and networked medical instruments [353] will begin to emerge along existing medical robotics [354], [355]; and physiological closed-loop control systems will become visible within the clinical, home-care, and ambulatory environments. Ultimately, it is possible that many of these devices will be linked to an electronic health network, which could maintain critical patient data or even allow physicians to monitor and adjust treatment goals remotely.
The integration of feedback control and artificial intelligence in medical systems has the potential to improve adherence to prescribed treatment regimens and protocols, rapidly adapt to new or changing therapy strategies, provide access to treatment in remote areas, optimize the utilization of available resources in scenarios with supply limitations, and increase the capacity of the individual clinician in disaster response and humanitarian crises. The consistency of treatment and availability of data that follows automation may be a useful tool for evaluating the performance of certain clinical practices, validating new treatment options, creating innovative therapies, and enabling the wide-spread availability of personalized medicine. These achievements, and others yet to be conceived, will not be obtained without navigating substantial challenges but appear within reach during the 21st century.