A New Adaptive Disturbance/Uncertainty Estimator Based Control Scheme For LTI Systems

This paper introduces a machine learning assisted disturbance/uncertainty estimator based control scheme. The aim of the proposed method is to update the nominal model directly used by the conventional disturbance observer based control architecture and approximate it to the perturbed/uncertain system using machine learning approaches. This enhances the disturbance rejection performance of the system remarkably. The performance deterioration capacity of lumped disturbances, which are the mixed effect of disturbances entering through the control channels and modeling uncertainties, are decomposed in our approach and handled separately. For this study, harmonic disturbance model and constant unstructured uncertainty model are considered, and $\epsilon $ -Support Vector Regression approach is used together with an online adaptation algorithm. A numerical example is given to demonstrate the merits and effectiveness of the proposed approach. Simulation results show that the proposed method outperforms the conventional disturbance/uncertainty estimator based control architecture by increasing disturbance estimation performance of the system.


I. INTRODUCTION
Disturbance/Uncertainty Estimator (D/UE) based control, or in other words, disturbance observer based control (DOBC) that compensates the external disturbances and system uncertainties is one of the efficient robust control approaches and they are frequently used in modern control systems. Numerous research outcomes have been reported on DOBC so far, which increases the robustness of the system by estimating the total difference between the nominal model and the perturbed/uncertain system without affecting the system performance, and a certain level of closed loop performance has been reached [1].
DOBC was first proposed by Ohnishi in the 1980s [2]. Following this, in the 1990s, extended state observer (ESO) was proposed by Han [3]. DOBC that rejects not only external disturbances but also unknown uncertainties has received a The associate editor coordinating the review of this manuscript and approving it for publication was Wentao Fan . great deal of attention with the active disturbance rejection control (ADRC) including extended state observer (ESO) that was proposed by Han [4]. Simultaneously, the equivalent input disturbance (EID) approach was presented and discussed from a theoretical standpoint, and applied to many practical systems [5], [6]. In [7], the two-degrees-of-freedom nature of UDE-based controllers was presented and compared with time delay control (TDC). The authors of [8] presented a robust autopilot design including a newly proposed time domain disturbance observer approach for bank-to-turn missiles. In [9] and [10], an output error based D/U estimator based control scheme is proposed and the method is applied to a high precision gimbal control system and a pan-tilt system. Robust stability, performance and bandwidth requirements of newly proposed scheme is derived. The works in [11], [12], and [1] present a comprehensive explanation of the studies on this subject. Especially, [1] and [12] provide a broad perspective and analysis about DOBC from the past to the present.  [9].
The most DOBC structures reported in the literature generally assume the existence of an equivalent input disturbance on the control input and estimate mixed effect of disturbance and uncertainty as a lumped signal, [13]. Utilizing the disturbance observer's dynamical description, it becomes nearly impossible to figure out how much of the lumped D/U estimations are associated to the disturbance and how much is associated to uncertainty. This sets up our motivation. We propose a new adaptive method based on machine learning approaches that increases disturbance estimation performance by approximating to the amount of system uncertainty. To our best knowledge, unmixing the lumped disturbances via an adaptive scheme is first attempted in the current study. Adaptive DOBC structures in the literature include generally composite controller design, data driven and nonlinear controller based augmented structures [14], [15], [16], [17]. In [14], a novel control scheme combining nonlinear DOBC with H ∞ control structure was presented for complex multipleinput-multiple-output (MIMO) flight control system. The authors in [15] designed an adaptive multi-variable finitetime disturbance observer (FDO) to estimate model uncertainties, external disturbances, and actuator faults for reusable launch vehicles (RLV). For piezoelectric ultrasonic actuator (PUA)-based surgical device, an enhanced adaptive robust DOBC scheme including sliding mode was proposed in [16]. In [17], a data driven disturbance observer based control scheme including ADRC approach is discussed. However, the cited body of literature estimates the lumped D/U and remedies are based on the lumped effect of the disturbances and plant uncertainties.
The lumped estimation uses the difference between the nominal model and the perturbed/uncertain plant. Our purpose is to update the nominal model iteratively to match its response to that of the perturbed/uncertain system by using machine learning approaches thereby leading to an improvement in the disturbance rejection performance of the system. The proposed method is applicable to all DOBC schemes that exploit the nominal plant information. In order to exemplify the efficacy of the proposed technique, we use the algorithm proposed in [9]. This paper advances the subject area towards decomposition algorithms that handle the adverse effects of input disturbances and plant uncertainties separately. Machine learning offers a framework based on numerical data & optimization algorithms and we exploit the observed quantities towards unmixing a mixed signal in a feedback control framework.
The contribution of the current study is to postulate an algorithm for handling the input disturbances by adaptively modifying the nominal plant dynamics.
The remainder of this paper is organized as follows. Section II handles the conventional D/UE based control scheme and the proposed method. The third section presents a numerical example and set of simulation studies to show the effectiveness of the proposed method. Finally, concluding remarks are presented.

II. METHODOLOGY
A. DISTURBANCE/UNCERTAINTY ESTIMATOR BASED CONTROL SCHEME For an LTI system, the general equivalent input disturbance representation of it can be given aṡ Fig. 1 illustrates the disturbance/uncertainty based control scheme proposed in [9], where, K is the main controller, u(t) ∈ R is the output of the main controller, η(t) ∈ R is the observer error, K obs is the observer controller, d(t) ∈ R is the equivalent input disturbance andd(t) ∈ R is the mixed estimations of disturbance/uncertainty. The perturbed plantP is as follows:P where P, W T and are the nominal plant, robustness weight function and unstructured uncertainty function, respectively.  The transfer function of the nominal plant (P) is given as below.
B. PROPOSED SCHEME Controller design procedures of K and K obs in Fig. 1 are given in [9] and [10] for equivalent LTI systems (1). Robustness figures of the closed loop system can be generated for designed K and K obs using the following co-sensitivity and sensitivity expressions.
After designing the main controller and observer controller structures, we implemented an adaptive method using support vector machine approach, which is a powerful machine learning technique for regression and classification problems, presented in Figs. 2-4. While Figs. 2-3 illustrate the learning phase of proposed scheme, Fig. 4 shows the overall online adaptation scheme after learning phase. According to the figure, one understands that the adaptive scheme matches the plant uncertainty iteratively using ML techniques and the new nominal system is used in the lowest disturbance prediction loop to cancel out the disturbance d.
Supervised learning process in the block diagram presented in Fig. 2 consists of four steps and is depicted in Fig. 3.
Receiving and saving data periodically constitute the first step of this process. An important issue in this step is to save datasets that contain as much variation as possible in the timedomain using different disturbance and uncertainty models. This is critically important to distinguish the components of a mixed signal. In this paper, we consider harmonic disturbance model and constant unstructured uncertainty model, i.e. | | ≤ 1, ∈ R. Time-domain sinusoidal disturbance model is defined as For constant unstructured uncertainty model, while weight function W T in (2) is i th-order transfer function with poles and zeros in a Butterworth pattern to meet the specified gain constraints, ∈ ( min , max ).
The data-sets constitute the crux of the approach. We perform several experiments to collect the numerical data. In the first set, input disturbances (d(t)) are available yet there is no plant uncertainty ( ≡ 0). In the second set, we have plant uncertainty ( ) yet no disturbance (d(t) ≡ 0) in the control channel. Such a data-set describes the decoupled effect of each factor on the output signal and constitutes a labeled input to a learning agent. Each data-set contains a certain duration time-domain D/U estimation signal sampled at a certain period the system is in the steady regime. In a real scenario, the experiments without plant uncertainty might not be conducted and the best known nominal model could be used to generate the training data to execute the proposed algorithm.
In the feature extraction step, N -point Fast Fourier Transform (FFT) is computed and the FFT magnitudes are used in the sequel. For each data-set, a feature vector is created. Feature vector is an m-dimensional vector consisting of the After feature extraction step, machine learning approaches can be applied to the obtained data-sets. For the proposed ML assisted disturbance/uncertainty estimator based control scheme, we have used -Support Vector Regression ( -SVR) as the regression machine learning model. -SVR solves the following primal problem: where x i ∈ R p is training input vectors (i = 1, . . . , n), y ∈ R n is a vector containing regression (output) values and C is a penalty term. The value of defines a margin of tolerance where no penalty is enforced over errors. In the above optimization problem, φ stands for the kernel trick, [18]. The main goal is to find w ∈ R p and b ∈ R. The dual problem is as given below and it is a convex optimization problem that can be solved.
where e is a vector composed of all ones, Q is n × n positive semi-definite matrix, is the vector of coefficients of the dual problem. An in-depth treatment of support vector machines and the optimization algorithms can be found in [18] and [19]. The data-sets used for the optimization of -SVR contain samples, in which the output is zero if only uncertainty is active, one if only input disturbance is active. Input vector of the -SVR is m-dimensional feature vector given in the feature extraction step. Such a data-set structure enables us to define the boundary of disturbance-active region and uncertainty-active region in the input space and it further lets us interpolate between these regions if both disturbance and uncertainty are active and mixed at different levels. The machine learning model obtained with the minimum mean squared error (MSE) value after the training and testing processes is obtained first and it is used in the online plant adaptation process as shown in Fig. 4. Fig. 4 presents the proposed overall control scheme including online adaptation process. The main purpose is to update the nominal model iteratively to match its response to that of the perturbed/uncertain system. The adjustable nominal plant is defined as where is the estimate of . ∈ [ min , max ) ⊂ R and initial value 0 = 0. As a result, initially P(s) = P(s).
The following items describe the modules in the proposed scheme seen in Fig. 4. Capture time-domain data 15: if not found &d b is ready then (λ) related to how much of the lumped D/U estimations are associated to the disturbance and how much is associated to uncertainty for the given feature vector (F v ) by using the machine learning model that has already been obtained in the learning phase. λ ∈ [0, 1].
• Adaptation: '' Adaptation'' module updates value with δ step resolution according to the ML Model output (λ) by considering a threshold value in the range of (threshold, 1). Algorithm 1 describes the algorithmic flow of the proposed method including online adaptation processes. Step response of the nominal closed loop system.

III. NUMERICAL EXAMPLE
In order to exemplify the proposed scheme, we consider a second order LTI system, which allows the user to reproduce the results. The dynamic system in (11) represents the nominal plant transfer function of the system under consideration.
Remark 1: The plant model is chosen deliberately simple to demonstrate the goals of this study. We aim to devise an algorithm that senses the effect of the proportions of disturbance and uncertainty in an observed output variable. Choosing a more complicated (possibly nonlinear and multidimensional) model would make understanding the contributions of the current work difficult. We avoided the plant specific difficulties to discuss and unfold the algorithm-specific issues.
The main controller K is designed for the nominal plant and it is a proportional-integral-derivative (PID) controller meeting the performance criteria, i) 32 rad/s bandwidth and ii) 90 degrees phase margin. These specifications indicate that a reasonably fast response is requested. The controller K satisfying these specifications is defined as where, K p = 320, K i = 796 and K d = 32.2 are proportional, integral and derivative gains, respectively. Fig. 5 illustrates the step response of the nominal closed loop system. The perturbed plant is chosen aŝ where = 0.67 and W T (s) = 3s+5.774 s+28.87 . K and K obs can be designed together as defined in [9] and [10] by considering weighting function (W T ) defining performance requirements. For simplicity, we set K obs ≡ K . We have depicted sensitivity (S) and co-sensitivity (T ) functions in Fig. 6 by using (4) and (5). When we inspect the data in Fig 6, we see that K obs is enough to estimate and reject disturbance/uncertainty, yet one can pursue better K obs designs than the choice K obs ≡ K . Fig. 7 illustrates predicted mixed disturbance/uncertainty results for the perturbed plant given in (13) and below harmonic disturbance model is adopted.
VOLUME 10, 2022  According to Fig. 7 and its window plots, we observe that uncertainty ( ) causes steady state errors in estimating the disturbance that enter through the control channel. The response seen in the figure displays a fast transient and the steady regime is reached after almost 1 second. The window plot (a) shows the initial transient, (b) shows the predicted disturbance and (c) demonstrates the ground truth. It is evident that the presence of constant causes a constant shift in the disturbance estimations. Our goal is to improve the disturbance estimation performance by eliminating these steady state errors to approximate to the true value of d(t).
Remark 2: In a general scenario, for a ML model to distinguish the effects of input disturbances and structural uncertainties, the design engineer is expected to perform a number of tests that guide the ML model and develop a reasonable decision boundary to unmix the mutual effects. This tightly depends on the numerical data and the feature set that embodies the ML model's input vector.
In order to apply the proposed method to the D/U estimator based control scheme in Fig 1, we first need to create a data-set as described in the learning phase steps. In Table 1, the data-set features are given. A total of 1000 data-sets are created. Each data set has a size of 5-seconds timedomain D/U estimation signal sampled at 1 ms during the steady state regime of the system and is generated. Random numbers adopted here distribute uniformly over the ranges determined by the maximum and minimum values given in Table 1. Then, the feature vectors of them are created by adding the associated MAV and ZC values. Each feature vector is m = 40 dimensional vector and its first N values (N = 38) come from the single sided magnitude of calculated 4096-point FFT. The 39 th entry is the mean absolute value (MAV) and the 40 th entry is the zero crossing (ZC) value of time-domain signal. Fig. 8-11 show sample disturbance and uncertainty datasets, where the rightmost components augment the selected N -element FFT magnitude array with MAV and ZC values. While Fig. 8 demonstrates disturbance estimations and feature vectors of them for 1.25 Hz and 2.58 Hz harmonic input disturbance frequencies, Fig. 9 displays the same graphics for 3.25 Hz and 4.86 Hz harmonic input disturbance frequencies.
In Fig. 10 and 11, uncertainty estimations and feature vectors of them are given for 0.18, 0.36, 0.58 and 0.86 constant uncertainty values. In the figures, feature vectors are shown as log of magnitude. Fig. 12 illustrates the 3D principal component analysis (PCA) plot of the whole data-set. PCA analysis clearly demonstrates that the disturbance and uncertainty are separable and the usability of data-sets with the learners of machine learning approaches.
As the next learning phase step, we have imported -SVR regression model from the support vector machine (SVM) class of scikit-learn Python library, [20]. We have chosen the model parameters as SVR(kernel = 'rbf') (with default parameters) and reserved 75% of the data-sets for the training. After training process terminates, we observed that the obtained model reaches a mean squared error value of 0.00708 (MSE test ) for the testing data-set.
Remark 3: In machine learning applications, the eventual performance depends the critically on the available numerical data. As the number of observations decreases, the performance deteriorates. However, the abundance of recorded observations enables the designer to obtain an accurate model. In the current paper, the number of experiments determines the eventual performance of the SVM based machine learning model. Therefore, one may not assure absolute success or absolute failure in such applications. In our experiments, the number of training data is sufficient to show the enhancement in the overall performance. If the number of training data is increased, naturally, one should expect better performance.
For simulation test cases, the ''Data Capture'' unit receives the D/U estimation values (d) at 1 ms intervals for 15 seconds time frame and transmits the last 5 seconds of received data (d b ) to the ''Feature Extraction'' module to ensure that the steady state regime is reached.
Figs. 13-21 illustrate the simulation results. Fig. 13 and Fig. 14 (4.37πt)). In Fig. 15 and Fig. 16, ML Model outputs (λ) corresponding to are depicted for these test cases. For the first test case, we can see from Fig. 15 that the value of is correctly found above the specified threshold line. The same can be said for the second simulation test case. However, in Fig. 16, we see that the ML Model generates a result   close to the threshold line when is approximately equal to 0.13 value. From this, it can be deduced that the obtained ML Model may find wrong values when there is a mixed D/U including features close to min and f max values in the system. This problem can be called the early convergence problem. The sharp drop in ML Model output after early convergence can be used to solve this problem. In addition, increasing the data-set size and adding the new feature extraction methods will eliminate these problems. Fig. 17 shows the ML Model  outputs of a different simulation test case that produces ML Model outputs below the specified threshold line. In such a case, corresponding to the maximum value of λ is the correct value.
Figs. 18-21 illustrate the D/U estimation results for two simulation test cases. In Fig. 18 and Fig. 20, the mixed D/U estimations that are predicted by conventional D/U estimator VOLUME 10, 2022   are given. Fig. 19 and Fig. 21 show the proposed ML assisted D/UE based control simulation results. Our proposed ML assisted D/U estimator found = 0.26 for the first test case and = 0.84 for the second test case ( = 0.84). With the proposed method, the actual disturbance is estimated   over time. However, conventional D/U estimator predicts the disturbance with the steady state error due to uncertainty.
When the simulation results are examined, it is obvious that the proposed approach enhances the disturbance estimation capability of the system when compared to the classical D/UE based control scheme. Furthermore, Table 2 presents    results of 40 simulation test cases for different and disturbance values. The studied set of simulation results prove that the proposed approach outperforms the classical methods by increasing disturbance estimation performance of the system. To obtain more precise predictions, the size of the data-set can be increased and different splitting percentages for training and testing data-sets can be adopted. As in all ML applications, feeding the learning system by diverse data leads to accurate spot of the decision boundary. Enhanced input vectors may play the same role as long as the newly added features' roles are examined well.

IV. CONCLUSION
In this paper, a novel approach to unmix the disturbance and uncertainty is presented. The classical approaches reconstruct the disturbances entering through the control channels and the process is subject to the presence of plant uncertainty, which leads to the prediction of a lumped effect that do not cancel out the input disturbance totally. The approach presented here uses an adjustable nominal model and an -SVR approach to decompose the percentages of the mixture. Such an approach distinguishes the effect of disturbance and the effect of uncertainty thereby leading to precise cancellation of the input disturbances. The performance of the presented technique is subject to that of all machine learning systems, i.e. the amount of training data, chosen learner type, representational diversity of the input vector, training termination criteria and so on. The claims have been exemplified on a second order LTI system to avoid the interference of plant specific difficulties. Results demonstrate that numerical dataoriented methods can offer alternative solutions to decompose a mixed signal and treat its components separately.