On Online Adaptive Direct Data Driven Control

Based on our recent contributions on direct data driven control scheme, this paper continues to do some new research on direct data driven control, paving another way for latter future work on advanced control theory. Firstly, adaptive idea is combined with direct data driven control, one parameter adjustment mechanism is constructed to design the parameterized controller online. Secondly, to show the input-output property for the considered closed loop system, passive analysis is studied to be similar with stability. Thirdly, to validate whether the designed controller is better or not, another safety controller modular is added to achieve the designed or expected control input with the essence of model predictive control. Finally, one simulation example confirms our proposed theories. More generally, this paper studies not only the controller design and passive analysis, but also some online algorithm, such as recursive parameter identification and online subgradient descent algorithm. Furthermore, safety controller modular is firstly introduced in direct data driven control scheme.

trol, i.e. identification for control. Roughly speaking, two 23 steps are taken separately. The first step is to apply system 24 identification to identify one mathematical equation for the 25 considered plant, so the plant model is periodically updated 26 on the basis of previous estimates and new measured data-27 identification and control may be performed concurrently. identified plant model is named as the model-based control, 33 whose control performance depends on the plant model. 34 Although system identification could be aimed at deter-35 mining if the plant to be controlled is linear or nonlinear, 36 finite or infinite dimensional, and has continuous or discrete 37 event dynamics. As the mission of system identification is 38 to identify or construct one mathematical equation for the 39 considered plant through some statistical methods, so as new 40 idea was put forth in these recent years, i.e. can system identi-41 fication be applied to design the unknown controller directly? 42 It means the observed input-output data sequence are used 43 to obtain the priori knowledge of the unknown controller 44 without any intermediate system identification process, being 45 called as direct data driven control. To describe the more 46 detailed description about direct data driven control, consider 47 one closed loop system structure with unknown plant and 48 unknown controller simultaneously. That traditional model-49 based control firstly identifies the plant model, the sec-50 ondly this identified plant model is for the next controller 51 design. But for our considered direct data driven control, the 52 unknown controller is devised from the observed input-output 53 data sequence without the system identification process for 54 that unknown plant. The feasible of this new control strategy 55 is that lots of important and intrinsic information about the 56 relation between system identification theory, generally he 113 step of experiment design concerns determining which phys-114 ical quantities will be measured, how those quantities will be 115 measured, what the test conditions will be and how the system 116 being studied will be excited [15]. 117 During these recent years, the first author studies this 118 direct data driven control too, for example, the closed rela-119 tion between system identification and direct data driven 120 control [15], and data driven model predictive control [16]. 121 A new interesting subject about persistently of excitation 122 is studied again in data driven control and model predic-123 tive control.Willem's fundamental lemma from [17] gives 124 a data based parametrization of trajectories for one linear 125 time invariant system. Based on this Willem's fundamental 126 lemma, one parametrization of linear closed loop system is 127 derived to pave a way to study important controller design 128 problems [18]. Reference [19] asserts that all trajectories 129 of a linear time invariant system is obtained from a single 130 given one on the condition that a persistently of excitation. 131 One necessary and sufficient condition on the informativity 132 of data is derived for some data driven control and anal-133 ysis problems [20]. In recent years, more novel ways ar 134 explored to develop direct data driven control, for example, 135 the idea of data driven, mentioned above, is combined with 136 model predictive control to yield a new control strategy-137 data driven model predictive control. In [21], data driven 138 model predictive control is applied to design the classical PID 139 for a deterministic continuous time system. For the case of 140 switching controllers in some industries, data driven model 141 predictive control is also benefit in regulating the switching 142 rule [22]. Consider the uncertain factors exist in the closed 143 loop situation, one robust data driven model predictive control 144 is proposed to alleviate and suppress the bad effect, coming 145 from these uncertainties [23].To be convenient for the use of 146 data driven model predictive control, some existed softwares 147 are produced for researchers, such as in python package [24] 148 and in its intelligent form to control the heat treatment electric 149 furnace [25].

150
By the way, in this early year our two new contributions 151 propose new ways for direct data driven control. For exam-152 ple, paper [26] introduces an adaptive idea into direct data 153 driven control to design the forward controller and feed-154 back controller simultaneously. The parameter adjustment 155 law is modified to satisfy Lyapunov stability. Then paper 156 [27] combines model predictive control and direct data driven 157 control to form our proposed direct data driven model ref-158 erence control. Furthermore, some control performances are 159 studied in that paper, such as stability validation, synthe-160 sis analysis and application engineering for flight simula-161 tion table, etc. Moreover during the conclusions of these 162 two papers, we point out latter emphasis are concerned on 163 adaptive direct data driven control, robust direct data driven 164 control and learning direct data driven control for linear or 165 nonlinear controller, so this new paper is our ongoing work 166 about our previous contributions on direct data driven control 167 scheme. online subgradient descent algorithm is proposed to yield one 217 optimal safety controller, and some optimal analysis are also 218 given.

219
Generally, the main contributions of this new paper are 220 formulated as follows.

221
(1) Adaptive direct data driven control scheme is 222 proposed, and one parameter adjustment mechanism is 223 constructed.

224
(2) Passive analysis is studied for the input-output property, 225 being similar to the classical Lyapunov stability.

226
(3) One additional safety controller module is added in 227 the closed loop system structure, and the detailed process of 228 solving this safety controller is given. 229 Moreover, online means two aspects:

230
(1) The unknown controller parameters are identified 231 online within that parameter adjustment mechanism.

232
(2) Online subgradient descent algorithm is applied to get 233 one optimal safety controller.

234
Direct data driven control appears based on the classi-235 cal model based control, it is suited for the data science 236 period. This paper adds one adaptive mechanism and apply 237 the online subgradient descent algorithm, then its compu-238 tational complexity is increased surely. But in data sci-239 ence times, it is tolerable due to some advanced physical 240 devices.

241
This paper is organized as follows. In section 2, adaptive 242 direct data driven control scheme is proposed to design the 243 unknown controller for one closed loop system, while no 244 any information of that unknown plant. For one parame-245 terized controller, the parameter adjustment mechanism is 246 constructed with online parameter estimation. Similar to 247 the Lyapunov stability analysis, passive analysis is given 248 in section 3, where some necessary conditions are shown 249 to satisfy this input-output properties, i.e. passive property. 250 To validate whether the designed controller be appropriate, 251 one additional safety controller is added to pass through the 252 designed controller, within the case of an desired or expected 253 controller in section 4. The idea of model predictive control 254 is introduced to solve that safety controller, while combining 255 online subgradient descent algorithm. Section 5 uses one 256 numerical example to illustrate the effectiveness of our con-257 sidered online adaptive direct data driven control scheme. 258 Finally, section 6 ends the paper with final conclusion and 259 points out the next subject.

261
Data driven control is developed from the classical model 262 based control,being satisfied for the requirement of data 263 science times. it is well known that in our data science 264 times, lots of data are easily yielded from some advanced 265 physical devices or sensors. The important information about 266 the considered plant or unknown controller are all included 267 in these lots of data, so the mission of data driven control 268 is to extract these important information for the unknown 269 controller from the data. Whatever the important informa-270 tion exist in the data points explicitly or implicity, statistical 271 method, machine learning or other reinforcement learning etc 272 can be applied to extract these useful information as much as 273 possible.

274
As two system structures exist around our normal life, i.e. 275 open loop or closed loop system structure, here the consid-276 ered closed loop system structure is plotted in the following 277 Figure 1, due to its more complex than the single open loop 278 system. where in Figure 1, r(t) is the external input signal, y(t) 280 is the output signal for the whole closed loop system, 301 so prediction error equation ε(t) is that.
where {φ r (w), φ y (w)} are power spectral, corresponding to 320 external input r(t) and output signal y(t) respectively.
An online recursive algorithm is used to yield the opti-328 mal controller parameter through computing some partial 329 derivations.
Making use of above partial derivation, an online recursive 334 algorithm is formulated as follows.
where this online recursive algorithm is similar to the classi-339 cal gradient algorithm.In equation (8),θ (t + 1) andθ (t) are 340 the optimal controller parameter at time instant t + 1 and t 341 respectively.

342
Consider one parameterized controller in the closed loop 343 system structure, then the problem of designing controller is 344 transformed into one problem of parameter estimation. This 345 process of parameter estimation can be completed into one 346 adaptive mechanism, being plotted in Figure 2. where in Figure 2, three kinds of signals {r(t), u(t), y(t)} 348 are all sent to that adaptive mechanism, then optimal 349 controller parameterθ ,generated by online recursively 350 from equation (8), is substituted into that parameterized 351 controller C(z, θ).

VOLUME 10, 2022
In that online recursive algorithm,i.e. equation (8), we can 353 add one adaption gain η(t) to consider the time varying 354 property.
where above adaption gain η(t) is a time varying factor, but 357 in equation (8)  number of data point is always limit, so we relax this con-371 dition as that θ (t + 1) < 0.5. It means one approximated 372 controller parameter is used in practice.

374
After designing that controller C(z) by our proposed adaptive 375 direct data driven control scheme, stability consideration is 376 needed as unstable system is useless.

378
Consider that plant P(z) in Figure 1, define its input-output 379 product as follows. i.e. the input-output product between input u(t) and output 389 y(t) satisfies the above inequality. Due to the following relations hold.

u(t) = C(z)r(t) − C(z)P(z)u(t) − C(z)d(t)] (13) 397
It holds that substituting equation (14) into η 2 (0, t),we have Making use of the property that externa, input r(t) is indepen-406 dent of external noise d(t), it holds that Equation (19) shows input-output power spectral φ r (w) must 414 have one lower bound, while guaranteeing passive property. 415 Generally, let us consider the whole closed loop system 416 with input r(t) and output y(t), the passive property is for-417 mulated in Theorem 1.

418
Theorem 1: Consider the feedback interconnection 419 between plant P(z) and controller C(z), the input-output 420 product must satisfy that.
The proof of Theorem 1 is very easily only through some 427 basic substitution operations. To implement this controller validation process, another mod-451 ular safe controller is added in closed loop system structure, 452 plotted in Figure 3. More precisely, this additional modular wants to guarantee 465 the formal control u(t) approach to its desired control u d (t), 466 i.e. u(t) → u d (t), so the following receding horizon problem 467 is constructed.
define some vectors as follows Then that receding horizon problem is reduced to one 475 quadratic programming problem, i.e.
where inequality constraint corresponds to the safety region. 479 The following derivation result is used in equation (27).
Consider the quadratic programming problem (27) again, 488 construct its Lagrange function to be that.
where λ is Lagrange multiply.

491
Applying the optimality KKT condition to satisfy that. able v, i.e.
Then on the basis of optimization theory, our mission is to 512 solve the optimal optimization variable v * to guarantee that.
Online subgradient descent algorithm is the following recur-515 rence. 521 We always assume that V = φ and that the subgradients 522 L (v) reported by the first order oracle at point v ∈ V .

523
As subgradient operations L (v) are needed in that recursive 524 form (33), here we derive some subgradients as follows.
To demonstrate the merit of online subgradient descent algo-528 rithm, the following two propositions are given to achieve it.

533
Proof: Let w ∈ V , and 0 ≤ t ≤ 1, we have In particular Which completes the proof of the Proposition 1.

547
Proposition 2: For that subgradient descent algorithm, then 548 for every w ∈ V , we have Proof: By using above Proposition 1, we have.
Summing up inequalities over t = 1, 2, · · · N , we get Further it holds that Then we have the following upper bound in equation (43).
The above inequality shows the convergence results for the 566 online subgradient descent algorithm.

568
Consider one closed loop system structure in Figure 1, where 569 plant is The true controller C 0 (z) is one parameterized controller with 572 a sequence of orthogonal basis function, i.e.
Its parameterized form is that.
During the whole simulation process, let that exter-  identification algorithm. The final identification results, i.e. 606 controller parameters are shown in Figure 6. Comparing the controller parameter curves in Figure 6 and 608 their true controller parameters in equation (47), we see 609 although some deviations in 10 steps, but finally all parameter 610 estimations will converge to their own true values.

611
To testify the safety controller modular, the designed con-612 troller output is u(t) = C(z, θ)e(t) = C(z, θ)[r(t) − y(t)]. 613 Suddenly, we set one desired or expected controller output 614 be that u 0 (t) = C(z, η)e(t) = C(z,  Figure 7. Specifically, when the safety property 637 is considered, then the controller parameters must be changed