Direct Data Driven Scheme for UAV Flight Control

Consider one special closed loop system structure with one unknown plant and two unknown controllers,i.e. feed forward controller and feedback controllers, our missions are to identify this plant and design these two controllers without any physical principles. Based on the measured input-output data sequence, direct data driven scheme is proposed to achieve these dual goals. For the case of parameterized plant and two parameterized controllers, two kinds of unknown parameter vectors are estimated from the data sequence, and the iterative idea is applied to identify them recursively until to be convergence. In addition, controller validation modular is added to testify whether the designed controllers are satisfied, being from the idea of model validation in system identification. Then theoretical analysis on direct data driven scheme can be implemented to guarantee UAV flight control structure work well, bringing the detailed UAV flight control performance.


I. INTRODUCTION
Automatic control theory is the area of application oriented mathematics that deals with the basic principles underlying the analysis and design of control systems. To control an object means to influence its behavior so as to achieve a desired goal. In order to implement this influence, engineers build devices that incorporate various mathematical techniques. These devices range from watt's steam engine governor, designed during the English Industrial Revolution, to the sophisticated microprocessor controllers found in consumer items, such as industrial robots, airplane autopilots and unmanned aerial vehicle (UAV). The study of these devices and their interaction with the object being controller is the subject of automatic control theory. While on the one hand one wants to understand the fundamental limitations that mathematics imposes on what is achievable, irrespective of the precise technology being used, it is also true that technology may well influence the type of question to be asked and the choice of mathematical model.
Roughly speaking,there have been two main lines of works in automatic control theory, which sometimes have seemed The associate editor coordinating the review of this manuscript and approving it for publication was Engang Tian . to proceed in a very different directions but which are in fact complementary. One of these is based on the idea that a good model of the object to be controlled is available and that one wants to somehow optimize its behavior. For instance, physical principles and engineering specifications can be used in order to calculate that trajectory of a spacecraft which minimizes total travel time or fuel consumption. Two main control structures exist in automatic control theory, i.e. open loop structure and closed loop structure. Open loop structure only uses one feed forward controller to regulate the considered object and neglects the output tracking property or other important facts, so open loop structure appears to alleviate the deviation between the desired value and real or actual value. More specifically, in closed loop structure, the actual output is returned back to the input part. After subtracting their values, the deviation will drive the feedback controller to operate, until the zero deviation. Although feed forward controller and feedback controller exist in closed loop structure simultaneously, so the closed loop controller design is more complex than the open loop controller design, but it is tolerable in today's new technology development. Due to the deviation can be alleviated to achieve the perfect matching for the closed loop system, now closed loop system is widely applied in lots of engineering application, VOLUME 10, 2022 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ for example, UAV flight control, considered here. It is well known that feed forward controller and feedback controller exist in the same closed loop system, i.e. two controllers are needed to design, while satisfying some desired or expected closed loop properties. For the problem of designing the unknown controller whatever in open loop of closed loop system, two different control strategies are used, i.e, modelbased control and direct data driven scheme. Model-based control firstly constructs one mathematical model for the considered plant, then uses this mathematical model to devise the unknown controller. On the contrary, direct data driven scheme only applies the measured data set to identify the considered plant or design the unknown controller. It means not only the unknown plant, but also the unknown controller are all yielded from the measured data set through some statistical methods or data mining. They correspond to the system identification and controller design respectively, where the numerical optimization theory is combined with identification and control.
To show the closed relations among identification, control and optimization, here one example of our ongoing studied direct data driven control is used to illustrate the importance of the optimization theory. As the application of direct data driven approach widely in control field, and the similar point between direct data driven approach and system identification, we call their combination as identification for control, i.e. system identification for direct data driven control. More specifically, we describe a concise introduction or contribution on system identification for direct data driven control, which belongs to data driven approach. In case of the unknown but bounded noise, one bounded error identification is proposed to identify the unknown systems with time varying parameters. Then one feasible parameter set is constructed to include the unknown parameter with a given probability level. In [1], the feasible parameter set is replaced by one confidence interval, as this confidence interval can accurately describe the actual probability that the future predictor will fall into the constructed confidence interval. The problem about how to construct this confidence interval is solved by a linear approximation/programming approach [2], which can identify the unknown parameter only for linear regression model. According to the obtained feasible parameter set or confidence interval, the midpoint or center can be deemed as the final parameter estimation, further a unified framework for solving the center of the confidence interval is modified to satisfy the robustness. This robustness corresponds to other external noises, such as outlier, unmeasured disturbance [3]. The above mentioned identification strategy, used to construct one set or interval for unknown parameter, is called as set membership identification, dealing with the unknown but bounded noise. There are two kinds of descriptions on external noise, one is probabilistic description, the other is deterministic description, corresponding to the unknown but bounded noise here [4]. For the probabilistic description on external noise,the noise is always assumed to be one white noise, and its probabilistic density function (PDF) is known in advance. On the contrary for deterministic description on external noise, the only information about noise is bound, so this deterministic description can relax the strict assumption on probabilistic description. In reality or practice, bounded noise is more common than white noise. Within the deterministic description on external noise, set membership identification is adjusted to design controllers with two degrees of freedom [5], it corresponds to direct data driven control or set membership control. Set membership control is applied to design feedback control in a closed loop system with nonlinear system in [6], where the considered system is identified by set membership identification, and the obtained system parameter will be benefit for the prediction output. After substituting the obtained system parameter into the prediction output to construct one cost function, reference [7] takes the derivative of the above cost function with respect to control input to achieve one optimal input. Set membership identification can be not only applied in MC, but also in stochastic adaptive control [8], where a learning theory -kernel is introduced to achieve the approximation for nonlinear function or system. Based on the bounded noise, many parameters are also included in known intervals in prior, then robust optimal control with adjustable uncertainty sets are studied in [9], where robust optimization is introduced to consider uncertain noise and uncertain parameter simultaneously.
To solve the expectation operation with dependence on the uncertainty, sample size of random convex programs is considered to replace the expectation by finite sum [10]. Generally, many practical problems in systems and control, such as controller synthesis and state estimation, are often formulated as optimization problems [11]. In many cases the cost function incorporates variables that are used to model uncertainty, in addition to optimization variables, and reference [12] employs uncertainty described as probabilistic variables. Reference [13] studies data driven output feedback controllers for nonlinear system, and applies event triggered mode to analyze the robust stability. Data driven estimation is used to achieve hybrid system identification [14], whose nonlinearity is described by one kernel function. During these recent years, reference [15] studies this direct data driven control too, for example, the closed relation between system identification and direct data driven control, and data driven model predictive control [16]. Based on above descriptions on direct data driven control and our existed research about system identification, model predictive control, direct data driven control and convex optimization theory etc, our mission in this paper is to combine our previous results and apply them in practical engineering. During these two years, a new interesting subject about persistently of excitation is studied again in data driven control and model predictive control.Willem's fundamental lemma from [17] gives a data based parametrization of trajectories for one linear time invariant system. Based on this Willem's fundamental lemma, one parametrization of linear closed loop system is derived to pave a way to study important controller design problems [18].
During these recent years, we concentrate on this research about direct data driven control scheme deeply. For example, two special cases of direct data direct control are studied,i.e. iterative correlation tuning control [19] and virtual reference feedback tuning control [20]. From our previous contributions, we introduce lots of interesting topics from system identification into direct data driven control strategy. Specifically, model structure validation for closed loop system identification [21] is modified to yield the iterative correlation tuning control. Adaptive idea is combined with iterative correlation tuning to generate an innovative way to design two unknown parameterized controllers [22]. From a point of engineering application, direct data driven control and model predictive control are combined for flight simulation table [23], where a new concept-direct data driven model reference control is proposed recently. Generally, data driven strategy includes some detailed ideas, for example data driven control, data driven identification, data driven fault detection, data driven leaning etc. Date driven strategy means to extract some useful information from data directly. In this paper, our direct data driven control is to use data to get the controller.
Based on above concise description and our previous contributions on direct data driven control scheme, this new paper shows our new mathematical derivations on this innovative direct data driven control from two points of theoretical analysis and engineering application. For the sake of completeness, one closed loop system structure is considered with two unknown controllers, i.e. feedback controller and feed forward controller. It means there are three unknown parts, existing in the considered closed loop system. As the mission for system identification is to construct the mathematical model for that unknown plant, but to design the controllers for direct data driven control, so this paper achieves this dual missions simultaneously,i.e. identifying the unknown plant and designing the two unknown controllers. The reason about why these dual missions can be achieved lies that they all are dependent on the measured input-output sequence. In addition, iterative idea is proposed to identify the unknown plant and devise those two unknown controllers, until the final plant and controllers are satisfied. In order to measure this level of satisfactory, model structure validation from system identification is introduced to check controller validation. The ideal case for controller validation is to guarantee the correlation function between the prediction input error and prediction output error be as small as possible, corresponding to another direct data driven control strategy-iterative correlation tuning control. In order to better understand the considered direct data driven control scheme, it is applied into UAV flight control system,whose detailed system structure is described. After explaining UAV flight control system as the same with our considered closed loop system with one unknown plant and two unknown controllers, i.e. feed forward controller and feedback controller, lots of processes about applying direct data driven control and simulation results are given to prove the efficiency.
The main contributions of this new paper are formulated as follows.
(1) To implement the dual missions for identification and controller design from the measured input-output data sequence, direct data driven control strategy is proposed iteratively.
(2) To check whether the designed controllers are satisfied, controller validation is added to guarantee the correlation function be as small as possible.
(3) UAV flight control structure is introduced, and the detailed application about this new direct data driven control is shown.
This new paper is organized as follows. In section 2, the considered closed loop system with two unknown controllers and unknown plant is described in a picture. Furthermore, some background about direct data driven control scheme is shown for the sake of completeness. Then direct data driven control is yielded to identify the unknown plant and design the two unknown controllers simultaneously, through the measured input-output data sequence directly in section 3, where iterative idea is also used. To check whether the designed controllers are satisfied, section 4 gives one additional controller validation, being similar to the model structure validation in system identification theory. From a practical point of view, UAV flight control structure is reviewed in section 5, and some simulation examples are given to show our considered direct data driven control scheme. Finally, conclusions and future work are presented in section 6. Roughly speaking,this new paper proposes iterative direct data driven control for one special complex close loop system structure, being similar to UAV flight control structure. Further, controller validation is added to testify whether the designed controllers are satisfied. By the way it is the first time to introduce direct data driven control scheme into UAV flight control modular.

II. CLOSED LOOP SYSTEM STRUCTURE
In order to better understand the direct data driven control scheme for UAV flight control system, firstly our new contributions on direct data driven control are given, then secondly its application is exemplified in detail.
Consider the following closed loop system structure, plotted in Figure 1. where in Figure 1,r(t) and y(t) are the reference signal and the observed output of the plant P(z), C 1 (z) and C 2 (z) are two transfer functions of the two components of the two degree of freedom regulators, i.e. feed forward controller and VOLUME 10, 2022 feedback controller respectively. External noise v(t) may be one statistical noise or unknown but bounded noise, u(t) is the input signal for the plant P(z). z is the backward shift operator.
Observing Figure 1 again, three elements or components {P(z), C 1 (z), C 2 (z)} are all unknown. The goal of direct data driven control strategy is to collect the input-output sequence to give the estimations for these three elements {P(z),Ĉ 1 (z),Ĉ 2 (z)} respectively, where N is the total number of the measured data. It means the plant estimation and two controller estimations are all extracted from the measured data sequence.
From Figure 1, it is easy to get some existed equations, such as i.e. or The above equations (2) and (3) will be benefit for later theoretical analysis.

III. DIRECT DATA DRIVEN CONTROL
To show our considered direct data driven control, we emphasis that our goal is to apply the measured data sequence {r(t), y(t)} N t=1 to identify that unknown plant P(z) and two unknown controllers {C 1 (z), C 2 (z)}. Here we also introduce two unknown parameter vectors {θ, η}, being for parameterizing the plant and controllers, i.e. {P(z, θ), C 1 (z, η), C 2 (z, η)}. This kind of parameterized forms always exist in the transfer function form, for example, PID controller. Then the parameterized closed loop system structure is re-plotted in Figure 2. Based on the above parameterized forms {P(z, θ), C 1 (z, η), C 2 (z, η)}, the error function ε(t, θ, η) is defined as follows.
where e jw is the frequent variable, and {φ r (w), φ v (w)} correspond to the power spectral for external input signal r(t) and external noise v(t) respectively.
(θ, η) = arg min (θ,η) (7) where equation (5) is used in equation (7). Observing equation (7), the unknown parameter vector θ exists only in P(e jw , θ), but the other unknown parameter vector η is only for two transfer functions {W 1 (e jw , η), W 2 (e jw , η)} simultaneously. Consider the optimization problem (7) with two unknown parameter vector {θ, η} as the decision variables, the iterative strategy is applied to be the iterative identification and control scheme, whose main steps are formulated as follows.
With those unconstrain optimization problems,the special optimization algorithm is the commonly used negative gradient algorithm. To guarantee the iterative convergence of the negative gradient algorithm, the cost function must be one convex function. If it does not hold, then the other improved online subgradient descent algorithm can be applied by adding one projection operation to pull back the iterative value near its global optimum at each step.
Step k: Generate the kth parameter estimations {θ k , η k } then combine them together as one sequence, i.e.
where . is one defined norm.
If inequality (8) holds, then terminate the above iterative steps, or go to step 2 and step 3 recursively. After substituting the kth parameter estimations {θ k , η k } into their own parameterized forms, then the plant and two controllers are generated as {P(z, θ k ), C 1 (z, η k ), C 2 (z, η k )}.

IV. CONTROLLER VALIDATION
As in open loop identification, it is the model validation that will tell us on one hand if the identified plant is acceptable and on the other hand it will allow us to select the best plant among the plants provided by various identification methods. The goal of the model validation in closed loop is to find what plant model combined with the current controller provided the best prediction of the behavior of the closed loop system. Generally, the model validation in closed loop system will depend upon the controller, being used. In this section, we extend the model validation from system identification to controller validation.
The idea about controller validation considers the residual prediction error between the output of the plant, operating in closed loop and the output of the closed loop prediction. An uncorrelation test is used to compute the correlations between the residual closed loop output error and the components of the predictor regressor vector. This type of test is motivated by the fact that uncorrelation between the observations and the closed loop prediction error leads to unbiased parameter estimates. This uncorrelation implies the uncorrelation between the closed loop output error and the closed loop input error be zero or as small as possible. But our controller validation is to compute the uncorrelation between the prediction output error and prediction input error, whose detailed computation processes are listed as follows.
From Figure 1, we have the input-output equations as that Their one step ahead prediction output and prediction input are given respectively.
So their corresponding prediction output error and prediction input error are Controller validation guarantees the above error be uncorrelated, i.e.
substituting all above spectral (14) into the expectation equation (13), we have Combining above mathematical derivations together to get Equation (16) means the correlation function between the prediction output error and prediction input error is one constant, i.e. φ v (w), based on our designed two controllers through the proposed direct data driven control scheme. IF noise power spectral φ v (w) is chosen sufficiently small, then that correlation function will converge to zero. Comment:The added controller validation process does not exist in practice, but only for theoretical analysis for our direct data driven control scheme. Two controllers must be designed to guarantee the expected closed loop property. This considered closed loop property is to let the two errors be zero, but zero is one ideal case, as it can not be achieved, so one tradeoff is to make the error as small as possible during tolerable range. The closed loop system with above added controller validation is seen in Figure 3.
Based on the obtained prediction input error ε 1 (t) and prediction output error ε 2 (t), another direct data driven control is yielded,i.e. designing the controller parameter and plant parameter to satisfy the following four equities.
The above direct data driven control is called as iterative feedback tuning control, which devices the parameters as follows.
(θ, η) = arg min (θ,η) 1 N N t=1 ε 1 (t)y(t) (18) Due to space limit, the detailed process about iterative feedback tuning control is neglected here. The interested reader can refer to our previous paper. Comment: Combining above two contents, i.e. direct data driven control and controller validation, they have one same property, i.e. extracting some useful information from the observed data sequence, so data are very important for them. The detailed process of dealing with data is described as follows.Firstlysome physical devices or sensors are installed around the considered system or plant, then we use them to collect lots of data, whatever input and output data. These data are divided into two groups for direct data driven control and controller validation respectively. When to consider the first direct data driven control, we describe each pair of input-output data in one same plane, then our proposed idea is applied to design the controller parameters in order to fit the input-output data.

V. UAV FLIGHT CONTROL
To prove our proposed direct data driven control strategy and its controller validation process, UAV flight control structure is suited for our engineering application.

A. BACKGROUND ABOUT UAV
Unmanned aerial vehicles (UAV) is an unmanned aerial vehicle that can take off and land vertically,being operated by radio ground remote control or autonomously control system. As UAV can complete all the flight actions, such as vertical ascending and descending,long-term hovering, turning, and their maneuverability, it is more widely used and studied during all over the countries. UAVs,also known as aerial robot, is an aircraft with some nice properties, such as remote control, automatic, semi-autonomous and fully autonomous flight capabilities. Due to UAV's simple structure, low cost, good operation, strongly adaptive environment and load lots of equipment, research on UAV is becoming vast. Specifically, with the development of intelligent technology, sensor technology, microelectronics and digital communication, etc, some problems, existing for many years have been gradually overcome. For the aspect of military, UAV is often applied to perform tasks such as search and reconnaissance, battlefield situation monitoring and target tracking or detection. Some kinds of products of UAV have been in world war, and the more famous are listed as follows, American global hawk, predator, fire scout, etc. So far, 55 countries have been equipped with UAVm it means UAV has nice development prospects in both the military and civilian fields.
According to the flight mission requirements of UAV, the flight process of UAV can be divided into several different flight control modes. During the whole flight, UAV receives the remote control command from the ground and analyzes the command, so that UAV is in the corresponding flight control mode. The commonly used flight control modes and descriptions of UAV are as follows: (1) Manual, (2) Automatic, (3) Heading hold, (4) Altitude hold, (5) Elevation rate control, (6) Hovering, (7) Forward flight, (8) Autonomous 1, (9) Autonomous 2, (10) Return.

B. UAV FLIGHT CONTROL STRUCTURE
As UAV is a multi variable coupling system, so it is difficult to establish an accurate nonlinear mathematical model for UAV flight control system. Here our considered flight control system is called dual machine simulation in the circuit simulation, and it is also a physical object or a test device. The circuit structure of the flight control system is shown in Figure 4. where in Figure 4, the flight control system includes the simulation computer, the measurement computer and the flight control computer together within one closed-loop system. Each part in that closed loop system is connected through a digital interface. After the flight control computer receives the flight parameters from the simulation computer, it calculates the control value of the rudder surface according to the control law, and transmits it to the simulation computer through the digital interface. After the parameters are sent to the measurement computer for display monitoring, then the measurement and control computer will also send necessary continuous or discrete instructions to the simulator or flight control machine. The flight control computer in-loop simulation is a real-time simulation, whose main purpose is to evaluate the correctness and performance.
Observing Figure 4 again, the characteristics of the sensor in the closed loop system include the flight parameters being calculated by the mathematical model. These flight parameters can be measured by sensors, such as inertial navigation, gyroscope, GPS, atmospheric computer, etc. The sensor loop can be seen in Figure 5. where in Figure 5, after receiving the instructions from the measurement computer, the simulation computer performs the model calculation, and sends the flight parameters required to the control computer. Similarly, the physical quantity is then sent to the flight control computer, which sends the control quantity to the simulation computer, according to the control law. From this we can see UAV is not only the carrier of the sensor but also the exciter of the sensor.
The steering gear can affect the flight aerodynamics and control the flight attitude of UAV. The servo loop is formed by adding a servo system to the flight control computer loop. The structure diagram is shown in Figure 6. where in Figure 6, during the simulation of the steering gear in the closed loop system, after the flight control computer receives the instruction from the measurement and control computer, it no longer directly transmits the control to the simulation computer, but transmits it to the steering gear to drive its movement, and measures the angular displacement of the steering gear. Then the linear displacement signal is sent to the simulation computer. After the simulation computer receives the output of the steering gear, the current flight parameters are calculated by the mathematical model, and then sent to the flight control computer for the calculation of the control law at the next moment.

C. NUMERICAL SIMULATION
During this numerical simulation, the steering gear measuring device adopts the photoelectric encoder to detect the actual position of the steering gear in real time. Through the mechanism rod system or the coaxial installation scheme, the connection between the measuring element and the electric steering gear is realized, and the steering gear is completed when the steering gear is working normally. After the photoelectric encoder reads out the rotation angle, corresponding to the displacement of the steering gear, it converts the incremental pulse signal into the steering angle value through the data conditioning module, and then outputs the data to the simulation computer through the interface to form the control parameters.
The loader adopts an elastic torsion spring as the steering gear. It is a passive electric loading device and has the characteristics of fast response, small size, convenient control and high precision. There are two types of typical force loaders, as shown in Figure 7. The force loader system used here is composed of electric actuation device and control system. Among them, the electric actuation device is composed of six parts, for example, motor, reduction box, gear transmission pair, ball screw, linear potentiometer and push-pull force sensor, which can realize the linear motion of the output rod. The control system of the force loader is composed of speed measuring machine, power amplifier, displacement sensor, control computer, electrical control circuit and interface circuit. More specifically, the driving element of the force loader is an electric motor, which belongs to a high-precision control system. The control system is composed of a current loop, a speed loop, a position loop and a force loop. The overall control loop is shown in Figure 8. From above analysis of the steering gear force loading device, we see that there are many control parameters, and the trial-and-error method has strong randomness, so it is not convenient to design the optimal control parameters. Therefore, in order to solve these problems, this section uses our proposed direct data driven strategy to identify the optimal tuning of control parameters. The position control loop of the steering gear force loader can be represented by Figure 9. As Figure 8 is one three-loop closed structure, and its inner loop is extracted in Figure 9. Consider Figure 9, the steering gear force loader is regarded as plant P(z), and two additional controllers {C 1 , C 2 } are deemed as feed forward controller and feedback controller. During the later numerical example, the plant is chosen as.
Before to do simulation, external noise v(t) is one white noise with zero mean and unit variance. Input-output data sequence {r(t), y(t)} N t=1 are collected within the considered closed loop environment, where the total number of data sequence N = 1000. External reference signal r(t) is used to excite the whole closed loop system, whose excitation signal r(t) is the sine wane, and the excitation way is yielded one by one. Due to the chosen parameterized controllers, the controller design problem is to identify these unknown parameter vectors {θ, η}, being estimated by our proposed iterative identification with initial parameter values 0.02I . The identification results are shown in Figure 10 and 11, which show all parameter estimation curves. From Figure 10-11, we see all parameter estimations converge to their real or true values although some deviations exist. The reason about deviation is that the chosen while noise is not appropriate, as in reality white noise is an ideal case, so we think to use unknown but bounded noise to testify above identification accuracy in next paper.
Observing Figure 10 and 11 again, ten controller parameters, existing in the feed forward controller and feedback controller respectively, are needed to design through our proposed direct data driven strategy. Ten initial values are chosen around two constant vales, i.e.0.1 or -0.02, then the iterative algorithm is used to generate a sequence for each controller parameter. After connecting all parameter sequence for each controller parameter, ten curves are obtained. From these ten curves, we see all ten curves will converge to their own true parameter values after 15 iteration steps, although some deviations appear within these 15 iteration steps.
For completeness, parameter estimation for unknown but bounded noise is different with parameter estimation results in Figure 11, where parameter estimation converges to its real or true value,i.e. one constant value. But for bounded identification, the identification result is not a constant value, but one confidence interval, including the parameter estimation with a desired probabilistic level. For example. take zonotope parameter identification from [24] to design the controller parameters, two confidence intervals, showing in Figure 12, include the controller parameters 0.13 and 0.24 respectively.

VI. CONCLUSION
This paper achieves the dual missions simultaneously,i.e. identifying the unknown plant and designing the two unknown controllers. The reason about why these dual missions can be achieved lies that they all are dependent on the measured input-output sequence. In addition, iterative idea is proposed to identify the unknown plant and devise those two unknown controllers, until the final plant and controllers are satisfied. In order to measure this level of satisfactory, model VOLUME 10, 2022 structure validation from system identification is introduced to check controller validation. The ideal case for controller validation is to guarantee the correlation function between the prediction input error and prediction output error be as small as possible. To better understand the considered direct data driven control scheme, it is applied into UAV flight control system,whose detailed system structure is described.
According to future research, we will develop adaptive or on line direct data driven strategy to design closed loop controller, while introducing dynamic programming, cooperative strategy and game theory into direct data driven control. Furthermore, as stability analysis here corresponds to our preliminary work, its deep results about stability for data driven model predictive control are our ongoing work.