Improved LQR Control Using PSO Optimization and Kalman Filter Estimator

The electric motor is a device that converts electrical power into mechanical power. The DC motor has some advantages compared to the AC motor on its easier way to control the speed or position and wide adjustable range. However, it still has some issues, such as uncertainty and disturbance; most existing controllers still provide bad system performance or have major drawbacks in solving these issues. The alternative solution for a control method with simple, easy to design, and can be implemented in a multi-input multi-output system is the integral state feedback control technique, such as in Linear Quadratic Regulator (LQR). The weakness of LQR control is that there is no exact method to determine the best value of matrix Q and R used to calculate the state feedback gain; trial or manual tuning method is mostly used. Therefore, this paper proposed an improved LQR control using Particle Swarm Optimization (PSO) method and the Kalman filter estimator. The PSO was used to find optimal Q and R-value, while the Kalman filter reduced the number of sensors used to measure the system state. Both performance and energy consumption were compared using the Integral of Absolute Error (IAE) and total energy consumption. The simulation results showed that the proposed method is superior in performance and energy compared to manually-tuned LQR. However, based on the hardware implementation, although the manually-tuned LQR was found to have the smallest energy consumption, the proposed method has the smallest IAE, which can reduce the IAE by 11.28% with only 1% higher energy compared to manually-tuned LQR.


I. INTRODUCTION
The electric motor is a device that converts electrical power into mechanical power. There are two types of electric motor based on their electric supply: DC motor and AC motor. However, the DC motor has some advantages over the AC motor. DC motor is easier to control its speed or position and has a wider adjustable range [1], [2]. It also has low power and good performance [3]. DC motors have been widely applied in various applications, such as in line follower robot [4], [5], mobile robots [6], [7], mini conveyor [8] and electric vehicles [9].
In systems with DC motors, the speed control is generally conducted by changing its terminal voltage [10]. The main issue while controlling the DC motor is its angular speed control with uncertainty and disturbance [11]. Some control methods have been proposed for controlling the speed of DC motors, such as PID [10], LQR [12], [13], SMC [14], integral state feedback [15], and LQG [16]. The most popular among them is PID control because it is simple and easy to be implemented in real hardware [17], [18]. However, PID control has a constraint, which is a sudden change in parameter variation, making the PID controller give a bad system response [19]. An alternative solution of a control method that is simple, easy-to-design, and can be implemented in a multi-input multi-output system is the integral state feedback control [15], [20].
Moreover, some tuning methods can be applied when implementing integral state feedback control. One of them is LQR which has been successfully implemented for several complex systems such as inverted pendulum [21], aircraft [22], missile guidance, autonomous vehicle, and active car suspension [23]. Unfortunately, LQR still has a weakness too: no exact method to determine the best value of matrix Q and R used to calculate the state feedback gain or LQR gain. The number of elements in Q and R matrices depends on the number of state and input variables, respectively. Some previously proposed methods to find the optimal value of the Q and R matrices are PSO and GA [23]. The superiority of PSO over GA in finding optimal weighting matrices of LQR has been proven by [24] and [25].
In addition, some researchers that used PSO also have different ways to optimize the LQR method. Most of them, as in [21], [26], and [27], only use PSO to determine the optimal LQR gain (K LQR ) by finding the best value of the state and input weighting matrices. In this paper, the improvement of LQR control was proposed using PSO optimization and Kalman filter. Since the combination of LQR and Kalman filter is also known as LQG, the proposed method is named Optimized LQG (OLQG). Kalman filter is used in LQR control to estimate some unmeasured state variables as a replacement for physical sensors since LQR control needs all states to be measured [28]. Thus, this paper aims to provide a detailed analysis of the improved LQG using PSO, and a comprehensive performance comparison of the proposed method with similar LQR-based methods, based on simulation and hardware realization results. The system performance of the proposed method is compared with a manually-tuned LQR (LQR1), a PSO-tuned LQR gain (LQR2), and a PSO-tuned LQR and integral gains (LQR3).
Meanwhile, main contributions of the paper can be listed briefly as follows: 1) This paper presents an alternative optimization control method based on PSO, Kalman Filter, and LQR control technique, along with its implementation to real system hardware. 2) An objective function of optimization control based on Integral of Absolute Magnitude of Error (IAE) and the voltage input of the system is formulated in the paper; voltage input is also considered as variable in the objective function of optimization control. 3) A parametric identification modeling of DC motor in state space representation will be presented in the paper; therefore, the obtained state-space model that was used in simulation is highly close to the actual input-output representation of system hardware.
The rest of the paper will be organized as follows. Section 2 will provide a review of the hardware used in this research, LQR control, PSO optimization, and the proposed method. Then, the results of simulation and hardware implementation will be discussed in Section 3. The last section is the conclusion.

A. HARDWARE DESIGN AND SYSTEM MODELING
The system used in this research is a DC-driven mini conveyor. The specifications of the DC motor used are 12V, 5.5A supply with 250rpm maximum speed and 10.6kg/cm maximum torque. The block diagram of the system is shown in Fig. 1(a), while a real picture of the hardware system is shown in Fig. 1(b). Three measurement sensors are used: speed, current, and voltage.
In the state feedback control, all states of the system are required. Also, the state-space model is required to design the LQR control. Hence, prior to controller design, system modeling can be done by conducting one or combining these two methods: mathematical equations and input-output data from the system. The latter method is called black-box modeling [29]. Some researchers used the first method (mathematically based), such as in [30]- [32]. Meanwhile, researchers in [16] and [33] used black-box modeling by analyzing inputoutput experimental data to find the system model.
In the first method, system parameters are required; therefore, it is not suitable for hardware implementation where the VOLUME 10, 2022 system parameters are unavailable. Hence, the second modeling method was chosen for this research. The system model was obtained by taking input and output data in the form of voltage and speed of DC motor using a data logger. The state-space model of the system was derived using MATLAB System Identification based on these data, as shown in (1). The control algorithm was designed and tested in MATLAB/ Simulink before being implemented in the real hardware system.ẋ = 0.91 0.04 LQR is one of the optimal control techniques that consider the system's states and control input to make optimal control decisions. According to [34], this method is simple and robust. The state feedback control is which is derived from the minimization of cost function as shown in (2). The matrices Q and R determine the relative importance of the error and energy expenditure [35]. The LQR gain, K LQR , is derived from (3). The P matrix in (4) which is ARE need to be solve first. This calculation can be computed manually or using software such as MATLAB.

C. THE PROPOSED METHOD: OPTIMIZED LQG (OLQG)
After the model is obtained, then the controller design process can begin. The structure of the control system used is the integral state feedback, which can be seen in Fig. 2.
The Optimized-LQG means that PSO optimization is used to find the optimal LQR gain (K LQR ) while considering the objective function, whereas no optimization applied for estimator gain. The PSO optimization is conducted through the simulation, which involves the state-space model of the plant, known as PSO offline tune. The block diagram of LQR optimized by PSO and the proposed method can be seen in Fig. 2(a) and Fig. 2(b), respectively. The speed error and controller output are used as inputs to PSO with the objective function is formulated as in (7). Fig. 3 is the flowchart of OLQG, starting from finding the system model to creating the simulation model in MATLAB Simulink. The LQG design was done under simulation. The optimization using PSO was then applied. After the simulation results met the design criteria, the process continued to the hardware implementation. On the opposite side, the optimization process is repeated. Finally, the data analysis was done to the results.
The data analysis includes the performance comparison made to other variants of LQR in a state feedback controller.  The proposed method, OLQG, is a control algorithm that applies PSO and Kalman Filter. PSO is used to optimize LQR and integral control gains, while Kalman Filter replaces the current sensor, as in Fig. 2(b). This is very useful since it can reduce the cost of the system [36].

D. PSO OPTIMIZATION CONTROL
PSO was first introduced by Injeti and Divyavathi [37], which was made based on the social and communication behavior of birds. Every individual is assumed as a vector position and represents a potential solution to optimization problems. The advantages of the PSO algorithm are fast convergent speed, few control parameters, certain parallelism. On the other hand, the disadvantage of this method is simple to place down into local optima value [38].
The formulas for calculating speed (V ) and position (X ) of PSO algorithm are shown by (5) and (6), respectively [18].
where g = 1, 2, . . . , G, i = 1, 2, . . ., population size, r 1 and r 2 is random values between 0-1, c 1 is a local learning factor, c 2 is a global learning factor. Commonly, c 2 is bigger than c 1 . The parameters used in PSO are listed in Table 1. The objective function used in this study is the combination of Integral of Absolute Magnitude of Error (IAE) and the controller's output (u), as shown in (7). According to [39], the control output is proportional to the energy consumed. Physic, the controller output is sent to the PWM generator, which drives the DC motor.

III. RESULTS AND DISCUSSION
Four control algorithms were tested and compared in simulation and hardware tests. Those control algorithms were LQR1, LQR2, LQR3, and OLQG. The Kalman filter gain, K f , was manually calculated without an optimization algorithm. Both performance and energy consumption were compared based on IAE and total energy consumption. The four algorithms were designed and tested in a MATLAB Simulink environment before being implemented in a real hardware system. The controller gains used are listed in Table 2. The control tolerance for the test is ± 2%.

A. SIMULATION TEST
A simulation test was done before hardware implementation in the MATLAB Simulink environment. A speed tracking test was chosen to test the performance of the proposed control algorithm. In this test, there are three set-points of speed, starting from 100 rpm, 200 rpm, and finally 80 rpm. These values were chosen to assess the system response when the speed set-point was increased and decreased. The integral gain, LQR gain, and Kalman filter gain used are listed in Table 2.
The system performances were scored based on settling time, overshoot/undershoot, and IAE specifications. The energy parameters were obtained from the total energy consumption. Fig. 4 shows the block diagram of the simulation test in the MATLAB Simulink environment; the upper part of the block diagram is the LQR, and the lower is the LQG.
Simulation results are shown in Fig. 5, where Fig. 5(a) is the speed profile, Fig. 5(b) is the IAE profile, and Fig. 5(c) is VOLUME 10, 2022 the energy profile of augmented systems. The total simulation time is 100s with 0.335s sampling time. According to the speed profile in Fig. 5(a), all controllers could perform well, although there were some overshoots and undershoots in the speed changes.
At the first set-point, which is 100rpm, most responses had overshoots except for OLQG; the response in LQR1 had the highest overshoot. OLQG was also superior to others based on the settling time, while LQR1 had the longest settling time.
At the second set-point (200rpm), it can be seen from the speed profiles that all control algorithms had overshoots in their responses, whereas OLQG has the lowest overshoots. Moreover, most control algorithms had no undershoot in their responses. This is due to the controller tuning being done with a set-point of 100rpm. In the settling time, the fastest performance was shown by the OLQG. Although the response still had an overshoot over the set-point, it is under the controller tolerance, which is ± 2%.
The last given set-point, 80rpm, was used to test the performance where the set-point decreases. All of the responses have the undershoot except for OLQG. The highest undershoot in response is obtained from LQR1, which is 45.66%. In the settling time, LQR2 is the fastest with a small undershoot, which is 3%.
The IAE and energy curve depict the same result as the speed profile, where a higher overshoot/undershoot will make the IAE curve high and make the controller consume more energy than a lower overshoot/undershoot. In the simulation, the energy is calculated based on the integral of controller input based on [39]. The IAE curve will increase when the error increases due to the set-point changes and remains constant in the steady-state region. On the other side, the energy curve will drastically increase with speed changes because changing the speed controller needs more energy; it increases gently when the speed is constant.
The IAE and the energy are accumulated from the simulation start until the end. Therefore, the final value is the total sum of IAE and energy from the system. According to Fig. 5(b) and (c), the lowest and the highest IAE are OLQG and LQR1, respectively. Whereas, from the energy curve, the lowest and the highest energy are OLQG and LQR3. The quantitative result from this test is resumed in Table 3. Performances of all algorithms are compared with the performance of a manually-tuned LQR (LQR1) to show the performance and energy improvement clearly. Therefore, the %IAE and %Energy are set to zero. It can be seen that %IAE and %Energy of the proposed method is −33.11% and −17.27%, respectively. The minus sign indicates that the OLQG has a smaller IAE and energy compared to the LQR1.

B. HARDWARE IMPLEMENTATION
The hardware implementation was done after the simulation test had been successful. In this test, both set-point and testing time is the same as the simulation. Since LQR control needs all system states, the speed and current sensors are used. The voltage sensor is also added to calculate the power and total energy consumed by the system during the testing. The energy analysis will be clearly described using voltage, current, power, and energy curve. Fig. 6 shows the test result of the hardware implementation. The figure presents six graphs: speed profile, IAE profile, voltage profile, current profile, power profile, and energy profile. The first two graphs VOLUME 10, 2022 are used for performance analysis, whereas the rest are for energy analysis. In this test, the sampling time is the same as the simulation, which is 0.335s. The detailed performance parameters and energy are resumed in Table 4. Fig. 6(a) shows that all responses can follow the setpoint well. This can be seen better than in the simulation because the actual hardware response is slower. In addition, the moving average filter is used to smooth the data from the sensors. The quantitative data in Table 4 shows that the settling time in responses of LQR1, LQR2, and OLQG for all set-points, is the same. Meanwhile, there is a variation in LQR3. There is no overshoot at the first and second set-point for all controllers. On the other side, in the last set-point, small undershoots are found: 2.5%, 2.5%, 0%, and 3.75% for LQR1, LQR2, LQR3, and OLQG, respectively. The IAE graph, Fig. 6(b), clearly shows the accumulated error from each controller. It can be seen that the smallest into the highest IAE are OLQG, LQR2, LQR1, and LQR3, which matched with the data from Table 4. Fig. 6(c) -(f) shows analysis based on the energy point of view. The voltage and current graph inform the voltage and current consumed by the system during the test. It can be seen that both voltage and current have similar patterns with the speed profile. If the speed increase, then the voltage and the current will increase too and vice versa. The power curve is calculated from voltage times the current, which also gives the same pattern. The last one is the energy graph calculated by the integration of power multiplied by the sampling time. This energy graph is the accumulation from starting until the end of testing. Therefore, the final value depicts the total energy consumption. Based on Table 4, the lowest to highest energy consumption order is LQR1, OLQG, LQR3, and LQR2. The %IAE and %Energy show that the proposed method has the lowest IAE and energy compared to LQR2 and LQR3. On the other hand, compared to LQR1, the proposed method has 11.28% lower IAE, with only 1% higher energy.

IV. CONCLUSION
The proposed Optimized-LQG control, which used PSO, was successfully implemented in simulation and hardware testing. Four algorithms were tested and compared both in simulation and hardware tests: a manually-tuned LQR (LQR1), a PSO-tuned LQR gain (LQR2), a PSO-tuned LQR with integral gains (LQR3), and OLQG. Based on the system performances (settling time, overshoot, IAE, and energy consumption) from the simulation test, it can be concluded that the proposed method is superior to other algorithms. Results from the hardware implementation prove that the proposed method has the smallest IAE, whereas the smallest energy was consumed by LQR1. Compared with LQR1, the proposed method can reduce the IAE by 11.28% and has higher energy by 1%. These results can still be improved by adjusting the PSO optimization.
The PSO optimization in this paper was only applied to optimize integral gain and LQR gain; however, it can also be used to determine an optimal filter gain in future work.