Nonlinearly Activated IEZNN Model for Solving Time-Varying Sylvester Equation

Zeroing neural network (ZNN) is an effective method to calculate time-varying problems. However, the ZNN and its extensions separately addressed the robustness and the convergence. To simultaneously promote the robustness and finite-time convergence, a nonlinearly activated integration-enhanced ZNN (NIEZNN) model based on a coalescent activation function (C-AF) has been designed for solving the time-varying Sylvester equation in various noise situations. The C-AF with an optimized structure is convenient for simulations and calculations, which promotes NIEZNN accelerates convergence speed without remarkable efficiency loss. The robustness and the finite-time convergence of the NIEZNN model have been proved in theoretical analyses. Furthermore, the upper bounds of convergence time of the NIEZNN model and the noise-attached NIEZNN model have been deduced in theory. At last, numerical comparative results and the application to mobile manipulator have validated the efficiency and superiority of the NIEZNN model based on the designed coalescent activation function.


I. INTRODUCTION
Sylvester equation has a wide field of application in disturbance decoupling [1], image processing [2], linear least square regression [3], control system [4] and robots [5]. Therefore, the study on the solution of Sylvester equation has been investigated greatly [6], [7], [8], [9], [10], [11]. The Bartels-Stewart algorithm is a classical direct method to solve Sylvester equation [12]. Based on this algorithm, some extensions were proposed in [13] and [14]. Nevertheless, the Bartels-Stewart algorithm was proven that it can complete the calculation within a time complexity O(n 3 ). When used to large-scale real-time application, the Bartels-Stewart algorithm cannot satisfy the requirement. In addition, based on gradient means, a mass of iterative algorithms were exploited to solve Sylvester equation [15], [16], [17], [18]. Those algorithms worked well in static Sylvester equation. However, The associate editor coordinating the review of this manuscript and approving it for publication was Yilun Shang. when applied to the dynamic (i.e. time varying) Sylvester equation, these algorithms fail because the sampling rate is too high in time-varying Sylvester equation.
In recent decades, the recurrent neural network (RNN) as a typical representative of parallel algorithm, was greatly applied in optimization [19], robotics [20], [21], game theory [22], prediction [23]. Different from iterative algorithms, RNN has excellent performance in parallel-distributed processing and convenient hardware implementation. As a competitive computational tool, RNN plays a pivotal role in solving Sylvester equation [4], [24]. According to above literatures, it is known that the solution error of RNN vanished to zero in time-invariant case, and the error failed in vanishing to zero in time-varying case. For overcoming the defect, a special kind of RNN was first proposed in [24], which named zeroing neural network (ZNN). By exploiting the velocity compensation of time-varying coefficients, ZNN can exponentially converge to the theoretical solution of timevarying Sylvester equation [25], [26]. The design formula of conventional ZNN (CZNN) is presented as follows: where E(t) is the error monitoring function, the parameter µ > 0. Due to the significant improvement in convergence property, ZNN was studied deeply. The in-depth study can be mainly summarized into two branches. One is robustness. Noise is unavoidable in the solution task of ZNN, such as some realization errors in hardware implementation, the environmental disturbance. Sometimes, the noise has significant impacts on the accuracy of ZNN. Therefore, it is worth investigating the robustness of ZNN with different noise. The evolution formula of CZNN has an essential design defect: sensitive to noise [27], [28]. To overcome this flaw, in [27] and [28], the evolution formula was ameliorated by the integral unit aṡ where the parameters µ > 0, ν > 0. Due to the design of the integral unit, (2) is named as integration-enhanced ZNN (IEZNN). IEZNN owns a noise-suppression property. The improvement of robustness expanded the practical application of ZNN. The other is finite-time convergence. As time is valuable in solving time-varying issues, the research on finite-time convergence has became a hot point in the field of ZNN. As noted in literature [29] that appropriate nonlinear activation function can accelerate the convergence. Which inspires researchers to explore certain nonlinear activation functions to improve convergence of ZNN. For example, in [30], [31], [32], [33], and [34], three nonlinear activation functions (i.e. the bi-polar-sigmoid activation function, power activation function and power-sigmoid activation function) were explored and widely adopted. It is worth pointing out that these mentioned nonlinear activation functions have superior convergence speed than the linear one, but cannot actualize the finite-time convergence. The unified design formula of the nonlinear activated ZNN (NAZNN) is presented aṡ where µ > 0, A(·) denotes a nonlinear activation function. For the purpose of enabling ZNN to converge in finite time, in 2013, a sign-bi-power activation function (named SBP-AF) was fisrt presented in [25], which actualizes the finite-time convergence. After that, the finite-time convergence of ZNN has been widely investigated [35], [36], [37], [38], [39], [40], [41]. With its further study, the SBP-AF is found that the structure is too redundant, which interferes simulation calculation and hardware implementation. Therefore, for improving the finite-time convergence, the structure of SBP-AF can be further optimized. Based on the above analysis, it can be concluded that the robustness and finite-time convergence of ZNN can simultaneously investigated by designing a new unified formula combined Eq. (2) and Eq. (3). The purpose of our work is to establish a new nonlinearly activated IEZNN (NIEZNN) model for solving the time-varying Sylvester equation in different noise environments. Three cruces are considered in the design of NIEZNN, one is the stability, to obtain effective computation; the other is the robustness, to own good noise immunity; and the last one is expediting the finite-time convergence, to meet higher requirements in solving time-varying problems. A designed coalescent activation function (C-AF) is embedded to the NIEZNN model, which impels the NIEZNN possess better finite-time convergence and noise suppression in solving the time-varying Sylvester equation. In addition, theoretical conclusions are derived, and the superiority of NIEZNN under additive noise is verified via comparison of numerical experiments and application on controlling mobile manipulator. The novelty and contribution of this paper are summed up as below.
• A general framework of NIEZNN is proposed for solving time-varying Sylvester equation. Besides, a novel nonlinear function is presented to activate the NIEZNN mode, which is more convenient for simulations and calculations.
• The robustness and the finite-time convergence of the NIEZNN model have been proved in theoretical analyses. Furthermore, the upper bounds of convergence time of the NIEZNN model and the noise-attached NIEZNN model have been deduced in theory.
• The existing neural networks are applied to solve time-varying Sylvester equation for comparative purposes, and abundant experimental validations (including the application to mobile manipulators) demonstrate the better finite-time convergence and robustness of the proposed NIEZNN based on a novel C-AF.

II. PROBLEM FORMULATION AND MODEL
In this section, we are concerned with the following time-varying Sylvester equation.
where A(t), B(t) and C(t) present known time-varying matrices, Y (t) is an unknown time-varying matrix. The aim of our work is to obtain the solution of Y (t) within finite-time in different noises by taking advantage of the NIEZNN. Following the general design procedure of ZNN model, the design process of the NIEZNN model for time-varying Sylvester equation can be obtained as below.
At first, define the error monitoring function: Then, define a evolution formula for E(t), which simultaneously achieves finite-time convergence and noise tolerance. Combining the design characteristics of (2) and (3), a novel evolution formula is defined as following: where µ > 0 and ν > 0 are the scale parameters, A 1 (·) and A 2 (·) denote the nonlinear activation functions, which are odd and monotone increasing. As we know that the SBP-AF designed in [25] can endow ZNN finite-time convergence, but the structure is too redundant, which interferes simulation calculation and hardware implementation. If we continue to expedite the convergence speed, much efficiency will be sacrificed. Therefore, we propose a new coalescent activation function (named C-AF) combined the simplified form of SBP-AF with a linear activation function as following: where sgn θ (·) is defined as To illustrate the proposed C-AF, Fig. 1 plots the curves of the C-AF with different parameters values. From the figure, we can see that the effect of α and β is more significant than θ, a larger value of the parameters α and β prompts function (7) has a faster rate change. At last, combining (5) and (6), the NIEZNN model is obtained for solving time-varying Sylvester equation.
As we know that noise is inevitably in the practical environment. In general, especially in noise disturbed situation, a zeroing neural network might develop to an unstable state (e.g. oscillation, divergence and chaos). In this case, noise leads to failure of the computing task. In order to investigate the anti-noise performance of NIEZNN model (9) for solving time-varying Sylvester equation, we extend the NIEZNN model (9) by appending a random noise as follow noise-attached NIEZNN model: where o(t) denotes a random noise matrix, the random noise may be constant noise or time-varying noise. Remark 1: The parameters µ and ν can adjust the convergence of neural network systems of NIEZNN model (9). A large value of µ and ν can accelerate the convergence. As to the robustness, a large value of µ and ν is apt to make the neural network system sensitive to noise. The integral unit in NIEZNN model (9) can eliminate instability of control systems with additional noise. Therefore, in applications, parameters µ and ν are set as large as hardware permits.
Remark 2: The matrix-form NIEZNN model is transformed into an initial-value ODE problem via MATLAB, which makes simulation and calculation more easy and efficient.

III. ROBUSTNESS AND FINITE-TIME CONVERGENCE ANALYSES
In this section, theoretical analyses are presented, which show the robustness and finite-time convergence of the NIEZNN model with the proposed C-AF (7).
Theorem 1: In front of an unknown additive noise, the state matrix Y (t) generated by noise-attached NIEZNN model (10) is stable and globally converges to the theoretical solution Y * (t) of (4), that is to say, E(t) will globally converges to zero with time.
Proof: The evolution formula of noise-attached NIEZNN model (10) is presented aṡ (11) the ijth entry ofĖ(t) of (11) can be obtained as below: (12) we set the derivative of (13) can be gotten as: Finally, substituting (13), (14) into (12), we havė for analyzing the stability of (12), a Lyapunov function candidate is constructed as following Because of the given condition that A 2 (·) is an odd monotone increasing function, we can conclude that According to Eq.(15), we have the conclusion lim t→∞˙ ij (t) = 0.
based onė ij (t) =˙ ij (t) − µA 1 (e ij (t)) again, with the time goes on, we haveė which is the NAZNN design formula, we can construct a Lyapunov function candidate Z ij (t) = e 2 ij (t)/2, obviously, Z ij (t) is positive definite. Besides, the derivative of Z ij (t) can be obtained easily as: from the given condition that A 1 (·) is an odd monotone increasing function, we haveŻ ij (t) ≤ 0, i.e.,Ż ij (t) is negative definite, that's to say e ij (t) globally converge to zero with time going on in noise situation. The proof is completed.

B. FINITE-TIME CONVERGENCE
Theorem 2: The state matrix Y (t) generated by the NIEZNN model with the proposed C-AF (7) is stable and globally converges to the theoretical solution Y * (t) of (4) in finite-time where e + (0) and e − (t) represent the largest and the smallest values of E(0). Proof: Based on the evolution formula (6), the element ofĖ(t) can be obtained as below: Based on the design of the proposed C-AF (7) and NIEZNN (9) model, considering the definition of sgn θ (·) in (8), we have 1) If (0) > 0, (17) can be derived aṡ which is equivalent to To solve the differential equation (18), we set ϕ(t) = ( (t)) 1−θ , then So, (18) can be written as The solution of above differential equation is As we know that (t) converges to 0 at the finite time t * , that is (t * ) = 0 and ϕ(t * ) = 0. Thus, Solving the above equation, in the case of (0) > 0, we obtain the upper limit of convergence time 2) If (0) < 0, similarly, we have the result that 3) If (0) = 0, obviously, Therefore, we can have the conclusion that (t) converges to 0 when t > Because of (0) = e(0), for (t), the upper bound of the finite convergence time t 1 of the NIEZNN model is obtained as When t > t 1 , (t) = 0 and˙ (t) = 0, fromė(t) =˙ (t) − µP 1 (e(t)), we haveė (t) = −µA 1 (e(t)), adopting the same way in˙ (t) = −νA 2 ( (t)), for e(t), the upper bound of the finite convergence time t 2 of the NIEZNN model is obtained as Finally, we conclude that NIEZNN (9) model can converge to zero in finite time, and the upper bound of convergence time is estimated as The proof completes.
To investigate the finite-time convergence of noiseattached NIEZNN model (10), we have the following theorem.

Theorem 3. In front of an unknown additive noise o(t) with its element satisfying o ij (t) ≤
( > 0), the state matrix Y (t) generated by noise-attached NIEZNN model (10) with the proposed C-AF (7) is stable and globally converges to the theoretical solution Y * (t) of (4) in finite-time where e + (0) and e − (t) represent the largest and the smallest values of E(0). Proof: Based on the Theorem 1, the element ofĖ(t) can be obtained as below: We also set (t) = e(t) + µ t 0 A 1 (e(s)) ds, then, we havė (t) = −νA 2 ( (t)) + o(t), where A 2 (·) adopts the proposed C-AF (7), o(t) is set as θ (t). Eq.(20) is represented as: according to the definition of sgn θ (·) in (8), when (0) > 0, Eq.(21) can be derived aṡ which is equivalent to We set ϕ(t) = ( (t)) 1−θ , then so, Eq. (22) can be written as The solution of the differential equation (23) is As we know that (t) converges to 0 at the finite time t * , that is (t * ) = 0 and ϕ(t * ) = 0. Thus, Solving the above equation, in the case of (0) > 0, we obtain the upper limit of convergence time When (0) ≤ 0, similarly, we have the result that Therefore, we can have the conclusion that (t) converges to 0 when t > So, the upper bound of the finite convergence time t 1 of the noise-attached NIEZNN model is obtained as When t > t 1 , (t) = 0 and˙ (t) = 0, from (t) = e(t) + µ t 0 A 1 (e(s)) ds, we haveė(t) =˙ (t) − µP 1 (e(t)), theṅ e(t) = −µA 1 (e(t)), where A 1 (·) adopts the proposed nonlinear activation function (7), for e(t), as Theorem 2, the upper bound of the finite VOLUME 10, 2022 convergence time t 2 of the noise-attached NIEZNN model is obtained as Finally, we conclude that noise-attached NIEZNN model (10) can converge to zero in finite time, and the upper bound of convergence time is estimated as In summary, the finite-time convergence of the noise-attached NIEZNN model is proven.

IV. SIMULATION AND VERIFICATION
The robustness and finite-time convergence characteristics of NIEZNN model are analyzed in Section III. In this section, numerical experiment results are conducted to validate the superiority of the presented NIEZNN model (9) for solving time-varying Sylvester equation in noise situations. For comparative purposes, on the one hand, the IEZNN, NAZNN and the proposed NIEZNN model are adopted. On the other hand, linear activation function, some representative nonlinear activation functions (including PS-AF and SBP-AF) and the proposed C-AF are embedded in NIEZNN model (9) and noise-attached NIEZNN model (10). Without loss of generality, we set θ = 0.2, α = β = 1, µ = ν = 2.
Example To solve Y (t) of (4), the coefficients for A(t), B(t) and C(t) are chosen as following The effectiveness and accuracy of NIEZNN model (9) are illustrated in Figs.2-3. The overlap of the values synthesized by NIEZNN model (9) and the actual solution is demonstrated in Fig. 2. This figure shows, over a   Fig. 5(a). For further verification, a linear time-varying noise o(t) = 2t is considered, the simulation results are depicted in Fig. 5(b). When faced time-varying noise, the residual error of the proposed NIEZNN model still quickly converges to zero, which takes no more than 1 s. On the contrary, the residual error of IEZNN model does not decline to zero and remains at approximately 0.8 s. Moreover, the residual error of the NAZNN model keeps climbing over time.

V. APPLICATION TO CONTROL MANIPULATOR
In this section, we introduce a mobile manipulator to certify the applicability of the NIEZNN model. The repetitive motion scheme of the mobile manipulator can been found in [42]. The position level, velocity level and the acceleration level can be written as where κ := 2χφ(t) + χ t (φ(t) − φ(0)), here χ > 0, and φ(t),φ(t),φ(t) respectively denote the vectors of the manipulator's joint-angle, joint-velocity and jointacceleration; J (φ(t)) represents the manipulator's Jacobian matrix;ω  (t) :=ω(t) −J (φ(t))φ(t). Based on the repetitive motion scheme, a kinematics equation can be obtained by using Lagrangian multiplier L(t) as where Then, we utilize the method of the proposed NIEZNN model to control the motion of the mobile manipulator under the a constant noise o(t) = 5. In this simulation experiment, the mobile manipulator is allocated to track a circular route (with the radius being 0.5 m ). The initial joint-angle of the mobile manipulator is set as φ(0) = [0, 0, π/12, π/6, π/12, π/6, π/12, π/6] T , and experimental duration is 5 s. The experiment results synthesized by the NIEZNN model are displayed in Fig. 7-8. As shown from Fig. 7, the mobile manipulator excellently performs the tracking task. Fig. 7(a) indicates the path of joint-angle from initial values to final values. Fig. 7(b) indicates the synthesized trajectory overlaps with the theoretical trajectory. Fig. 7(c) depicts simulated velocity the coincides with the theoretical velocity. In addition, Fig. 8 reveals the position error, acceleration error and velocity error are less than 1.4 × 10 −5 , the accuracy can meet the needs of practical application. The results of the trajectory-tracking about mobile manipulator fully confirms the design method of the NIEZNN model possesses stronger effectiveness and accuracy.

VI. CONCLUSION
In order to simultaneously promote the robustness and finite-time convergence, a nonlinearly activated integrationenhanced zeroing neural network (NIEZNN) has been designed for solving the time-varying Sylvester equation in various noise situations. In addition, a new coalescent activation function (C-AF) with simpler structure has been designed to embed into the NIEZNN model, which is convenient for hardware implementation, and accelerates convergence speed without remarkable efficiency loss. Furthermore, the robustness and the finite-time convergence of the NIEZNN model have been proved in theoretical analyses. Moreover, the upper bounds of convergence time of the NIEZNN model and the noise-attached NIEZNN model have been deduced in theory. Finally, the advantages of the proposed NIEZNN model based on the designed C-AF have been verified by comparing the NIEZNN model with existing ZNN models for computing time-varying Sylvester equation in noise situations. The comparing results and the application to mobile manipulator have demonstrated the efficiency and superiority of the NIEZNN model and the new coalescent activation function. Compared with fixed parameters, timevarying parameters in ZNN have significant superiority in stability, convergence and robustness. In the future, exploring a delicately time-varying parameter NIEZNN to solve timevarying issues is a valuable direction.