Solving Time-Varying Complex-Valued Sylvester Equation via Adaptive Coefficient and Non-convex Projection Zeroing Neural Network

The time-varying complex-valued Sylvester equation (TVCVSE) often appears in many fields such as control and communication engineering. Classical recurrent neural network (RNN) models (e.g., gradient neural network (GNN) and zeroing neural network (ZNN)) are often used to solve such problems. This paper proposes an adaptive coefficient and non-convex projection zeroing neural network (ACNPZNN) model for solving TVCVSE. To enhance its adaptability as residual error decreasing as time, an adaptive coefficient is designed based on residual error. Meanwhile, this paper breaks the convex constraint by constructing two complex-valued non-convex projection activation functions from two different aspects. Moreover, the global convergence of the proposed model is proved, the anti-noise performance of the ACNPZNN model under different noises is theoretically analyzed. Finally, simulation experiments are provided to compare the convergence performance of different models, which simultaneously verifies the effectiveness and superiority of the proposed model.


I. INTRODUCTION
I N recent years, Sylvester equation has become widely available for many research fields, such as control engineering, image processing, and communication engineering [1]- [6], etc. According to the previous studies, there are mainly two types of methods for solving Sylvester equation: one is the method of serial processing, and the other is parallel processing. Some serial processing methods mainly belong to numerical algorithms. For example, the Bartels-Stewart algorithm and the Hessenberg-Schur algorithm are two effective methods to solve the static or time-invariant Sylvester equation [7], [8]. The Bartels-Stewart algorithm can significantly save computer time when solving the time-invariant Sylvester equations [7], and the time complexity to complete the calculation is O(n 3 ). Compared with the Bartels-Stewart algorithm, the Hessenberg-Schur method [8] simplifies the matrix in the Sylvester equation to the Hessenberg-Schur form, which increases the calcula-tion speed by 30% − 70%. Unfortunately, these numerical algorithms can only be widely applied to the solving of timeinvariant Sylvester equation, and it is not suitable for the Sylvester equation with time-varying parameters.
In the past two decades, the recurrent neural network (RNN) [9]- [11] has been widely used as a parallel processing model in sign language recognition [12]- [14], robotics [15]- [17], Encoder-Decoder [18]- [20], etc. Generally speaking, RNN model can efficiently process the sequence characteristics, especially the time sequence information and semantic information in the data can be mined. In particular, the gradient neural network (GNN) model [21]- [23] and zeroing neural network (ZNN) [24]- [26] model are constructed to deal with time-varying problems, such as time-varying matrix inversion. The GNN model uses the residual error as a performance indicator, sets several initial values, and performs iterative operations in the direction of negative gradients until the residual error converges to a reasonable VOLUME 4, 2021 range and then stops.
Nonetheless, the parameters in time-varying Sylvester equation (TVSE) change with time. Jin et al. [27] point out that the GNN model cannot effectively use the time derivative information, which leads to the separation of the state solution at the last moment and the state solution at the next moment. In other words, the state solution generated by solving the time-varying problem will generate a time lag error, and the corresponding residual error cannot converge to zero over time. Hence, the GNN model is not suitable for solving TVSE. Compared with the GNN model, the ZNN model can effectively use the time derivative of timevarying parameters [28]. When time tends to infinity, the resulting residual norm converges to zero. In paper [29], Li et al. introduce a sign-bi-power activation to activate the ZNN model so as to solve the TVSE in a limited time. It is worth noting that the ZNN model activated by sign-bi-power activation here is simulated under noise-free conditions, and the activation function does not have noise immunity [30], [31].
On the basis of the above research, this paper extends the application range of solving TVSE from the real-valued domain to the complex-valued domain and proposes a complexvalued ZNN [32] model to solve time-varying complexvalued Sylvester equation (TVCVSE). In the TVCVSE solution process, the residual norm will become smaller as time increases. At the same time, the coefficients of the traditional ZNN model are fixed values, which cannot perfectly follow the change of residual error. Therefore, this paper proposes adaptive coefficients [33] to construct a adaptive coefficient and non-convex projection zeroing neural network (ACN-PZNN) model, which not only can overcome the excessively long convergence time and convergence accuracy of the ZNN model, but also achieve the effect of adaptive system changes. Then, this paper applies the saturation function as the activation activation of the ACNPZNN model, and further shortens the convergence time on the basis of the ACNPZNN model by reading the literature [34]- [36]. Furthermore, considering the convex constraint problem, this paper uses a non-convex bound activation function to activate the ACNPZNN model and to relax the convex constraint. In addition, the nonconvex bound activation function has strong noise immunity under the premise of ensuring satisfactory convergence time and convergence accuracy.
The remaining parts of this paper can be organized as follows. In Section II, the Sylvester equation problem formulation is provided, and some existing derivation processes are introduced. The adaptive coefficients are designed to establish the ACNPZNN model in Section III. And in addition, a method of constructing complex-valued activation function is proposed. Section IV proves the convergence performance of the ACNPZNN model under two activation conditions through theoretical analysis. Moreover, based on Section IV, Section V analyzes the robustness of the ACNPZNN model under constant noises and random noises, respectively. Finally, Section VI gives a complex-valued example of solving the Sylvester equation and analyzes the state solution of the ACNPZNN model under multiple sets of initial values and the convergence of the ACNPZNN model under different noise environments through the experimental results. The main contributions of this paper are summarized as follows.
• A novel adaptive complex-valued zeroing neural network (ACNPZNN) model is creatively proposed to solve the TVCVSE in this paper, which can efficiently deal with complex-valued time-varying problem and ensure the high precision of the solution results. • A non-convex saturated framework in the complexvauled domain is presented for constructing the ACN-PZNN model. Furthermore, this paper analyzes the antinoise performance of the framework under different noise conditions and the performance comparison of other models through theorems and proofs. • In this paper, four sets of different initial values are set and executed. A series of experiments show that the simulation results basically coincide with the theoretical analyses, which verify the superiority and feasibility of the proposed ACNPZNN model.

II. PROBLEM FORMULATION AND RELATED WORK
In general, the form of the TVCVSE problem can be expressed as follows: where t denotes time, t 0 and t f are the starting time and final time, respectively; time-varying complex-valued matrices M (t) ∈ C m×m , N (t) ∈ C n×n and K(t) ∈ C m×n are known coefficient matrices; X(t) ∈ C m×n is the unknown complex-valued time-varying matrix to be obtained. In addition, if and only if the difference between any two eigenvalues of the time-varying matrix M (t) and N (t) is not equal to zero at any time t, Eq. 1 has a unique solution.
Suppose that X * (t) represents the unique theoretical solution of the TVCVSE problem (1), then vectorizing both sides of Eq. (1) and we can get: where vec(·) denotes the vectorization of a matrix. According to the defination of the vectorization operation, the conclusion can be drawn that vec( Consequently, the equation (2) can be converted into the equation as follows: vec (M (t)X(t)) − vec(X(t)N (t)) = − vec(K(t)).
For further transition, the equation (3) can be written as where I n ∈ C n×n and I m ∈ C m×m are the identity matrices. In the light of the property of vectorization , that vec(EXF ) = (F T ⊗ E)vec(X) and equation (4), we have where T denotes the transpose of the matrix or the vector. ⊗ is the Kronecker product. The Eq. (5) above can be expressed as the following form, where,

III. SOLVING MODEL CONSTRUCTION
In this section, we solve the TVCVSE (1) problem by setting the error function, and propose the definition of adaptive coefficients to construct the proposed ACNPZNN model.

A. THE CONSTRUCTION METHOD OF THE ACNPZNN MODEL
In order to solve the TVCVSE (1) problem, we divide the solving process into the following three steps.
Step 1: An error function can be constructed as Compared with the original problem Eq. (1), the above equation cleverly transforms the TVCVSE (1) problem into an error function for finding the zero point problem. Deriving Eq. (7), we can get: Step 2: A traditional ZNN model can be written as: where η > 0 represents a constant that controls the convergence speed of the error function, Ψ(·) represents the complex-valued activation function. Then, we will introduce an adaptive parameter η( (t)) : C m×n → C, and substituting it into equation (9), we can get a novel solution model as follows: where the adaptive coefficient is designed as η( (t)) = 10 × exp(|| (t)|| 2 ) + 10.
Step 3: Combine the Eq. (8) and Eq. (10) to obtain: Only keep S(t)ẋ(t) on the left side of Eq. (11), move the remain terms on the right side of the equation to get: Finally, we can get the following equation:

B. CONSTRUCTION METHOD OF COMPLEX-VALUED ACTIVATION FUNCTION
Different from the traditional real-valued activation function design method, the complex-valued activation function needs to consider the real and imaginary parts of the data. In fact, there are two methods for constructing the complex-valued activation function. The first method is to extract the real part and the imaginary part to activate, respectively, and then regenerate a new data. The second method is to extract the modulus and argument of the input data to activate, respectively, and then synthesize a new data.
1) Type-I conplex-valued activation function 2) Type-II conplex-valued activation function In the type-I activation function (14), parameter i is the imaginary unit, and B ∈ C m×n and D ∈ C m×n are the real and imaginary parts of the input data, respectively. In the type-II activation function (15), Λ ∈ C m×n and θ ∈ [0, 2π] m×n represent the modulus and argument of B + iD, respectively. denotes the Hadamard product.
In addition, note that whether it is type-I activation function (14) or type-II activation function (15), they are all complex-valued mapping functions. Meanwhile, if the activation function ψ(·) is a linear activation function, the type-I Ψ 1 (·) and the type-II activation function Ψ 2 (·) are equivalent; if not, the type-I activation function Ψ 1 (·) and the type-II activation function Ψ 2 (·) are different.
The definition of the bound or non-convex bound activation function is as follows: Y σ (Z) =argmin G∈σ ||G − Z|| 2 with 0 ∈ σ. According to the construction methods of two complex-valued activation functions (14) (15),˙ (t) = −η( (t))Ψ( (t)) can be reformulated aṡ where r (t) and i (t) represent the real and imaginary parts of (t), respectively. The following bound constraint activation function (B-CAF) and nonconvex bound activation function (NBAF) can be used to implement the type-I activation function (14): where z s represents the sth element of the vector z.
Parameters κ 1 , κ 2 and κ 3 satisfy the following relationships: The following is example of constructing type-II activation function (15) with ball situation with saturation (BSWS): The above-mentioned three instances can be applied as activation functions of the proposed ACNPZNN model (13). Besides, in order to investigate the global convergence of the proposed ACNPZNN model activated by the activation function (18) and (19), the following two theorems are provided in the next section.

IV. CONVERGENCE OF ACNPZNN MODEL
The analysis and proof of the proposed ACNPZNN model (13) activated by the nonconvex bound activation function are contained in this section. Specifically, the global convergence of the BCAF activated ACNPZNN model (13) and NBAF activated ACNPZNN model (13) is arranged in Theorem 1 and Theorem 2, respectively. In order to further analyze and investigate, the following two related theorems are proposed.
Theorem 1: Beginning with arbitrary time-varying smooth initial matrices M (t), N (t), and K(t), the proposed ACN-PZNN model (13) activated by the type-I activation function Ψ 1 (·) is globally convergent to the theoretical solution of the TVCVSE problem (1).
Proof: The sth subelement in˙ (t) = −η( (t))Y σ1 ( (t)) can be expressed asξ s (t) = −η(ξ s (t))Y σ1 (ξ s (t)). After the above element is activated by the type-I activation function (14), it can be expressed as the following: where the parameters b s (t) and d s (t) represent the real and imaginary parts of the ξ s (t), respectively. In order to prove the global convergence of the real part of ξ s (t), the Lyapunov function candidate is defined as Consequently, the above function P b (t) is positive definite. Derivating of Lyapunov function candidate (23): It is apparent thatṖ b (t) < 0 when b s = 0, andṖ b (t) = 0 only when b s = 0. Therefore,Ṗ b (t) is negative definite. According to the Lyapunov stability theory, b s (t) will eventually converges to zero. Sorting out the above derivation process, we can easily deduce that the imaginary part d s will globally converge to zero. In summary, we can conclude that the error function (t) (7) globally converges to zero. The proof is thus completed.
Theorem2: For any time-varying smoothly complexvalued matrices M (t), N (t), and K(t), the results of the ACNPZNN model (13) activated by the type-II activation function (15) converge globally to the theoretical solutions of the TVCVSE (1).

V. STABILITY OF ACNPZNN MODEL UNDER TWO KINDS OF NOISES
Two theorems are provided in this section to analyze the robustness of the proposed ACNPZNN model (13) under two kinds of noise interference, concretely, the complex-valued constant noises and bounded random noises, respectively.

A. THE ROBUSTNESS OF THE ACNPZNN MODEL UNDER COMPLEX-VALUED BOUNDED RANDOM NOISES
Theorem 3: Under the interference of complex-valued bounded random noises ρ(t), the residual error || (t)|| 2 is generated by the proposed ACNPZNN model (13). When the time is infinite, the upper value of the residual error || (t)|| 2 is limited by the fixed value (t) 2 = mn h=1 ϑ 2 η( (t)) , of which ϑ = max ρ(t) 2 .
Proof: Firstly, the proposed nonconvex bound activation function activated ACNPZNN model (13) is defined as (t) = −η( (t))Ψ( (t)). Select the linear activation function Ψ( (t)) = (t) to discuss the situation of complex-valued bounded random noises ρ(t) interference. Hence, the evolution formula of the ACNPZNN model (13) with the complexvalued bounded random noises interfered can be written aṡ Consequently, the formula (13) is described aṡ It is known that the general solution of the first-order nonhomogeneous linear differential equation Therefore, the general solution of the equation (32) can be expressed as Converting equation (33) into inequality, it can be obtained as follows: When time t is infinite, let the maximum value in the domain of definition be taken as ϑ = max|ρ(t)|, then inequality (34) can be transformed into |ξ s (t)| ≤ ξ s (0)e −η(ξs(t))t + ϑ η( (t)) .
In light of the definition of the adaptive coeffcients η( (t)), it is clearly to draw a conclusion that lim t→+∞ η( (t)) = a and the parameter a is a constant. Therefore, when t → +∞, the upper bound of the residual error (t) 2 generated by the ACNPZNN model (13) converges to a certain value. The proof is thus completed.

B. THE ROBUSTNESS OF THE ACNPZNN MODEL UNDER COMPLEX-VALUED CONSTANT NOISES
Theorem 4: Defining the upper and lower bounds of the bounded constrained activation function as v + , v − , respectively. If the real ρ r and imaginary parts ρ i of the complexvalued constant noises ρ(t) =ρ satisfy the following conditions: when lim t→+∞ η( (t)) = a makes 0 ≤ρ r ≤ av + , 0 ≤ρ i ≤ av + or 0 >ρ r > av − , 0 >ρ i > av − , the residual error (t) 2 of the ACNPZNN model (13) will converge globally to a fixed value composed of complex-valued constant noisesρ.
Proof: The proposed ACNPZNN model under the interference of complex-valued constant noisesρ can be rewritten as˙ the subsystem of the equation (37): At the same time,˙ s (t) has another way of expression: Furthermore, the following two equations can be obtained as where the parameter γ s (t) and τ s (t) represent the real and imaginary part of the s (t), and the parameterρ rs andρ is denote the real and imaginary part of theρ s . Regarding the above equation (40), the Lyapunov function can be designed as Then, the time derivative of W s (t) is calculated aṡ Together with (38), we geṫ Obviously, lim t→+∞ η( (t)) = a, so the equation (44) can be written as lim t→+∞Ẇ s (t) = −γ s (t)(aY σ (γ s (t)) −ρ rs ).
(45) Therefore, we need to discuss the value of γ s (t) to determine the value ofẆ s (t). The following is a discussion of the three cases of γ s (t): 1) In the subcase of γs(t) < 0 • aY σ (γ s (t)) −ρ rs < 0 Based on the above conditions, it is obviously inferred thatẆ s (t) < 0. W s (t) is positive definite for the reason that when γ s (t) = 0, W s (t) > 0, but only when γ s (t) = 0, W s (t) = 0. Therefore, it can be summarized that γ s (t) globally converges to zero based on Lyapunov stability theorem.
2) In the subcase of γs(t) = 0 • For this subcase, Y σ (γ s (t)) = 0. And substitute it into equation (40), q s (t) =ρ rs . It can readily draw the conclusion that the value of s (t) depends entirely on the value ofρ rs . That is to say, γ s (t) will be transformed into subcase 1 or subcase 3 after this moment because it is a transient state.
3) In the subcase of γs(t) > 0 Under this circumstance,Ẇ s (t) < 0. In view of W s (t) > 0, thus γ s (t) globally converges to zero based on Lyapunov stability theory. Significantly, as aY σ (γ s (t)) continues to decrease, there is a situation where aY σ (γ s (t)) =ρ rs is satisfied at a certain moment, which is the issue discussed in case 2 of subcase 3.
In such a case, it can be converted toẆ s (t) = 0. Considering Lyapunov stability theory, γ s (t) can globally converge to a fixed value Y −1 σ (ρ rs /a). • aY σ (γ s (t)) −ρ rs < 0 Out of consideration for the known conditions, it is easy to getẆ s (t) > 0. Whenρ rs ≤ av + , the model will meet condition aY σ (γ s (t)) −ρ rs = 0 and return to case aY σ (γ s (t)) −ρ rs = 0, which makes become stabilized. Therefore, the real parts of the error function will converges to Y −1 σ (ρ rs /a).
Similarly, the imaginary part p s (t) can also be proved to globally converges to Y −1 σ (ρ is /a) in the same way. Given what has been discussed, we can come to the conclusion that, if the real and imaginary parts of complex-valued constant noisesρ s match conditions 0 ≤ρ r ≤ av + or 0 >ρ r > av − , and 0 ≤ρ i ≤ av + or 0 >ρ i > av − , the residual error (t) 2 of the ACNPZNN model (13) can converge to a certain fixed value (Y −1 σ (ρ rs /a)) 2 + (Y −1 σ (ρ is /a)) 2 . The proof is thus completed.

VI. SIMULATION EXPERIMENTS AND COMPARISONS
In this section, in order to compare the robustness and convergence of the ZNN model, GNN model, and the proposed ACNPZNN model, an example of the TVCVSE 1 problem is designed, which illustrates the performance under three conditions: noises-free, complex-valued constant noises, and complex-valued bounded random noises.

A. TIME-VARYING COMPLEX-VALUED SYLVESTER EQUATION EXAMPLE
For the sake of simplicity, the illustrative example of TVCVSE (1) problem can be constructed as follows: , At all times, the common eigenvalues of the time-varying matrices M (t) and N (t) satisfy the special requirement of the Eq. (1). Next, the corresponding time-varying matrices S(t), k(t) in Eq. 6 can be listed as follows: At the same time, an example of constructing type-I complex-valued activation function (14) by using NBAF (19) is as follows: Besides, the adaptive coefficient is designed as η( (t)) = 10 × exp(|| (t)|| 2 ) + 10.
In addition, four set of initial values are provided in Fig. 1.

B. PERFORMANCE COMPARISONS AMONG DIFFERENT MODELS FOR SOLVING TVCVSE WITH NOISE-FREE
The simulation visual results of traditional ZNN, traditional GNN, and the proposed ACNPZNN (13) models for solving TVCVSE problem under the noise-free condition are shown in Fig. 2. The linear and logarithmic forms of the residual error || (t)|| 2 of the proposed ACNPZNN model (13) are illustrated in Fig. 2 (a) and (b), respectively. Obviously, it can be observed that the residual error (t) 2 of the ACNPZNN model (13) can globally converges to zero in noise-free condition, and the best of these three models in terms of convergence performance is the proposed ACNPZNN.

C. PERFORMANCE COMPARISONS AMONG DIFFERENT MODELS FOR SOLVING TVCVSE WITH DIFFERENT NOISES
To study the robustness of ACNPZNN model (13) and other models under different noise conditions, the real and imaginary parts of the complex-valued constant noise used to  (13) (red solid curves) model with non-convex bound activation function (46) polluted by complex-valued constant noises, respectively. It is apparent that, the ACNPZNN model (13) achieves better performance when perturbed by complexvalued constant noises. Figure 3 (c) and (d) demonstrate the linear and logarithmic representations of the residual error || (t)|| 2 of traditional ZNN model (blue dash curves), traditional GNN model (purple dotted curves), and the proposed ACNPZNN (13) (red solid curves) model with non-convex bound activation function (46) polluted by complex-valued constant noises, respectively. The proposed ACNPZNN coincides with Theorem 2. It is easy to see, in the case of bounded random noise interference, the ACNPZNN model (13) still maintains a reliable robustness.

VII. CONCLUSION
Aiming at effectively solving the TVCVSE problem, this paper proposes a residual-based adaptive coefficient and nonconvex projection complex-valued zeroing neural network (ACNPZNN) model. Compared with the constant scale parameter of traditional ZNN model, the proposed ACNPZNN model has achieved advanced performance in terms of the solution accuracy and convergence rate. In addition, under various noise conditions, the performance of the proposed ACN-PZNN model with the non-convex bound activation function is more robust or reliable. Besides, to further quantitatively investigate the robustness performance of the ACNPZNN model, two corresponding theorems have been presented. Finally, the corresponding experiments and simulations have been designed to further verify the superiority of the model. In summary, both the qualified simulation experiments and theorems have verified the high global convergence speed and reliability of the ACNPZNN model.