A Non-Convex Activation and Noise-Suppressing Zeroing Neural Network for Finding Time-Varying Matrix Square Root

In this paper, a zeroing neural network model with properties of non-convex activation and noise suppressing (NANSZNN) is proposed for finding the square root of a time-varying matrix. In comparison with the existing zeroing neural network models, the proposed NANSZNN model relieves the limit of convex constraint and makes a breakthrough in noise-suppressing. Furthermore, theoretical analyses and strict proofs are provided in detail for the global convergence and noise-suppressing performance of the proposed NANSZNN model under the circumstances of various noises by using Lyaponov stability theory. Finally, numerical experiments and comparative analyses are offered to further illustrate the availability, effectiveness, and robustness against different noises of the proposed NANSZNN model.


I. INTRODUCTION
Solving the matrix square root problem has been set an essential research direction in many scientific and engineering areas, e.g., control and optimization theory [1], Kalman filtering, and signal processing [2], [3]. In the recent years, many predecessors have a lot of effective algorithms to deal with the matrix square root problem [4]. In general, conventional algorithms can only be developed to solve the square root problems related to static matrices, but they are not competent to solve the problems related to time-varying (or dynamic) matrices. In fact, most existing algorithms cannot be directly applied to time-sensitive areas.
However, recurrent neural networks (RNNs) get through parallel distributed nature and probable hardware The associate editor coordinating the review of this manuscript and approving it for publication was Dost Muhammad Khan . implementability to acquire an accurate solution related to time-varying matrices [5]. Therefore, it is natural to develop some RNNs to solve the time-varying matrix square root (TMSR) problem [6]. It is the most remarkable is that enormous researches have been devoted to solving the TMSR problem with gradient neural networks (GNN) [7], but these traditional program is difficult to eliminate the lag error, which result in its inadequate accuracy in dealing with largescale time-varying issues, even the disintegration of the solution system.
It is known that the traditional matrix square root problem is commonly solved based on static algorithms, but these algorithms cannot be combined with any time-varying information [8]. To consider time-varying information, Zhang et al. present an original zeroing neural network (OZNN) model [9], which can zero the error function using the time derivative information and online find the solution of the concerned problem. Therefore, the constructed models for solving TMSR problem based on the OZNN model greatly promote efficiency and improve accuracy [10]. However, the accuracy of the traditional OZNN model will be greatly reduced under various disturbance noises [11]. Besides, its convergence rate and solution accuracy usually cannot satisfy the engineering requirements [12]. Hence, in recent years, many researches aim to improve the performance of OZNN model [13], [15]. Xiao et al. designed another finite-time zeroing neural network (FTZNN) model [14] to solve the problem, which is improved based on the OZNN model and immensely accelerates the error to globally converge 0 within a finite time. On this basics, Li et al. built a nonlinearly activated sign-bi-power function into the OZNN model to improve the convergence rate, which is the finite-time convergent ZNN (FTCZNN) model [16].
After these improvements, the accuracy and convergence rate of the system is improved significantly and much faster than the OZNN model. However, in practice, the shortcomings of the model are obvious, and the biggest one is that it is quite sensitive to disturbance noises. That is to say, its solution accuracy will descend distinctly in the existence of disturbance noises [17]. Because various noises are unavoidable in engineering applications [18], Liao et al. design a predefined-time convergent ZNN (PTZNN) model [19], which combines two novel activation functions. Meanwhile, the predefined-time convergence properties of the PTZNN model has better performance than the other models under perturbation conditions. Although the PTZNN model has the aforementioned advantages, it still has several inevitable defects. For instance, the scale parameter of the PTZNN model is stationary, which means that we need to manually set and adjust the parameter according to the actual situation. Jin et al. devise the modified ZNN (MZNN) model in [20], and the integral information is introduced into the OZNN model for the first time. Being compared with the PTZNN model, the MZNN model increases the capacity in resisting noise. On the basis of these work, a novel Newton-Raphson iteration algorithm is presented in [22] to further enhance the stability and robustness. Nevertheless, the aforementioned models are limited to the activation function of convex constraint. In this paper, combining with the advantages of the aforementioned zeroing-type neural network models, a non-convex activation and noise-suppressing ZNN (NANSZNN) model is proposed and applied to solve the TMSR problem under disturbance noises for the first time. Especially, performance comparison among the existing state-of-art models and the NANSZNN model for solving the TMSR problem is shown in Table 1.
The rest of this paper is expressed in five sections. The problem preliminaries and benchmark scheme for the TMSR problem is arranged in Section II. Section III proposes the design framework with non-convex activation function and noise-suppressing performance of the NANSZNN model. In Section IV, the global convergence and robustness of the NANSZNN model under various disturbance noises are proved theoretically. Section V shows the simulative experiments and research results in figures and tables, and the corresponding analyses are given in detail. In section VI, a time delay of arrival (TDOA) dynamic localization scheme is proposed based on the NANSZNN model. At the end of this paper, the conclusion and the future work is given in Section VII. The main contributions of the paper are as follows.
• A non-convex activation and noise-suppressing mechanism are presented to solve TMSR problem for the first time, which accelerates the convergence rate and improves the robustness of the ZNN model.
• We prove the convergence and robustness of the proposed NANSZNN model under noise-free or various disturbance noise circumstances with three mathematical theorems.
• We design and perform a series of numerical experiments to illustrate the superior performance of the VOLUME 10, 2022 convergence and computational accuracy of the NAN-SZNN model under different disturbance noises.
• A new dynamic localization scheme with more convergence speed and robustness is proposed, which verify the practicality and validity of the NANSZNN model.

II. PRELIMINARIES AND RELATED SCHEME FORMULATION
In this paper, we describe the TMSR problem [23] as the following form: where t ∈ [0, +∞] is the time of the system; the time-varying matrix A(t) ∈ R n×n is positive definite; the unknown time-varying matrix X (t) ∈ R n×n is to be acquired. For the traditional ZNN system, the error function of the corresponding problem is set out to facilitate the solution [24], [25]. Aiming at finding the solution matrix X (t), we make the hypothesis of the error function as According to the design framework of the OZNN model [9], the evolution direction of above function (2) should satisfy that:Ė where the constant κ ∈ R + represents a scaling coefficient and (·) : R n×n → R n×n is the projection function. Further, for solving the TMSR problem (1) the OZNN model can be represented aṡ For further analysis, equation (4) can be vectorized by Kronecker product operation [28]. Then, the OZNN model dealing with the TMSR problem (1) can be written as (5) where the symbol ⊗ denotes the Kronecker product; I ∈ R n×n is an identity matrix; vector x(t) = vec(X (t)) ∈ R n 2 ; vector a(t) = vec(A(t)) ∈ R n 2 . The operator vec(·) represents the vectorization of a matrix.

III. NANSZNN MODEL CONSTRUCTION
As mentioned in [21], the traditional RNNs can not well tolerate noises, which leads to reducing the accuracy of the system, or even collapsing the solution system. Therefore, in this section, a NANSZNN model is proposed to strengthen the robustness of the solution system under a variety of disturbance noises. Then, rigorous theoretical analyses and proofs are presented.
Specifically, the evolution formula of the NANSZNN model based on the error function E(t) can be directly denoted aṡ where the known scaling factors ξ > 0 and κ > 0, P (1) (·), P (2) (·) ∈ R n×n → R n×n , and E(t)P (1) (E(t)), E(t)P (2) (E(t)) > 0 represent two different activation functions, which can be convex or non-convex projection functions (PFs). In general, there are some commonly used convex and non-convex projection functions as follows.
Meanwhile, combining the proposed NANSZNN model (6) with TMSR problem (2), the the new construction of the formula can be obtains as follow: For the convenience of calculation, the corresponding vectorization principle is expressed as following: where the symbol ⊗ denotes the Kronecker product; I ∈ R n×n is an identity matrix. Furthermore, the proposed NANSZNN model (6) solving the TMSR problem can be vectorized as In order to simplify the corresponding representation during the theoretical analysis process, we define e(t) = (X T (t) ⊗ I ) x(t) − a(t) ∈ R n 2 , and then the following equivalent of the proposed NANSZNN model (14) can be given where, p(·) : R n 2 → R n 2 is the vectorized form of projection function P(·) : R n×n → R n×n .
However, an actual solution system is inevitably influenced with various disturbance noises. Generally, a TMSR problem with unpredictable noises can be represented as follows. (16) where the noise item (t) ∈ R n 2 .
In general, the convergence performance of a solution system is usually the most vital in practice. Then, a guaranteed theorem is presented as follows.
Theorem 1: For any solvable TMSR problem (1), the computational solution X (t) (vectorized form denoted as x(t)) of the proposed NANSZNN model (15) globally converges to the theoretical solution of the TMSR problem (1) from any random initialization state.
). Then, combining these two formulas with equation (15) gives˙ . Furthermore, the positive definite candidate Lyapunov function can be defined as w 1 . 2 2 , then, it can be reformulated as which implies that 0 Thus, the inequality can be expressed as Therefore, m(t) is globally convergent to zero. In addition, basing on LaSalle's invariance principle and considering the above analysis results that, lim t→∞ m(t) = lim t→∞ ( e(t) + κ t 0 p (1) ( e(ν))dν) = 0, we can get lim t→∞˙ e(t) = − lim t→∞ κ p (1) ( e(t)) directly from equation (15). After that, we define the second positive definite candidate Lyapunov function as w 2 (t) = e T (t) e(t)/2 and its time derivative is ). According to the definition of p (1) (·), the proof of w 2 (t) can be similarly derived from the steps above.
In summary, the proposed NANSZNN model (15) can globally converge to the theoretical solution of the TMSR problem (1). The proof is thus completed.

IV. ROBUSTNESS OF THE NANSZNN MODEL UNDER DIFFERENT NOISES
The robustness performance of NANSZNN model (15) under different kinds of disturbance noises is analyzed in this section. The following two theorems are provided to analyze the robustness of NANSZNN model (15) perturbed by constant noises and random noises, respectively. Before the proof of the following theorem, some related operations are predefined at first. We define f ≺ h denotes that each element of h is larger than that of f . Meanwhile, f h denotes that each element of h is less than that of f . Theorem 2: Giving the upper and lower bounds of the bounded projection function p (2) (·) is p (2) and p (2) respectively. Considered that 0 (t) ξ p (2) or 0 ≺ (t) ≺ ξ p (2) , in the existence of unknown constant noise (t) = d ∈ R n 2 , the residual error || e(t)|| 2 of the proposed NANSZNN model (15) with bounded activation functions globally converges to 0 for p (2) is an auxiliary equation as same as Theorem 1, i.e., m(t) = e(t) + κ t 0 p (1) ( e(ν))dν.
Proof: To prove the convergence ability of NANSZNN model (15) with the constant noise, i.e., (t) = d. Similar as the proving process on Theorem 1, we can obtain: where d = 0 is the unknown constant noise and p (2) (·) is a vector-wise bounded non-convex activation function. Specifically, the i-th element of system (19) iṡ where q (2) (·) : R → R is the element-wise activation function extracted from the above vector-wise activation function p (2) (·).
Defining a candidate Lyapnuov function l i (t) = m 2 Then,l i (t) can be written as, The In theory, to ensure that the equationl i (t) is negative definite, three situations will happen: a) q (2) will be analyzed in detail as follows.
• In the (c) case of q (2) In this case, the value ofṁ(t) will decrease while the absolute value of q (2) (m i (t)) increases at the same time.
Considering that the projection function is bounded, one has d i ≤ ξ q (2) i (the i-th element of ξ p (2) is ξ q (2) i ). There is always a time instant t, at whichṁ i (t) decreases untilṁ i (t) = 0 and the system (20) will return to the (a) case and gradually maintain stability. Otherwise, for the situation of d i > ξ q (2) , lim t→∞ṁi (t) > 0 and thė m i (t) will diverge over time. 3) In the condition of m i (t) < 0, the analysis in this case is similar to m i (t) > 0, and it is omitted.  Thus, we can figure out that, given that 0 ξ p (2) or 0 ≺ ≺ ξ p (2) , in the existence of unknown constant noise , the residual error of the proposed NANSZNN model (15) with bounded projection function converges globally to zero for p (2) ( m(t)) − (t)/ξ ≥ 0. The proof is thus completed.
The following Therome 3 is provided to analyse the robustness of the proposed NANSZNN model (15) under the bounded random noise (t).
Theorem 3: The residual error E(t) of the proposed NAN-SZNN model (6) are bounded under the interference of bounded random noise. VOLUME 10, 2022 Proof: When projection functions P (1) (·) and P (2) (·) become the linear projection functions, equation (6) can be reformulated aṡ Following vector-form random noises, the i-th element of (22) is written asḟ with P being [−(ξ + κ), −ξ κ; 1, 0]; the i-th element of (t) is i (t); and H = [1, 0] T . According to the formula of the general solution of the first-order linear differential equation, the solution of the equation (23) can be expressed as where D is a constant, and P is a constant matrix. When, t = 0, D = f i (0). Thus, we gain Associating with the triangle inequality, we can deduce • When (ξ + κ) 2 > 4ξ κ, that is ξ = κ, the geneeral solution of the error function can be expressed as, where c 1 and c 2 are constant. When t = 0, c 1 + c 2 = E i (0) can be obtained directly. And then we set c 1 = Thus, the solution of f i (t) can be expressed in the following form, and In this case, l 1 = (−(ξ + κ) Further, two subcases are considered as follows.

V. SIMULATIONS
To better describe the performance of the NANSZNN model (15), comparative simulation experiments are conducted in this section. As the aforementioned discussions, the ZNN related models, such as OZNN model [9], NCZNN model [29], and PTCZNN model [19], have a benign performance in the TMSR-solving task (1). Thus, we compare these models with the proposed NANSZNN model. Noting that all of the simulative experiments are managed via MATLAB R2018a on a computer with Intel Core i5-7300HQ@2.50 GHz CPU, 8 GB memory, NVIDIA GeForce GTX 1050 GPU, and Windows 10 operating system.

A. STATIC EXAMPLE
In this example, a static positive definite matrix A ∈ R n×n is used as the known matrix. As same as [16], we assume the known static matrix A as a Toeplitz matrix, which is constructed as follows.
a 2 a 3 · · · a n a 2 a 1 a 2 · · · a n−1 a 3 a 2 a 1 · · · a n−2 . . . . . . . . . . . . . . . a n a n−1 a n−2 · · · a 1 where a 1 = 1 + sin(2) and a k = cos(2)/(k − 1) with k = 2, 3, . . . , n. For simplicity purpose, we define n = 5 and n = 10, respectively, for the simulative experiments. At the same time, the projection functions P (·) of the proposed NANSZNN model (15) are set as equations (7) and (8), respectively. The experimental results of the proposed model and other state-of-art ZNN related models under a noise-free environment are compared and shown in Fig. 2. From the visualized results in Fig. 2(a), the proposed NANSZNN model (15) evidently has the fastest convergence rate among the involved models. In addition, it is worth noting that, results in Fig. 2(b) indicate that, with the increasing of the matrix dimension, the proposed NANSZNN model (15) still has excellent convergence performance, and its stability is remarkable compared with other ZNN models.

B. TIME-VARYING EXAMPLE
In this experiment, a known time-varying matrix in the TMSR problem (1) are constructed as same as [30].
Correspondingly, we can obtain its theoretical time-varying square root For fair comparison purpose, the projection functions and other parameters are set as same as Example 1.

1) NOISE-FREE SITUATION
The simulation results of comparisons among the proposed model (15) and other state-of-art ZNN related models in the noise-free environment are shown in Figs. 3 and 4. Figure 3 visualizes the solved elements of the matrix square root of the known matrix A(t). We can easily find that, the corresponding results of the proposed NANSZNN model (15) converge more quickly to the theoretical solution with time less than 0.1 s.
Then, the residual error e(t) F of different models in linear and logarithmic coordinates are shown in Fig. 4 depicting their solution accuracy and convergence time, from which, when facing with the noise-free environment, the proposed NANSZNN model (15) has the first-ranking convergence speed and swiftly converges to the order of 10 −4 within 0.1 s, outperforming other models. Particularly, the NCZNN model has the lowest precision. In summary, the proposed NAN-SZNN model (15) has improved both the solution accuracy and the convergence speed compared with other models.

2) VARIOUS NOISE SITUATIONS
As for the analysis of Fig. 5, from comparisons among the OZNN model, NCZNN model, PTCZNN model, and the NANSZNN model, it is evident that the proposed NANSZNN model (15) has excellent anti-noise capability. Based on different noise situations, analysis of them will be carried out and discussed in detail. As appeared in Fig. 5(a) (15) is better than other models distinctly. Therefore, we can draw a conclusion which is that the proposed NANSZNN model (15) has superior accuracy, robustness and stability when facing random noises and linear noises compared with other models.

VI. APPLICATION OF LOCALIZATION
In this section, to reflect the feasibility and excellent performance of the proposed NANSZNN model (15), we apply the NANSZNN model (15) to a practical problem of localization based on the time delay of arrival (TDOA) algorithm. Given the propagation speed of the signal and the coordinates signal receivers, the TDOA algorithm can accurately calculate the localization of the signal source by measuring the time difference between the signals received by different signal receivers. Therefore, TDOA algorithm can effectively solve the positioning problem. Meanwhile, noting that all of the simulative experiments are managed via MATLAB R2018a on a computer with Intel Core i5-7300HQ@2.50 GHz CPU, 8 GB memory, NVIDIA GeForce GTX 1050 GPU, and Windows 10 operating system.
According to the TDOA algorithm, we define the relevant mathematical symbols as shown in Table 2.
Basing on the above definition and the derivation of previous papers, we can get, where the number of receivers should be satisfy m ≥ 4, and then i = 3, . . . m. Further, we have and then, Thus, the equation (34) can be simplified as following linear equation, Finally, we define the error function to be e(t) = H (t) p(t) − c(t). Then the proposed NANSZNN model (15) for locating the signal source problem can be reformulated as, In the application, the number of signal receivers m is set to 4 and the propagation speed of the signal v is set to the speed of sound 340.29 m/s, with κ = 10 and ξ = 50. And then, the experimental simulation results of the proposed NANSZNN model (15) is shown in Fig. 6. Fig. 6 describes the actual positioning results of the proposed NANSZNN model (15), where the circles represent signal receivers, the black solid line denotes the actual trajectory and the red dotted line with the diamond is the predicted trajectory. It can be seen from the Fig. 6 (a) that the predicted trajectory basically fits the actual trajectory. No matter what the initial value is, the predicted trajectory always tends to the actual trajectory of the target. When the predicted path is fitted to the actual path, the proposed NANSZNN model (15) always accurately predicts the trajectory of the object in a dynamic time. This shows that the system has the outstanding solving ability and low delay for the localization of the signals source. Besides, in the Fig. 6 (b), the residual error is kept within 10 −5 m and can maintain global stability. Meanwhile, the system error can globally converge to zero within 0.5 s. The proposed NANSZNN model (15) still has excellent positioning ability for the 3D application of localization. Set the number of signal receivers m = 10 with κ = 10 and ξ = 50. In this case, from Fig. 7 (a), even if the error between the initial value and the theoretical value is large, the proposed NANSZNN model (15) can reduce the error in a very short time, so that the predicted value is close to the theoretical solution. When the signals source moves in 3D space, the system can locate the signals source accurately and keep excellent robustness in the dynamic time. On the whole, whether it is two or three dimensions, the conclusions can be   drawn that the path predicted by the proposed NANSZNN model (15) coincides with the actual path. Moreover, the residual error generated by the system is within 10 −5 m and globally converges in a short time.
On the whole, in the positioning problem, the proposed NANSZNN model (15) has preeminent performance and efficient practicability.

VII. CONCLUSION AND FUTURE
In this paper, a non-convex activation and noise-suppressing zeroing neural network (NANSZNN) model has been proposed for finding the time-varying matrix square root. Different from the existing zeroing neural network models, the proposed NANSZNN model has been redesigned to liberate the limit of convex activation function of the solution system and simultaneously improve the robustness of the traditional models. Static and time-varying simulative experiments substantiate that, the proposed NANSZNN model not only has a significantly faster convergence rate than other common zeroing neural network models, but also maintains global convergence and robustness under constant, random, and time-varying noise conditions. Additionally, the proposed NANSZNN model is applied to dynamic localization effectively to prove its practicability.
In the future, we will apply the adaptive coefficient in the proposed NANSZNN model, so as to improve the final accuracy and noise-suppressing performance. Besides, investigating more non-convex activation function to hence speed up the convergence of the proposed NANSZNN model. In order hand, extending the NANSZNN model to solve the multitarget real-time detection and location problems are our main future research direction.