Active Sensing of Robot Arms Based on Zeroing Neural Networks: A Biological-Heuristic Optimization Model

Conventional biological-heuristic solutions via zeroing neural network (ZNN) models have achieved preliminary efficiency on time-dependent nonlinear optimization problems handling. However, the investigation on finding a feasible ZNN model to solve the time-dependent nonlinear optimization problems with both inequality and equality constraints still remains stagnant because of the nonlinearity and complexity. To make new progresses on the ZNN for time-dependent nonlinear optimization problems solving, this paper proposes a biological-heuristic optimization model, i.e., inequality and equality constrained optimization ZNN (IECO-ZNN). Such a proposed IECO-ZNN breaks the conditionality that the solutions via ZNN for solving nonlinear optimization problems can not consider the inequality and equality constraints at the same time. The time-dependent nonlinear optimization problem subject to inequality and equality constraints is skillfully converted to a time-dependent equality system by exploiting the Lagrange multiplier rule. The design process for the IECO-ZNN model is presented together with its new architecture illustrated in details. In addition, the conversion equivalence, global stability as well as exponential convergence property are theoretically proven. Moreover, numerical studies, real-world applications to robot arm active sensing, and comparisons sufficiently verify the effectiveness and superiority of the proposed IECO-ZNN model for the time-dependent nonlinear optimization with inequality and equality constraints.


I. INTRODUCTION
Solving the static (or to say, time-invariant) nonlinear optimization problems has a mature methodology due to its simplification and low requirement of real-time computation [1]- [3]. For examples, Hu and Zhang [1] introduced an effective recurrent neural network (RNN) model to address the time-invariant quadratic program problems with the convex condition and the capacity of global convergence. In addition, Nazemi and Nazemi [3] proposed an effective gradient The associate editor coordinating the review of this manuscript and approving it for publication was Jenny Mahoney. based neural network approach to handle the strict-convex quadratic program problems with detailed theoretical analyses. Recently, the studies on time-dependent problems become increasingly desirable and significant because of the ubiquitous involvement of complexity and requirement of real-time computation [4]- [7]. Differing from the time-invariant problems, time-dependent problems are much more difficult to be addressed for the fact that the problems involved, the system coefficients as well as the solutions are all time-dependent [8]- [11]. The conventional time-invariant methods and models generating the related time-invariant solutions would be invalidation due to the inevitable delay (or to say, the lagging behind) errors [12], [13]. Note that the conventional time-invariant methods commonly generate solutions at current moment based on the present known information. However, the time-dependent solutions change all the time. Therefore, the local time solutions would not be feasible solutions any more at the next time instant. That is the main reason that developing effective strategies or approaches for the time-dependent nonlinear optimization problems remains a challenging and significant topic in recent years [14]- [18].
Due to the fundamentality as well as universality of the time-dependent nonlinear optimization, a number of studies have been introduced on the development of such an issue together with different alternative approaches investigated [19]- [22]. Artificial neural network (ANN), as an intelligent system, is an important branch of artificial intelligence, which provides a strong impetus for the development of artificial intelligence. The ANN commonly referred to the neural network (NN), is a network designed and produced in imitation of the neural structure of the human brain and composed of a set of interconnected artificial neurons [23]. A mathematical model is built to describe the NN, and a computer program is used to simulate the learning mode similar to that of human brain. During the learning process, knowledge is obtained from the NN, and the obtained knowledge is stored according to the connection strength between neurons. In other words, the NN is a kind of biological-heuristic computation and optimization network aiming at simulating nerve cells or neurons in human or biological nervous system, which is an important methods to realize artificial intelligence [24]. Owing to the superiorities such as the parallel processing as well as feasibility on hardware implementation, the approaches and models by exploiting ANNs [25]- [32], especially the RNNs, have been deemed as the systematic solvers for the time-dependent nonlinear optimization [33]- [36]. Especially, the neural network models via the negative gradient direction strategy by constructing the scalar valued energy functions have been introduced and investigated for some online scientific problems handling [3], [37], [38]. For example, Chen et al. [37] developed an effective neurodynamics system by utilizing the conventional gradient search direction strategy to address the Lyapunov-matrix-equation problems. In addition, Yi et al. [38] developed an effective gradient neural network (GNN) model to handle the online solution of the Lyapunov-matrix-equation problems by supplying different activation functions. However, most solutions via the GNN models occur the above mentioned lagging behind errors, and shows a diverging property during the time-dependent nonlinear optimization problems solving [38].
Being quite different from the GNN by defining a scalar valued energy function, the zeroing neural network (ZNN) is a new kind of biological-heuristic optimization model [39]- [41]. The ZNN is able to selectively define a scalar valued, vector valued, or even matrix valued indefinite error function, which fully utilize the time-derivative information for handling the time-dependent problems [42]- [44]. Conventional solutions via ZNN models have achieved preliminary efficiency on time-dependent nonlinear optimization, and many systems and models on the basis of ZNNs have been generalized and developed [12], [21], [35], [45]- [47]. For instances, as an early attempt, Jin and Zhang [47] introduced an effective discrete time ZNN model to handle the time-dependent nonlinear optimization problems without consideration of inequality constraint or equality constraint. Afterward, Li et al. [21] developed a general square-pattern-discretization formula for the approximation of the first order derivative to handle the time-dependent optimization problems solving with only linear-equality constraints. Despite the preliminary success in the above studies on the time-dependent nonlinear optimization problems solving, the bottleneck still remains in the framework of ZNN. That is the time-dependent nonlinear optimization can not be handled acceptably subject to both the inequality and equality constraints due to the nonlinearity and complexity. The research on finding a feasible ZNN model for solving the time-dependent nonlinear optimization with inequality and equality constraints still remains stagnant.
The active sensing (or termed active motion control) of robot manipulators (or termed robot arms) can usually be described as the time-dependent nonlinear optimization problem solving incorporation with the performance index optimization, physical limits constraints, and primary tracking control tasks [48]- [57]. Robot arms change the real-time posture in order to achieve the effector sensing objective via the autonomous motion. Therefore, the active motion control of robot arms can be viewed on their active sensing. Conventional solutions to handle the active sensing of robot manipulators are usually described the primal active sensing problem as a time-dependent quadratic optimization problem, and solved by the primal dual neural network (PDNN) models [4], [14], [58]. For examples, Zhang et al. [4] developed an effective tricriteria optimization coordination active sensing approach for a dual arm robot to track different paths by leveraging the PDNN solver. In addition, Xiao and Zhang [14] introduced an effective repetitive-motion-planning approach resolved at the joint acceleration level to handle the so-called non-repetitive problems using the discrete time PDNN solver. However, it was theoretically proven that the PDNN solver for the solution of time-dependent quadratic optimization problem could only asymptotic converge to the time-dependent theoretical solution, instead of converging with the exponential property in a time-efficient manner.
To make new progresses on the ZNN for time-dependent nonlinear optimization problems solving, a novel biologicalheuristic optimization model, i.e., inequality and equality constrained optimization ZNN (IECO-ZNN) is proposed in this work. The proposed IECO-ZNN breaks the conditionality that the solutions via ZNN for nonlinear optimization problems can not consider the inequality and equality constraints at the same time. Firstly, the time-dependent nonlinear optimization problem subject to inequality and equality constraints is skillfully converted to a timedependent equality system by exploiting the Lagrange multiplier rule. Then, the design process for the IECO-ZNN model is presented together with the new architecture of the proposed model illustrated in details. Moreover, theoretical analyses on the conversion equivalence, global stability as well as exponential convergence property are rigorously presented. Finally, numerical studies, comparisons as well as the real-world applications to robot arm active sensing sufficiently verify the effectiveness and superiority of the proposed IECO-ZNN model for the time-dependent nonlinear optimization with both inequality and equality constraints. For better illustration, comprehensive comparisons of the proposed IECO-ZNN model with existing ones [1]- [3], [12], [19], [21], [33], [34] are presented in Table 1. To the best of the authors' knowledge, there is no an existing method or model for handling the time-dependent nonlinear optimization problem subject to both inequality and equality constraints, and successfully applying to robot active sensing with outstanding features of the proposed one.
The rest of the paper is structured as below. Firstly, the preliminaries of time-dependent nonlinear optimization problem formulation as well as conventional ZNN solutions are presented in Section II. Afterwards, the design process of the proposed IECO-ZNN model is presented together with theoretical analyses provided in Section III. Section IV illustrates numerical studies, real-world applications to robot active sensing as well as comprehensive comparisons. Section V concludes the paper. Before ending this section, the main contributions of the work are summarized and listed as below.
• This paper proposes a novel IECO-ZNN model for the first time to make new progresses on the ZNN for time-dependent nonlinear optimization problems solving, which breaks the conditionality that the solutions via ZNN models for nonlinear optimization problems can not consider the inequality and equality constraints at the same time.
• The time-dependent nonlinear optimization problem subject to the inequality and equality constraints is skillfully converted to a time-dependent equality system with the conversion equivalence guaranteed by exploiting the Lagrange multiplier rule.
• Complete theoretical analyses, numerical verifications as well as real-world robot active sensing applications sufficiently substantiate the validity, effectiveness and superiority of the proposed IECO-ZNN model for finding feasible solutions to nonlinear optimization problems, which makes new progresses in both theory and practice.

II. NONLINEAR OPTIMIZATION PROBLEMS AND CONVENTIONAL SOLUTIONS
In this section, by leveraging the Lagrange multiplier rule, the general description of time-dependent nonlinear optimization problems and the related ZNN solutions are presented as preliminaries and backgrounds. Generally, a time-dependent nonlinear optimization problem can be described as below: where objective function f (·, ·) : R n × [0, t f ] → R is a smoothly time-dependent nonlinear optimization function with the continuous second order derivative. Besides, f (·, ·) is convex with respect to time-dependent state vector x(t) ∈ R n for all t > 0. In addition, vector x(t) denotes the time-dependent states vector that is unknown, which needs to be solved. Conventionally, to handle the time-dependent nonlinear optimization problem (1), a differentiable-nonlinear-mapping function is usually defined as below: , t) denotes the ith element of (x(t), t) and i = 1, 2, · · · , n. The superscript T is the transpose of a matrix or vector. Afterward, the so-called capture points can be found via the following time-dependent set depicted in with the time instant satisfying The conventional solution via the ZNN model is usually obtained by defining a vector valued indefinite time-dependent error function as below: being the ith element of (t), for all i = 1, 2, · · · , n. Then, by exploring the ZNN design formula [59], [60]: which makes each element i (t) converge to zero. The parameter ζ denoting the ZNN predefined parameter for user to adjustment the convergence rate, and (·) : R n+m+p → R n+m+p denoting an activation-function vector mapping with each element being a monotonically-increasing odd function. Note that is the time derivative vector, and matrix H(x(t), t) ∈ R n×n denotes the Hessian matrix defined as below: respectively. With the assumption that the Hessian matrix H(x(t), t) is nonsingular for any time instant t, according to design formula (3), the ZNN model for handling the time-dependent nonlinear optimization problem (1) can be obtain as below: Model (4) can be explicitly rewritten as the dynamical equation with states vector x(t) as below: where superscript −1 denotes an inverse operator of a square matrix. Provided that the time-dependent nonlinear optimization problem is subjective to equality constraints, it can be further depicted in where h(x(t)) ∈ R m denotes the equality constraints with the rank of A(t) ∈ R m×n being always equal to m as well as vector b(t) ∈ R m . The goal of time-dependent nonlinear optimization is to find a feasible solution x(t) such that (6) with equality constraints holds true at any time instant t ∈ [0, +∞).
On the basis of the Lagrange multiplier rule [61], a Lagrange function can be defined in order to find a feasible solution x(t) of general nonlinear optimization (6): (7) where state vector λ(t) = [λ 1 (t), λ 2 (t), · · · , λ m (t)] T ∈ R m are the Lagrange multipliers that are corresponding to the equality constraint.
According to theoretical results related to the Lagrange multiplier rule [61], to handle the time-dependent of nonlinear optimization (6) with equality constraints equals to handle the following time-dependent of nonlinear optimization without constraints: Afterwards, the solution x(t) to the general time-dependent nonlinear optimization (6) can be obtained by solving the formulary as follows: which can be written as below: To handle the time-dependent nonlinear optimization (6) with equality constraints, an error function is defined still in a unified framework of ZNN as VOLUME 8, 2020 with the defending neural network states vector u(t) = [x T (t), λ T (t)] T ∈ R n+m , and utilizing ZNN design formula (3), one can readily obtain the ZNN model as with matrix K(u(t), t) describing as follows: Thus, it finally obtainṡ . (12) Unfortunately, the time-dependent nonlinear optimization subject to both inequality and equality constraints described as in a unified framework of the ZNN still remains unsolved.

III. IECO-ZNN SOLUTION AND THEORETICAL ANALYSES
Problem formulations for the time-dependent nonlinear optimization and conventional solutions via ZNN are presented in Section II without consideration of equality or inequality constraints as preliminaries. In this section, a novel IECO-ZNN model is proposed by consideration of both inequality and equality constraints. In addition, the conversion equivalence, global stability and exponential convergence are theoretical proven.

A. IECO-ZNN MODEL DESIGN
The remaining unsolved time-dependent nonlinear optimization (13) subject to both inequality and equality constraints in a unified framework of ZNN can be reformulated as the following vector form: where functions h(x(t)) ∈ R m and g(x(t)) ∈ R p denote the equality constraint and inequality constraints, respectively, with the rank of A(t) ∈ R m×n always equalling to m, and vector b(t) ∈ R m , matrix C(t) ∈ R p×n and vector d(t) ∈ R p . The goal of time-dependent nonlinear optimization with multiple types of constraints is to find a feasible solution x(t) such that (14) holds true at any time instant t ∈ [0, +∞).
On the basis of the Lagrange multiplier rule [61], in order to obtain a feasible solution x(t) for the nonlinear optimization (14), a Lagrange function can be defined as where state vectors ∈ R p are Lagrange multipliers that are corresponding to the equality constraint as well as the inequality constraint, respectively.
On the basis of theoretical results related to the Lagrange multiplier rule [61], to handle the time-dependent of nonlinear optimization with both inequality and equality constraints equals to handle the time-dependent of nonlinear optimization without constraints as below: Afterwards, the solution x(t) to the time-dependent nonlinear optimization (14) is obtain by addressing the formulary as below: which can be written as follows: As for the time-dependent nonlinear optimization problem solving, a vector valued indefinite error function can be defined in a unified framework of ZNN design process: with v(t) ∈ R p denoting a vector. One can further have The equation always holds true. Afterwards, it has The following variable substitution can be readily made: On can readily obtain where yields According to the above operation and variable substitution, it can be computed the time-derivative of error function (19) as below:ė (t) = ė 1 (t),ė 2 (t),ė 3 (t) T withė 1 (t) ∈ R n ,ė 2 (t) ∈ R m andė 3 (t) ∈ R p being respective as follows: where function-mapping (t) ∈ R p×p depicts in with operator diag(v(t)) : R p → R p×p for generating a p × p dimensional square matrix with the elements of vector v(t) ∈ R p on the diagonal, and each element of φ(·) : R p → R p denoting as below: In a unified ZNN design framework, by employing a dynamical design formula [59], [60]: the IECO-ZNN model for solving the time-dependent nonlinear optimization (14) subject to both inequality and equality constraints with the following dynamical equation which can be written as the following compact matrix form where matrix W (t) and vectorẏ(t) denote respectively as below: vector r 3 (t) = 0 and vector r(t) = [r 1 (t), r 2 (t), r 3 (t)] T . It is worth pointing out here that parameter ζ is an important convergence parameter for the proposed IECO-ZNN. Such a convergence parameter is predefined by practitioners. Theoretically, arbitrary values satisfying ζ > 0 ∈ R can be set. For the purpose of shorter convergence time, the value of predefined parameter ζ can be set as appropriately large as the hardware would permit in practical robot active sensing applications [62].
To make the proposed IECO-ZNN model (21) more computable, it can be reformulate as the following explicit form: with the ith (with i = 1, 2, · · · , n + m + p) neuron form of the proposed ZNN model being depicted as follows: where y i is the ith neuron of IECO-ZNN model (21), and time-dependent w ij is the ijth element of weight matrix W † (t).
Note that the first n elements of the neural network states y(t) are the solution of the time-dependent nonlinear optimization (14) with both inequality and equality constraints. The solution can be used as the joint control signalsθ (t) if the practitioners consider the application to the robot active sensing. For intuitively understanding and also for convenience of practitioners, the architecture of the proposed IECO-ZNN model (21) for solving the time-dependent nonlinear optimization (14) with both inequality and equality constraints with application to robot active sensing is illustrated in Fig. 1. As shown in Fig. 1, the proposed IECO-ZNN model (21) is a typical kind of RNNs, which is able to effectively handle (14) as well as the application to the robot active sensing.  (14) subject to inequality and equality constraints with application to robot arm active sensing.

B. THEORETICAL ANALYSES
To investigate the problem conversion equivalence, global stability and convergence property for the proposed IECO-ZNN model (21), theoretical analyses are presented in details. References [59], [60], [63]- [65] can be the supplementary materials. Firstly, two lemmas are presented to guarantee the existence of an optimal solution to general time-dependent nonlinear optimization (14). Lemma 1 [64]: If and only if there exist two vectors λ * (t) ∈ R m and κ * (t) ∈ R p such that the integrated vector y * (t) = [x * T (t), λ * T (t), κ * T (t)] T ∈ R n+m+p makes the Karush-Kuhn-Tucker (KKT) condition hold true: vector x * (t) ∈ denotes a KKT point as well as an optimal solution to general time-dependent nonlinear optimization (14).
Proof: It can be generalized from [64]. Lemma 2 [65]: Provided that general time-dependent nonlinear optimization objective function f (x(t), t) is a convex (or to say, time-dependent convex) at each time instant t e with the domain of x(t e ) denoted as e being a convex set for each t e for all x 1 (t e ), and x 2 (t e ) in the domain and all 0 ≤ ν ≤ 1 for the objective function satisfying the convexity inequality as (25) for any two points x 1 (t e ) and x 2 (t e ) in the domain e , and their line segment also belonging to e , i.e., x 1 (t e ) + (1 − )x 2 (t e ) ∈ e for all 0 ≤ ≤ 1, then vector x * (t) is the optimal solution to general nonlinear optimization (14) if and only if x * (t) is a KKT point of general time-dependent nonlinear optimization (14).
Proof: It can be generalized from [65].

Theorem 1 (Equivalence of Primal-Dual Problems Conversion for IECO-ZNN): Solving the formulary depicted in (18) for time-dependent nonlinear optimization (14) is equivalent to solving the set of equality system depicted in
where the ith element of function mapping ϒ + (·) : R p → R p denotes as with

v(t) ∈ R p denotes a vector, and v i (t) is the i the element of v(t).
Proof: It is required to prove that solving g(x(t)) ≤ 0, κ(t) ≥ 0, and κ T (t)g(x(t)) = 0, is equivalent to solving where is divided into two part as below.
Theorem 2: (Global Stability and Convergence of IECO-ZNN): Consider the time-dependent nonlinear optimization (14) subject to inequality and equality constraints. If a positive design parameter ζ > 0 and a monotonically increasing-odd activation function (·) are utilized, starting from an arbitrary initial neural network state y(0), then the closed-loop IECO-ZNN model (21) is globally stable in the sense of Lyapunov with the first n elements of state y(t) converges to an exact time-dependent solution x * (t) of the involved nonlinear optimization problem (14).
Proof: As for handling the time-dependent nonlinear optimization (14) subject to inequality and equality constraints, the neurodynamic equation of the closed-loop IECO-ZNN model (21) is depicted aṡ e(t) = −ζ (e(t)), (30) and the ith sub-system of (30) is further described as below: with predefined parameter ζ > 0, and (·) being a monotonically increasing-odd activation function with index i = 1, 2, · · · , p. Define a Lyapunov function candidate as Note that L(t) is positive definite in view of L(t) > 0 for e i (t) = 0, and L(t) = 0 for e i (t) = 0 only. Afterwards, the time derivative of L(t) is calculated: Due to the fact that (·) is a monotonically increasing-odd activation function, one can readily obtain: − (e i (t)) = (−e i (t)), which further yields: Therefore, it can be asserted the result thatL(t) is negative definite for time t ∈ [0, +∞) with predefined parameter ζ > 0. By applying the Lyapunov stability theory [63], the closed-loop IECO-ZNN model (21) is globally stable with each element of the error function e i (t) globally converging to 0. That is to say that the first n elements of state y(t) converges to an exact time-dependent solution x * (t) of the involved nonlinear optimization problem (14). This completes the proof. Theorem 3: (Exponential Convergence Property of IECO-ZNN): Consider the time-dependent nonlinear optimization (14) subject to inequality and equality constraints. If a positive design parameter ζ > 0 and a linear activation function, i.e., (e i (t)) = e i (t), are utilized, starting from an arbitrary initial neural network state y(0), then the first n elements of neural network state y(t) of the proposed IECO-ZNN model (21) is exponentially converges to an exact time-dependent solution x * (t) of the involved nonlinear optimization problem (14). VOLUME 8, 2020  Proof: Let us review the the ith sub-system of (30) as below:ė with a linear activation function, i.e., (e i (t)) = e i (t), utilized, it readily obtains: of which the analytical solution is obtained as follows: which evidently indicates that each element of error function e(t) is exponentially converges to zero with convergence rate being the predefined parameter ζ for the proposed IECO-ZNN model (21) activated by the linear activation function, with the first n elements of neural network state y(t) of the model (21) is exponentially converges to an exact time-dependent solution x * (t) of the involved nonlinear optimization problem (14). This completes the proof.

IV. NUMERICAL STUDIES, ROBOT APPLICATIONS AND COMPREHENSIVE COMPARISONS
Numerical studies are conducted via two time-dependent nonlinear optimization problems subject to inequality and equality constraints by exploiting the proposed IECO-ZNN model (21) in this section. Afterward, the real-world applications of time-dependent active sensing of PA10 robot arm under inequality and equality constraints is resolved. Finally, the comparisons with the conventional ZNN model as well as the GNN model are provided.
Example 2: Consider another time-dependent nonlinear optimization problem as follows: min. (cos(0.1t) + 2)x 2 1 (t) + (cos(0.1t) + 2)x 2 2 (t) + 2 sin(t)x 1 (t)x 2 (t) + sin(t)x 1 (t) + cos(t)x 2 (t) s. t. sin(4t)x 1 (t) + cos(4t)x 2 (t) = sin(4t), The above problem is reformulated as the compact-matrix form with time-dependent coefficients as The related numerical results synthesized by the proposed IECO-ZNN model (21) to handle the time-dependent nonlinear optimization problem (34) subject to inequality and equality constraints are illustrated in Fig. 3. Figure 3(a) and 3(b) present all the neural network states, i.e., x(t), λ(t) as well as κ(t) of the proposed IECO-ZNN model (21) during the time-dependent nonlinear optimization problem (34) handling. One can readily find that the time-dependent profiles of A(t)x(t) also comply with the equality constraint that is sin(4t)x 1 (t) + cos(4t)x 2 (t) = sin(4t) in Fig. 3(c). In addition, the profile of C(t)x(t) strictly complies with inequality constraint, i.e., C(t)x(t) = x 1 (t) + x 2 (t) ≤ 1.2 during the whole solving process (see Fig. 3(d)). The numerical results in this example verify that the time-dependent solutions x(t) synthesized by the proposed IECO-ZNN model (21) are the feasible solutions to the nonlinear optimization problem (34) subject to both inequality and equality constraints, which further demonstrates the effectiveness of the proposed IECO-ZNN model (21).

B. APPLICATION TO ROBOT ARM ACTIVE SENSING 1) APPLICATION 1
Let us consider a robot control problem in real world, i.e., the PA10 robot arm active sensing. It can be described as time-dependent nonlinear optimization problem subject to inequality and equality constraints as whereθ (t) is the joint control signal, and J (θ (t)) ∈ R m×n is the Jacobian matrix of end-effector, and the joint-velocity limits of robot are respectively set to beθ − = −0.4 rad/s andθ + = 0.4 rad/s. The end-effector is controlled to track a rosecurve-shaped path r d (t) = [r dX (t), r dY (t), r dZ (t)] T with each element in X-, Y-, and Z-axes are r dX (t) = ς cos(4πsin 2 (0.5π t/T d ))cos(2π sin 2 (0.5π t/T d )) − ς + 0.6891, r dY (t) = ς cos(π/6)cos(4π sin 2 (0.5πt/T d )) · sin 2 (2π sin(0.5π t/T d )) + 0.0069, r dZ (t) = ς sin(π/6)cos(4πsin 2 (0.5π t/T d )) · sin(2π sin 2 (0.5π t/T d )) + 0.1778, where the geometry parameter is set as ς = 0.06 m in this application. The above time-dependent nonlinear optimization problem subject to inequality and equality constraints can be reformulated as the compact-matrix form with coefficients as follows: x(t) = θ 1 (t),θ 2 (t), · · · ,θ n (t) T  The corresponding numerical experiment results of PA10 robot arm to track the rose-curve-shaped path synthesized by IECO-ZNN model (21) are presented in Fig. 4. Firstly, Fig. 4(a) presents motion process in the 3D view for the PA10 robot arm stably to track the rose-curve-shaped path during the whole task execution with the tracking task described as the equality constraint J (θ (t))θ (t) =ṙ d (t) and the joint-velocity limits described as the inequality constrainṫ θ − ≤θ (t) ≤θ + . Starting from an initial position, Fig. 4(b) illustrates the actual trajectory moves according with the predefined path and almost overlaps it in steady state, which verifies that the desired rose-curve-shaped path tracking task is completed well. This also indicates that the equality constraint is achieved successfully. In addition, Fig. 4(c) illustrates the joint control signal profiles that are constrained by the joint velocity limits, which depicted as the inequality constraintθ − ≤θ (t) ≤θ + . Finally, the position errors shown in 4(b) illustrate the high tracking control accuracy of the solution model. The above numerical experiment  results of PA10 robot arm in real-world application verify the effectiveness and availability of the proposed IECO-ZNN model (21) for the time-dependent nonlinear optimization problem subject to inequality and equality constraints, which possesses significant potential applications.

2) APPLICATION 2
Another time-dependent nonlinear optimization problem subject to inequality and equality constraints with application to PA10 robot arm can be reformulated as the compact-matrix form with time-dependent coefficients as x(t) = θ 1 (t),θ 2 (t), · · · ,θ n (t) T , A(t) = J (θ (t)), with helix-shaped pathl d (t) = [l dX (t), l dY (t), l dZ (t)] T , and each element in X-, Y-, and Z-axes are respectively defined as l dX (t) = τ cos(8π sin 2 (0.5π t/T d )) − τ + 0.6891, l dY (t) = τ cos(π/3)sin(8π sin 2 (0.5π t/T d )) + 0.0069, l dZ (t) = τ sin(π/3)sin(8π sin 2 (0.5π t/T d )) + 0.2t/T d where the geometry parameter is set as τ = 0.15 m in this application. Similarly, Fig. 5(a) shows motion process in the 3D view for the PA10 robot arm stably to track the helix-shaped path with the tracking task described as the equality constraint J (θ (t))θ (t) =l d (t) and the joint-velocity limits described as the inequality constraintθ − ≤θ (t) ≤θ + . Starting from an initial position, Fig. 5(b) illustrates the actual trajectory moves according with the predefined path and almost overlaps it verifying that the desired helix-shaped path tracking task is also completed well, which illustrates that the equality constraint is achieved successfully. Moreover, the joint control signal profiles that are constrained by the joint velocity limits, which is depicted as the inequality constraintθ − ≤θ (t) ≤θ + (see Fig. 5(c)). Finally, the related position errors show the high tracking control accuracy of the proposed solution model (see Fig. 5(d)). The above numerical experiment results of PA10 robot arm in real-world application verify the effectiveness and availability of the proposed IECO-ZNN model (21) for the time-dependent nonlinear optimization problem subject to inequality and equality constraints.

C. COMPARISONS WITH EXISTING SOLUTIONS
To show the superiority of the proposed IECO-ZNN model (21) for solving the time-dependent nonlinear optimization problem (14) in the case of inequality and equality constrained, the comparisons with the solutions via other existing neural network model are conducted. The conventional ZNN model for comparison is depicted in (12). Note that as a typical kind of RNN, many GNN models have been introduced and investigated as a feasible alternative for the online scientific problems solving. Specifically, for solving the time-dependent nonlinear optimization problem, a scalar-valued energy function is usually defined as E(t) = e(t) 2 E /2. Then, the GNN model can be constructed as  follows:ẏ where ρ is a predefined parameter for the GNN model. The compared results illustrated by neural network states x(t) and inequality constraints C(t)x(t) via the conventional ZNN model (12) and the GNN model (37) for solving the time-dependent nonlinear optimization problems (33) and (34) are shown in Fig. 6 as well as Fig. 7. As shown in these figures, the neural network states x(t) via both the conventional ZNN model (12) Fig. 2(a). Similarly, the profiles of x 1 (t) + x 2 (t) also pass over the inequality constraints as the time instants such as t ∈ [2.6, 2.7] s for ZNN model (12) and t ∈ [0, 0.2] s for GNN model (37), which is contrary to those results via the proposed IECO-ZNN model (21) in Fig. 3(d). Furthermore, the comparisons are also conducted in two real-world robot active sensing applications. As illustrated in Fig. 8 as well as Fig. 9, both the synthesized joint control signals exceed the joint-velocity limits via the conventional ZNN model (12) and the GNN model (37), which is contrary to those results via the proposed IECO-ZNN model (21) in Fig. 4(c) and Fig. 5(c) in two applications. The above comparison results all verify that the proposed IECO-ZNN  model (21) that is able to effectively handle the inequality and equality constraints simultaneously is superior to the conventional ZNN model (12) and the GNN model (37).

V. CONCLUSION AND FUTURE WORK
To make new progresses on the ZNN for time-dependent nonlinear optimization problems solving, this paper has proposed a novel biological-heuristic optimization model, the IECO-ZNN (21), which breaks the conditionality that the solutions via ZNN for nonlinear optimization problems can not consider the inequality and equality constraints at the same time. The time-dependent nonlinear optimization problem subject to the inequality and equality constraints has been skillfully converted to a time-dependent equality system by exploiting the Lagrange multiplier rule. The design process for the IECO-ZNN model (21) has been provided together with the new architecture of the proposed model illustrated in details. In addition, theoretical analyses on the conversion equivalence, global stability as well as exponential convergence property have been rigorously presented. Furthermore, numerical studies, real-world applications to robot arm active sensing together with comprehensive comparisons have sufficiently substantiated the effectiveness as well as superiority of the proposed IECO-ZNN model (21) for the time-dependent nonlinear optimization with inequality and equality constraints. Future work lies in the following facts: i) in-depth investigation on the robustness as well as the convergence performance of the proposed IECO-ZNN for the active sensing of robot arms; ii) further implementation of the proposed IECO-ZNN on hardware via the field programmable gate arrays as the advanced controllers for active sensing of robot arms; and iii) further development of a complete biological-heuristic optimization models library with different biological-heuristic algorithms. As a final remark of the paper, to the best of authors' knowledge, this is the first research to simultaneously address both the inequality and equality constraints for the time-dependent nonlinear optimization problem in a unified framework of ZNN with successful applications to robot active sensing.