Affine Memory Control for Synchronization of Delayed Fuzzy Neural Networks

This paper deals with the synchronization of fuzzy neural networks (FNNs) with time-varying delays. FNNs are more complicated form of neural networks incorporated with fuzzy logics, which provide more powerful performances. Especially, the problem of delayed FNNs’s synchronization is of importance in the existence of the network communication. For the synchronization of FNNs with time-varying delays, a novel form of control structure is proposed employing affinely transformed membership functions with memory element. In accordance with affine memory control, appropriate Lyapunov-Krasovskii functional is chosen to design control gain, guaranteeing stability of the systems with delays. Exploiting the more general type of control attributed by affine transformation and memory-type, a novel criterion is derived in forms of linear matrix inequalities (LMIs). As a results, the effectiveness of the proposed control is shown through numerical examples by comparisons with others.


I. INTRODUCTION
Neural networks are a focusing issue not only from research areas but also from industrial applications due to its outstanding performance and versatileness [1]. Inspired by human brain, neural networks usually is structured upon multilayered with a lot of neurons and the activation functions. The usage of neural networks can be seen several areas including image processing [2], pattern recognition [3], [4], and time series prediction [5].
Even neural network is fruitful, every intelligent technology has certain computational properties that make them suitable for a specific problem rather than another [6]. Neural networks are good at recognizing patterns but not good at explaining how to arrive at a decision. Comparing to that, fuzzy logic systems that can infer from inaccurate information are good at explaining decisions, but cannot automatically obtain the rules to make decisions. These limitations The associate editor coordinating the review of this manuscript and approving it for publication was Nasim Ullah .
have become a key driving force in creating intelligent hybrid systems that combine two or more technologies in a way that overcomes the limitations of individual technologies [7]. Hybrid systems are also important when considering the various characteristics of the application domain [8]. Many complex domains have different component problems and thus each problem may require a different type of efficient processing.
Fuzzy neural networks (FNN) are the one of representative hybrid algorithm of two distinct technologically important fields: fuzzy logic and neural networks [9]- [11]. Integration of fuzzy logic into the structure of a neural networks takes advantage of both methods [12] and thus it can describe more complicated neural networks including uncertainties or vagueness. Hence, it can improve the accuracy of algorithm by utilizing fuzzy logic as a ensemble algorithm [13]. Combination of fuzzy logic-based ensemble with multi neural networks belongs to one of useful application of fuzzy neural networks. Moreover, there exist numerous applications with the advantage of FNNs: force control of robot manipulators [14], biomedical computing [15], road lane prediction [16], and so on [17]- [20]. Therefore, FNNs have been received a lot of attentions during last decades.
On the other hand, the problem of synchronization has been studied from many researchers in various fields including information process [21], secure communication [22], chemical reactors [23], and biological system [24]. For the investigation of dynamic behavior, synchronization is one of important and major issues. Naturally, the synchronization of neural networks has emerged as promising research issue. In view of the significance of the synchronization, we can see that there are many important works have been developed to make a response system is identical to a driver systems (synchronization) but early studies [25] has not consider the effect of time delay which is inevitable physical constraint in most of dynamical systems.
Time delays in systems are known to be main causes for instability, periodic oscillation, bifurcation, or chaotic behavior [26] so it should be taken into consideration in the design of a synchronization controller. The systems with timedelay are widespread, and unavoidable in most of practical systems. Furthermore, most systems are composed through wire/wireless network, which have inherently delays between sender and receiver. In accordance with the fact, several strategies are proposed for the synchronization of delayed neural networks: pinning control [27], impulsive control [28], intermittent control [29], state feedback control [30], adaptive control [31], dissipative finite-time control [32], sliding mode control [33] and so on [31], [34]. Those approaches are valuable in common synchronization problem but they are limited in that the systematic attributes of Fuzzy neural networks with delays are not fully taken into account. That is, the existing synchronization methods for delayed FNNs have not carefully considered the natural characteristics of the fuzzy membership functions incorporated with compensation for time delay. Thus, the performance of synchronization is less impressive.
Motivated by above discussions, we propose affine memory control for the synchronization of delayed FNNs. The proposed control is constructed as a more general structure with a additive control form of state feedback and memory feedback control. It should be noted that memory feedback can effectively compensate for the effect of time-varying delays [35]. Upon the additive structure, modified parallel distributed control (PDC) with affine transformation of membership functions sums up a weighted average of the individual control laws, which fully exploit the information of fuzzy systems. Based upon Lyapunov-Krasovskii functional, synchronization controller is designed utilizing parameterization and improved reciprocally lemma that can handle integrals of states. Thus, novel advanced synchronization design criteria using affine memory control is presented for the synchronization of FNNs with time-varying delays in forms of linear matrix inequalities (LMIs). The effectiveness of employing the more general type of affine memory control is shown through numerical examples.
The main contribution of the manuscript can be summarized as follows: • New synchronization criterion is proposed to pull dynamics of distinguished fuzzy neural networks (FNNs) in sync. So called ''affine memory control'' is firstly introduced to improve the convergence and performance of synchronization.
• The affine memory control has an specialized structure designed to fit in the synchronization of delayed fuzzy neural networks. The membership function, which is basis of the blending in fuzzy logic, is fully utilized with affine transformation. Thus, enhanced convergence is achieved comprising with comparable complexity.
• The memory structure of the proposed control well compensates for delays of FNNs. Time delays in synchronization is unavoidable in network communication. Furthermore, the considered FNNs with time-varying delays are more general type of systems including delays in connection part, and thus it is applicable for multiple time-varying systems.
• The controller design criterion is presented in forms of linear matrix inequalities based on Lyapunov-Krasovskii functions. Considering time-delays, it is derived with the aid of reciprocally convex inequality and a zero equality for the relationship between membership functions of the systems and affine transformed membership functions of the controllers.
Notations: Fairly common notations are used throughout the paper. A > 0 implies that a matrix A is a positivedefinite matrix. Similarly, A > B denotes that A − B is a positive definite. Sym(Y ) represents Y + Y T , and diag{d 1 , d 2 , . . . , d n } means a diagonal matrix with diagonal elements of d 1 , d 2 , . . . , d n .

II. PROBLEM STATEMENT AND PRELIMINARIES
Let us consider the model of fuzzy neural networks (FNN)s with IF-THEN rules as follows: (1) where ζ i is premise variable, i = 1, 2, . . . , r is index variable, r is the number of IF-THEN rules, and F ij is jth fuzzy set of ith rule for j = 1, 2, . . . , r, m(t) = [m 1 (t), . . . , m n (t)] T is the state vector of neurons. Under the given fuzzy rules with a inference w i (ζ (t)) = r l=1 F il (ζ l (l)) and center average defuzzification µ i (ζ (t)) = w i (ζ (t)) n i=1 w i (ζ (t)) , the dynamic relation VOLUME 9, 2021 of master FNNs is represented as M : where θ i (t) is the premise variables, y m is the output vector, A i for i = 1, 2, . . . , r is the self-feedback matrix, W 1 , W 2 are weighting matrices, J i (t) for i = 1, 2, . . . , r is the external input vector, C is the output matrix, f (·) is a nonlinear activation function satisfying Lipschitz condition, and h(t) is a time-varying scalar with known bounds. Without loss of generality, it is assumed that time-varying delay and it derivative are upper-bounded.
where h M is a given maximum value of time delay, µ is a maximum bound of delay-derivative with a given scalar. Then, taking the following subsystem as a slave system S : where u(t) is the control input. Widely used parallel distributed control (PDC) is denoted as where e y (t) = y m (t) − y s (t). Then, where K j is the control matrix. Instead of using the presented control method, affine memory control is newly designed for synchronization, which is in forms as follows: where ϑ k = T (θ k ) for k = 1, 2, . . . , r, T (x) = px + q is a affine transformation function, p > 0 and q are scalar parameters, and K j , K mj are the control gain matrix. For simplicity, θ j , ϑ j are used instead of θ j (µ), ϑ j (µ) throughout the paper. Remark 1: It should be noted that the proposed control is designed utilizing affinely transformation of each membership functions from those of FNNs. Taking more general membership function as a synchronization control, it much relaxes the conservative of the widely used parallelly distributed control which uses exactly same membership function from those of master systems.
The relations of fuzzy weighting parameters regarding membership functions satisfy the following conditions.
For the master-slave synchronization, the overall error dynamics is derived from (2), and (4).
where g(e(t)) = f (m(t)) − f (s(t)). Without loss of generality, the nonlinearity f (·) fulfills where l i for i = 1, 2, . . . , r are the known constant bounds. The bounds of slope are integrated into S = diag{l 1 , l 2 , . . . , l r }. Remark 2: Under the synchronization problem of error dynamics in FNN, there is a difficulty in inducing the relations between the membership function of the master systems and that of slave systems with error system dynamics. Therefore, there should be a method that can handle the differently designed membership function. The introduced Lemma 1. make it possible to separate the systems dynamics for each rules, and reconstruct the equations in (8)- (10). Some important Lemmas are introduced to derive the main results.
Lemma 1 [35]: For a given vector = [θ 1 , θ 2 , . . . , θ r ], and a transformed vector¯ = T ( ) where T (x) = px + q for a positive contant p, the following inequality is satisfied if < 0 where ⊗ denotes a Kronecker product. Lemma 2 [36]: For a given positive matrix R, and any matrices X , Y , Z , and a scalar α ∈ [0, 1], the following inequality holds for a smooth function where R a = diag{R, 3R}, The role of Lemma 2. is to get a tight upper bound of integral terms in the design procedure. When the systems with time delay are handled via time domain approach, it is unavoidable to have integral terms which contains the information of the delayed states. Lemma 2. is the efficient tool to approving integrals in the literature.

A. THE DESIGN OF AFFINE MEMORY CONTROL
In this section, the design methodology for synchronization of FNNs is presented. Before denoting the main results to have an appropriate control gain, some notations are defined for simplicity. e i denotes ith identity matrix with zero matrices.
In addition, M s = M + M T , L s = L + L T are defined. Under given notations, the main theorem is induced.
Theorem 1: Suppose that there exist given scalars α, p ∈ [0, 1], q, r, µ, κ, and positive-definite matrices P ∈ R 3n×3n > 0, Q 1 ∈ R n×n > 0, Q 2 ∈ R n×n > 0, R ∈ R n×n > 0, diagonal matrices 1 ∈ R n×n , 2 ∈ R n×n , any matrices where then, the error dynamics is stabilizable with the gain K j = G −1 H j and K mj = G −1 H mj for j = 1, 2, . . . , r, which implies that the state of slave systems asymptotically follows the state of master systems. where Using Lemma 2, the upper bound of integral is estimated. Then, it is easily derived.
From the sector-bounded condition of the nonlinear activation function in (12), it is obtained.
Taking the equation of error dynamics, the following zero equality holds Summation above equations for the variables θ, ϑ, we havė VOLUME 9, 2021 From (8) to (10), the following inequalities are derived.
The relations of the membership functions from (22) to (24) can be reconstructed for the following conditions using additional auxiliary matrices L, M , N i for i = 1, 2, . . . , r.
Then, C is easily derived. Applying S-procedure and Lemma 1,V (t) < 0 can be ensured if (29) is guaranteed, which ends the proof. Remark 3: The proposed affine controller is a more general type of controller that reflects the characteristics of the fuzzy system. The fuzzy systems are composed of several rules that is dependent on each membership functions. It should be noted that parallel distributed control (PDC) is a type of control which share the exactly same membership functions with those of systems. However, it is not guaranteed that the method of PDC is the optimal membership functions for the controller, and thus the performance is constrained. Thereafter, affine transformed controller is a wider class of controller that can enhance stability and performance of the given control systems.
Remark 4: As reported in Remark 1, affine controller is a wider class of controller. Thus, the gain of PDC controller also can be derived utilizing Theorem 1. When the parameter of affine transformation p = 1, q = 0, the condition of Theorem 1 can be directly applied for PDC. The other advantage of Theorem 1 is that is can have distributed gains for each rules, which was not well conducted in the previous researches.

B. THE DESIGN OF AFFINE CONTROL WITHOUT MEMORY
For the purpose of comparison, the following affine controller without memory is considered.
Proof (The Proof of Corollary 1): is quite similar with that of Theorem 1, so it is omitted here for brevity.
Remark 5: The proposed synchronization control can be regarded as static output feedback control when its simple equivalent form in linear systems are considered. Therefore, it has a simple structure and easy to apply in real applications. However, its performance could be limited since the order of controller is simple comparing to observer-based control. It can be compensated through the proposed affine memory control.
The overall flow chart is displayed in Fig. 1. Working along the flow diagram of Fig. 1, the synchronization controller can be calculated with denoted parameters.
Remark 6: The design method of affine memory control is constructed upon the parameterization of the membership function. To utilize relations between membership functions, the conditions are derived for the extended variables. Thus, the LMI condition, which is dependent on the separated membership functions, possibly lead to computational complexity as the inference rules are increases.

Remark 7:
Recently, the switching controller in T-S fuzzy systems has proven its effectiveness in [39]- [43]. Sophisticated Lyapunov-Krasovskii functional also can improve synchronization performances employing parameter-dependent function [44] or line integrals type [45]. The consideration with presented method in detailed manner is a possible direction of further research. The main contribution of this manuscript is the proposition of the novel controller form which is affine memory control, thus the results are focused on it.

IV. NUMERICAL EXAMPLES
Example 1: FNNs with the following parameters is presented.
where (1, 1, 1),  (1, 1, 1). The membership functions for each rule i = 1, 2 are selected as The dynamic behavior of the given FNNs is displayed in Fig. 1. For the design of affine memory control, the parameters are chosen as κ = 0.1, and p = 0.5. From the given parameters, the membership functions of master systems and the control are depicted as Fig. 1. When p = 1, q = 0 is selected, the membership functions of control are exactly same as that of error systems so it is parallel distributed control (PDC) scheme in literatures. Therefore, the proposed affine control method is a much wider class of control scheme. By using YALMIP toolbox to solve the LMI problem in Theorem 1, the matrices can be calculated that the LMIs are feasible. The solutions are given by The dynamic behavior of master system is displayed in Fig. 2.
In the Fig. 2, it is shown that the trajectories are in a oscillatory motion. Consequently, the synchronization gain of affine memory control is designed as follows:   The state trajectories of master and slave systems are presented in Fig. 3. As shown in Fig. 3, each states of subsystems tracks well the states of master systems. With the synchronization control, the controlled trajectories of error states are presented in Fig. 3. For the numerical simulation, an initial state condition is chosen as m(t) = [−1, −0.5, 1.5]. Example 2: FNNs with the following parameters are selected as:   For the given systems, affine control is utilized and the allowable upper bound of nonlinear function is compared with that of PDC to show the superiority of the proposed control. For the comparison of possible slope bound for nonlinear activate function, the matrix S = diag{l 1 , l 2 , . . . , l r } is selected as S = s · I where s is a scaling parameter. The value of µ is selected as 0.8. Then, the allowable upper bound is presented for the cases with various delay in Table 1. The  table is shown to display clearly compare the results of the proposed control.  It can be seen from the table that as the upper limit of the time delay increases, the maximum possible bound of nonlinear function decreases. This means that the two variables are inversely proportional. As a time delay and nonlinearity in control systems are main sources of instability, the increasement of them reduces the allowable area of stable regions. Therefore, it is an adequate indicator for the comparison of performances. From the Table 1, it can be seen from the comparison of the table that the proposed method shows an progressive improvement in average. Fig. 4 shows the guaranteed stable regions for activate functions with h M = 0.5. Widely used activate functions (Hyperbolic Tangent, Swish function, ReLU) are drawn with the regions. As shown in figure, the given activate functions are not available under parallel distributed control (PDC) with memory while affine memory control covers all activate functions.
In Table 2, the maximum allowable bound for time delay is calculated for each method. From the Table 1, it is shown that the proposed control method guarantees the wider area of stable region for time delay. Furthermore, it should be noted that the improved results of the guaranteed region play an advantageous role in performance of the system. Table 3 shows the maximum allowable slope bounds for given upper bound of delays. Then, it is shown that the superiority of the proposed control by comparisons with the results of the existing results in the literature. The results in [32], [46] is modified to the apply to given model of FNNs. From the given Table 2 and 3, it is clearly shown that the proposed method provides wider stable regions in synchronization of FNNs.
Example 3 (Secure Communication): Secure communication is widely used application of synchronization. The parameters for FNNs are taken as follows: The block diagram of secure communication is shown in Fig. 6. The original signal h(t) is sent through the network, and to safely send the message, it is encoded with the signal of master systems in forms of fuzzy neural networks. The encoder and decoder is composed of the states from the chaotic fuzzy neural networks, the message can't be reconstructed without synchronization of master and slave systems. Furthermore, chaotic fuzzy neural networks is very sensitive to an initial condition, it is hardly to be synchronized. The encoder signalĥ(t) is chosen aŝ Then,h(t) is reconstructed by designing the decoder as follows.h Applying Theorem 1, the synchronization controller can be easily obtained, and the result is presented in Fig. 7. As shown in Fig. 7, the recovered messageh(t) is the identical to the original message h(t) when the master-slave fuzzy neural network is fully synchronized. Moreover, the transmitted signal h(t) is very distinct from the original signal, so the secure communication is well conducted using the synchronization of fuzzy neural networks.  For given system matrices, additional results are provided to present the effect of each part in the proposed method. Under the given condition, the system is over-stable, so it is restricted with the matrix C = diag{1, 1, 0}. By choosing it, the result shows off the reasonable value for the apparent comparison. The simulation is conducted for the given parameters h M = 3, µ = 1.1, κ = 5.0. Under the condition, the allowable upper bound of slope is derived using Theorem 1 and Corollary 1. In Table 1, the result of Corollary 1 (p = 1, q = 0) is the one without using any developed memory & affine algorithm. Theorem 1 (p = 1, q = 0) is the one using only state feedback with memory element, and thus it shows off the effect of memory control. Theorem 1 (p = 0.5, q = 0) is the one using affine memory control. As shown in Table 4, it can be noticed that the proposed control is effective and widen the region of stability. Therefore, it proves the improved synchronization performance of the proposed method.

V. CONCLUSION
Affine memory control is newly proposed for the synchronization of fuzzy neural networks with time-varying delays. Utilizing transformation of membership function which is differ from that of master systems, improved affine fuzzy control is designed with compensation for the time-varying delay from memory control. Hence, the design criteria for the enhanced control is derived based on Lyapunov-Krasovskii functional. Improved reciprocally convex inequality and parameterization results in designing method in forms of linear matrix inequalities for compositive controller gains. Numerical simulations have shown the validity and superiority of the proposed control scheme. The proposed control scheme can be extended to more complicated type of fuzzy neural networks including fuzzy cellular network, fuzzy impulsive networks, and so on. More progressive results could be investigated through integration with switching type control or adaptive mechanism. As a further study, the analysis and enhanced conditions with model uncertainties and physical constraints such as network limitations and input saturation also could be a promising and attractive issue.