Dynamics of Fractional-Order Neural Networks With Discrete and Distributed Delays

This paper is concerned with the stability and Hopf bifurcation of fractional-order neural networks with discrete and distributed delays. The novelty of this paper is to take into account the discrete time delay and the distributed time delay for fractional-order systems. By introducing two virtual neurons to the original network, a new four-neuron network only involving discrete delays is formed. The sum of discrete delays is adopted as the bifurcation parameter to demonstrate the existence of Hopf bifurcation. It is found that the critical value of bifurcation can be effectively manipulated by choosing appropriate system parameters and order. Finally, numerical simulations are executed to substantiate the theoretical results and describe the relationships between the parameters and the onset of bifurcation.


I. INTRODUCTION
As is known to all, the neuronal system is a complex nonlinear dynamic system. And the neuron is considered the basic processing unit that has simplicity and simulation. Neurons can generate and transmit actions, encode and decode information, and complete other neural signal processes through the vibration and turbulence process in discharge activity. Once the neuronal system is maladjusted, the physiological mechanism will be abnormal or even chaotic, eventually leading to the emergence of neurological diseases [1]. For example, epilepsy and Parkinson diseases are dynamic neurological disorders. From the perspective of nonlinear dynamical systems, the nature of dynamic disease is related to bifurcations caused by altering the regulatory parameters in the neuronal system. Parkinson disease, for example, is formed by excessive synchronization of the basal ganglia, leading to intensify the local potential field oscillations, and the Hopf bifurcation occurs simultaneously. Therefore, it is extremely important The associate editor coordinating the review of this manuscript and approving it for publication was Zhixiang Zou .
to seek the specific parameter intervals of bifurcation for the treatment of dynamic diseases.
In the past decade, much investigation on neural networks(NNs) [2]- [6] has sprung up ever since its first simplified model was put forward by Hopfield in [7]. It is now firmly established that NNs can be applied into associative memory, pattern recognition and artificial intelligence. In fact, double-neuron networks exhibit the dynamic behavior in accord with the multi-neuron ones, which can be served as prototypes so as to enhance understanding about the dynamics of complex multi-neuron networks. Furthermore, because the duplex structure consumes so little energy, the physiological brain has been imitated by creating different types of memory-based circuits [8], [9].
(1) VOLUME 8, 2020 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ They analyzed different states with time delays, and revealed that the time delay exerted significant influence on system dynamics. Henceforth, more and more scholars set out to construct artificial neural network models with time delay so as to get close to biological neural networks. Hence, plenty of results about the dynamics of NNs have been obtained [11]- [17]. For example, based on [10], Li and Hu [14] considered the following two-neuron system with discrete and diagonal distributed delays The direction of Hopf bifurcation and stability of bifurcation periodic solutions were given. Compared with system (2), Karaoglu et al. [15] proposed a more general neural network system that has different discrete delays and activation functions, but with only one distributed delay. Additionally, several studies have developed low dimensional systems into high dimensional ones concerning merely discrete delays [16], [17]. However, these works only dealt with integer-order models of neural networks.
With the development and applications of the fractional calculus, researchers have come to realize that the superiority of fractional-order derivatives. On the one hand, fractional derivatives can describe the memory and hereditary effect of different processes. On the other hand, the order is variable, which means that fractional-order systems have unlimited memory. Thus, fractional-order calculus have been widely studied in diverse disciplines, including engineering [18], [19], physics [20], [21], biology [22]- [24], economics [25], [26], control [27], electromagnetism [28] and so on. However, the fractional-order systems are more complex than integer-order one in terms of properties and analytical processes.
Recently, fractional calculus has been applied in various fields, such as fractional-order genetic regulatory networks [29], fractional-order congestion control algorithms [30], fractional-order predator-prey models [31], [32], and fractional-order neural networks [33]- [43]. Combining with fractional calculus, the fractional-order neural networks (FNNs) will better reflect the memory and genetic characteristics. In [33] and [34], Kaslik and Sivasundaram proposed fractional-order neural networks of Hopfield type with different structures. What's more, a variable-order fractional operator was introduced into NNs in [35]. Also, Song and Cao [36] investigated the FNNs and provided the existence and uniqueness of the nontrivial solution. Moreover, the authors analyzed a category of complex-valued FNNs with hub and ring structured, and presented the conditions of Hopf bifurcation [37]. However, previous works ignored the time delay [33]- [37]. Yang et al. [38] discussed the stability of FNNs without and with discrete delay respectively, and established an LMI-based uniform stability condition. Furthermore, Xu et al. [39] considered a double-neuron FNNs with two discrete delays, discussed four possible cases with delays, and finally revealed the effects of different time delays on the stability of networks. Chen et al. [40] discussed a category of FNNs, but only dealt with discrete delay. It should be noted that only discrete delay was considered in [38]- [40]. Different from previous studies, we will take into account a fractional-order neural system with distributed and discrete delays. As a consequence, our neural system not only embodies the genetic and memory characteristics of neural networks, but also reflects the imbalance of delay during information transmission. This paper's main highlights can be listed as follows: (1) Since fractional derivatives characterize the memory fetures more accurately, we have developed the integer-order neural network model with mixed delays into the fractional-order case to match for the practical biological neural networks more appropriately.
(2) Although there have been some results on the dynamics of stability and bifurcation for delayed fractional-order neural networks, only discrete delays are considered. If distributed delays are added, the imbalance of delays should be discussed in transmitting information. In this paper, both discrete delays and distributed delays are taken into account at the same time in fractional-order neural networks.
(3) By the coordinate transformation, we can convert the original neural network with mixed delays into an equivalent system involving only discrete delays, which eliminates the distributed delay terms. However, the two-dimensional system with order α becomes a four-dimensional system with two different orders, which makes the derivation more interesting.
(4) The influences of mixed delays and order on dynamical behaviors of fractional-order neural networks have been investigated. It is found that the critical value can be effectively manipulated by adopting appropriate system parameters and order.
The paper is arranged as follows: In Section II, several preliminaries are put forward. In Section III, the fractional-order double-neuron model is presented. In Section IV, we arrive at the sufficient conditions of Hopf bifurcation. The validity of theoretical results are verified by simulation in Section V. Finally, Section VI summarizes the paper and indicates the future research.

II. PRELIMINARIES
In this section, the definition of the Caputo derivative and the stability of n-dimensional linear fractional differential system are introduced.
There are various definitions of fractional derivatives, of which the commonly adopted are the Riemann-Liouville definition, the Grünwald-Letnikov definition and the Caputo definition. The definition of Caputo derivatives has the superiority in not limiting the initial conditions and making fractional-order systems simpler after the Laplace transform.
The n-dimensional linear fractional differential system is where the order φ i are rational numbers and φ i ∈ (0, 1], for i = 1, 2, . . . , n. The characteristic equation of model (4) is as follows Let Q be the lowest common multiple of Only when all the roots s of the equation satisfy |arg( )| > π 2Q , the zero solution of system (4) is locally asymptotically stable.
Remark 1: The stability mentioned in this paper is the asymptotic stability.

III. THE MATHEMATICAL MODE
In order to better describe the dynamic system, we make appropriate improvements based on previous research. More specifically speaking, a double-neuron network with mixed delays is put forward as follows: where p i (i = 1, 2) represents the state of the ith neuron at time t and v ij (i = 1, 2 and j = 1, 2) denotes real constants.
The order α is a rational number. Moreover, the delay kernel function F(·) is said to be non-negative when ∞ 0 F(s)ds = 1. F(·) represents the impact of past memory on current dynamics and its form is as follows: where κ is the decline rate of the effects of past memories and κ > 0. The weak delay kernel with n = 0 is considered and presented as Remark 2: Evidently, system (7) can be converted to system (2) when α = 1 and τ 1 = τ 2 . Therefore, system (2) discussed in [14] is a special situation of system (7) proposed in this paper.
In fact, f (·) is an activation function and we make the following assumption: ( where C 3 is the set of third-order differentiable functions. For simplicity, we introduce two virtual neurons, as follows Then system (7) turns into Remark 3: By introducing two virtual neurons p 3 and p 4 , the new four-neuron system (8) is formed only involving the discrete delays. Fig. 1 reveals the architecture of (8). VOLUME 8, 2020 Remark 4: System (8) is an incommensurate-order neural model, which shares the property of containing two integer-order differential equations and two fractional differential equations. Compared with the integral-order neural models in [14], [15] and the commensurate-order neural networks in [36], [38], the presence of incommensurate order usually makes the analytical work more challenging.
From (H1), we obtain that the origin O(0, 0) is an equilibrium point of system (7). Thus, O(0, 0, 0, 0) is the equilibrium point of system (8). We assume that the origin is an isolated equilibrium point for simplicity. This paper only takes into account the local dynamics near the origin.

IV. LOCAL STABILITY AND HOPF BIFURCATION
Assume that the rational number α = m n , where m, n ∈ Z + , (m, n) = 1, and Z + represents the set of positive integers.
Let λ = s 1 n . According to Lemma 1, the following theorem can be summarized directly.
Proof: It is easy to see that the highest order term of G(ω) is ω 4α+8 , and  (14) has at least one positive root. Namely, (11) has at least a pair of purely imaginary roots.

V. NUMERICAL SIMULATIONS
In this section, the theoretical results acquired in Sec. IV will be supported by several numerical simulations to certify the accurateness and feasibility.
In what follows, we discuss the relationships between the parameters α, κ and τ 0 for system (7).
Case 1: Fix κ = 1, then plumb the effect of the order α on the bifurcation point τ 0 for system (7). The order α exhibits a linear relationship with the bifurcation point τ 0 , as shown in Table 1 and Fig. 8. The value of τ 0 rises with the decrease of the order α.
Case 2: Set α = 0.5, 0.75 and 1 respectively, then observe the influence of κ on the bifurcation point τ 0 . From Table 2 and Fig. 9, we can get the following information. Firstly, for a fixed value of α, the parameter κ is inversely proportional to bifurcation point τ 0 . Even more intuitively, the value of τ 0 decreases as the parameter κ increases. Secondly, for a fixed value of κ, the smaller the order α is, the greater the critical value of τ 0 is. Remark 7: Fig. 8 clearly shows that the smaller the order is, the larger bifurcation point gets. It is indicated that by contrast with the integer-order model in [14], the presence of fractional order effectively delays the occurrence of Hopf bifurcation for system (7), so the stability region is broadened.
Remark 8: From Fig. 8, it can be also find that the smaller the order is, the greater the bifurcation point is  The influence of κ on the value of τ 0 for system (7) with α = 0.5, 0.75 and 1. and the larger the stable domain is. This fact is also true not only for fractional-order neural networks [48], [49], but also for the fractional-order genetic regulatory network [29], the fractional-order predator-prey model [31] and the fractional-order SIS epidemic model [45].
Remark 9: It can be seen that the decline rate κ is a parameter in the delay kernel function F(s), which may represent the distributed delay to some extent. Hence, Fig. 9 presents   FIGURE 9. The influence of κ on the value of τ 0 for system (7) with α = 0.5, 0.75 and 1.
the influence of distributed delays on the dynamical bifurcation in the fractional-order neural network (7). It is worth noting that the effect of distributed delays on bifurcation of fractional-order systems has not been reported.

Remark 10:
The activation function f (·) = tanh(·) used in Example 1 is not periodic, while the function f (·) = sin(·) chosen in Example 2 has the periodicity. Therefore, the selection of activation function is enormously influential for the occurrence of chaos in system (8).

VI. CONCLUSION
This paper studies the dynamic behaviors of a fractional-order neural network with discrete and distributed delays. Firstly, by introducing two virtual neurons, we construct an equivalent four-neuron system that contains two fractional differential equations and two integer differential equations. Next, by adopting the sum of discrete delay as the bifurcation parameter, the sufficient conditions for the stability of the original system have been obtained. Furthermore, the Hopf bifurcation has been ascertained successfully by exploring the derived characteristic equation. Finally, through the numerical simulations, the correctness of the results has been verified and the influence of the order and parameter on the onset of bifurcation has been given. The simulation results show that the neuron system turns out a Hopf bifurcation when the sum of discrete delays reaches a certain critical value. In addition, the order exhibits a linear relationship with the bifurcation point, and diminishing order can postpone the onset of bifurcation. It indicates that the fractional-order system has a larger stability region than the integer-order one.
Chaos plays an important part in the study of system dynamics. In the future, we will devote to probing into chaos in fractional-order neural networks.