SPICE Study of STDP Characteristics in a Drift and Diffusive Memristor-Based Synapse for Neuromorphic Computing

Neuromorphic hardware is a system with massive potential to enable efficient computing by mimicking the human brain. The novel system processes information using neuron spikes (Action Potentials) and the synaptic connections between neurons are trained using biologically plausible methods like spike-timing-dependent plasticity (STDP). Memristor is one of the promising candidates to implement such neuromorphic hardware. Two types of memristors, diffusive and drift, have been proposed to form a synapse showing faithful emulation of STDP, where the diffusion effect is used to trace the spike timing history crucial for STDP and the drift memristor keeps the weight information in a longer time scale. The purpose of this paper is to systematically investigate STDP characteristics in such a synapse with serially connected two memristors using SPICE models. The results show that STDP properties are strongly dependent on device parameters and even the shape of STDP curves is modified. Different shapes of the STDP curve were identified. The results and analysis could support the design of emerging device-based synapses, which can faithfully mimic biological STDP characteristics for future neuromorphic systems.


I. INTRODUCTION
Neuromorphic engineering is a bottom-up approach to develop a computing system mimicking the human brain using basic artificial neural components, such as synapses and neurons [1], [2]. The new computing paradigm receives growing attention due to its potential for building a low-power learning system processing sparse action potentials (APs), and voltage spikes fired from neurons [3], [4]. A synapse biologically refers to a physical gap between two neurons, and functionally, it transfers received information to the next neurons in proportion to synaptic weights, i.e. connection The associate editor coordinating the review of this manuscript and approving it for publication was Artur Antonyan . strength [5]. Importantly, during training, the weight values can be tuned to form desired connections expressing features and in biology, spike-timing-dependent plasticity (STDP) is widely known as one of the fundamental rules of synaptic plasticity [5]- [7]. The weight adjustment process depends on local information such as timing difference between a preand post-neuron spike and it is deeply related to the learning process in the human brain. In detail, the firing of a pre-spike before a post-spike causes long-term potentiation (LTP) and the weight modulation becomes stronger at a shorter time difference of two pulses. The opposite firing order, post-spike before pre-spike, leads to long-term depression (LTD).
To implement it in hardware, several technologies including conventional complementary metal-oxide-semiconductor (CMOS) and emerging devices have been intensively studied [8]- [11]. Among the alternative emerging device technologies, memristors, also called resistive switching devices, modulate their electrical resistance on-demand via the combination of ionic REDOX and migration processes [12]- [14]. With intriguing abilities mimicking biological behaviors, [15]- [17]. With intriguing abilities mimicking biological behaviors, the device, memristor, is considered one of the most promising candidates as an artificial synapse [15]- [17]. In STDP, synaptic plasticity is a strong function of the timing difference between action potentials, and in biology, Ca 2+ having short-term dynamics is thought to serve a role as a timer to measure elapsed time between spikes [18]- [20]. In most of the previous reports that mimic STDP curves using memristor, they have relied on pulse engineering to reproduce the time-dependent characteristics [7], [15], [21]- [25]. A spike of pulse with precisely controlled shape has been fed into synapses as APs to encode the elapsed time. Despite the successful demonstration of STDP curves, the pulse engineering approaches, unlike biology, combine the timer role of Ca 2+ with an action potential by modifying the pulse shape. Recently, several works have tried to faithfully emulate STDP by using device short-term dynamics instead of pulse engineering [26]- [28]. One of the works was using volatile dynamics of a diffusive memristor as a time tracer [29], in which two different memristors, diffusive and drift one, were serially connected to form a single synapse. It combined a volatile property in a diffusive memristor, representing spontaneous decay of Ca 2+ , and a non-volatile memory effect in a drift memristor as shown in Fig. 1(a). In practice, the synapse can be fabricated by sharing one of the electrodes of two memristors.
While the experimental data was successfully emulated biological synaptic plasticity and prove the concept, a detailed analysis of the device operation is still missing. Moreover, in such a synapse with multiple types of memristor, more factors like voltage division and threshold of each type of device can make a complicated effect and hence the operation should be carefully studied to produce desired STDP curves. In this paper, we systematically studied STDP behavior of synapses formed by diffusive and drift memristor using SPICE simulation. Synaptic plasticity, LTP and LTD, was strongly affected depending on the device parameters, e.g. time constant and resistance levels of the diffusive and drift memristor. As a result, four different shapes of STDP curves were found and therefore the parameters should be chosen carefully to produce desired STDP characteristics from the diffusive and drift memristor-based synapse. The results can help the design of emerging devices for faithful emulation of biological STDP.

A. DEVICE MODEL FOR SPICE SIMULATION
Memristor is a two-terminal device that can modulate the resistance by applying external voltage stimuli. In the well-known HP model, the behavior can be described by using a state variable w as shown in (1), which represents normalized conducting channel (filament) length inside the switching layer [30], [31]. Thus, the larger w is, the higher conductance the device has. Since in our synapse, two kinds of memristors, diffusive and drift device, are serially connected to form a single synapse between an axon (presynaptic neuron; pre-neuron) and dendrite (postsynaptic neuron; postneuron) as shown in Fig. 1(a) [29], the total resistance is the sum of R Diff . and R Drift , which deciding current flow through the synapse (2). Next, to describe the dynamics of the state variable w, we used the modified version of the HP model by Biolek et. al. [31]. Specifically, when a voltage pulse is applied, it leads to ion migration with a speed proportional to the electric field as well as ion mobility. Hence, w change is a function of mobility (µ v ) and switching layer thickness (D) as described in (3) and (4) [27], [31]. Additionally, window function f (w) was placed to simulate a non-linear conductance change in LTP and LTD as measured in actual devices and the w update only occurs at the voltage over the threshold [15], [16], [27], [30]- [33]. Actually, we have tested several other memristor models, where we confirmed the basic STDP characteristics were kept similarly regardless of the model [31], [34]- [36]. On the other hand, for a diffusive memristor model, the spontaneous decaying term of w was added using a time constant τ to incorporate the volatile property as shown in (6) and (7) [27]. It is notable that the absolute value of current was used in (6) to express that diffusive memristor can be programed regardless of the voltage polarity. Device parameters used in our simulation were carefully chosen based on reported experimental and simulation data as shown in Fig. 1(b) [27], [31], [37]- [46] and in our simulation, a wide range of R on and R off were systematically tested in the following sections.

B. STDP CURVE
Based on the above model, we performed a SPICE simulation to examine STDP characteristics of diffusive and drift memristor-based synapses. Applied pulses shown in Fig. 1(a) are composed of two parts: V Drift (blue), a relatively short duration pulse (50µs) but high amplitude (4V or 3V ), and the following V Diff . (yellow), a long duration pulse (0.5ms) with low bias (2V ) [29]. If one increases the amplitude or extends the duration of the applied pulses, the STDP curve simply changes to have a larger conductance update in a wide time range ( t). Thus, we chose the above values from simulations to enable an appropriate amount of plasticity. When the pulse is applied to a synapse in Fig. 1(a), since most of the bias drops in the diffusive memristor having higher off resistance, a diffusive device switches from the off-state to the on-state by V Diff . , whereas the state variable of a drift memristor has a negligible change. It is notable that the condition, R off _Diff . > R off _Drift , is therefore essential for the selective turn-on of the diffusive device before the drift one. Next, we assume that the second spike arrives at the synapse at a time t away from the first spike. Given the spontaneous decaying property of the diffusive memristor, if t is short enough for the on-state diffusive element to maintain its low resistance, our focus is then voltage dividing between R on_Diff . and R off _Drift since the diffusive memristor is now on-state. When the drift memristor is in the off-state, w = 0, the condition, R on_Diff . < R off _Drift , is required to make sufficient voltage dividing on the drift device for conductance changes. However, as the drift memristor state approaching the on-state, w = 1, the condition changes to R on_Diff . < R on_Drift . Here, since R on_Drift is always smaller than R off _Drift , R on_Diff . < R on_Drift , covers both conditions regardless of the state of the drift memristor. Therefore, we finally have two conditions, R off _Diff . > R off _Drift and R on_Diff . < R on_Drift , where the diffusive device should have a larger resistance dynamic range than that of the drift device for the proper plastic operation. In other words, the range of R Drift should be within the range of R Diff . . Fig. 1(c) shows STDP curves obtained from the SPICE simulation using parameters, R on_Drift = 5K , R off _Drift = 50K , R on_Diff . = 1K , R off _Diff . = 500K . When the timing of pre-and post-spike gets closer, i.e. small t, larger conductance change ( w) appears in both LTP and LTD, showing proper STDP operation with the above device models and parameters. In the following section, we go through what happens in STDP curves when varying device parameters.

III. DEPENDENCY OF STDP ON DEVICE PARAMETERS A. TIME CONSTANT (τ )
It is expected that a larger τ of a diffusive memristor would extend the time window of STDP, due to the longer decaying time (=duration of on-state) in a diffusive memristor [27]. Indeed, in Fig. 1(c), the maximum plasticity time max_ t, x-intercept of the plot, increases with growing τ , while the maximum conductance change max_ w at the y-intercept (precisely at t = 10µs) posed negligible changes. Thus, change of τ can selectively modify the time range of STDP. Note that the intercept values, max_ t and max_ w, are well expressing the shape of STDP curves and this paper will use them as indicators when comparing the shape of STDP curves in the following sections.

B. RESISTANCE OF A DRIFT MEMRISTOR
In a synapse having two serially connected memristors, the amplitude of applied voltage spike as well as its voltage division are of significant importance to decide STDP characteristics. In that respect, we tested the STDP curves under various resistance conditions: first for a drift memristor (R Drift ) after the fixing resistance of a diffusive device (R Diff . ). As mentioned above, one constraint is that R Drift should exist between R off _Diff . and R on_Diff . , hence we set them as 500K and 1K of the R Drift range as shown in Fig. 2(a) and tested all the possible cases. From the simulation, we confirmed that there are two cases in STDP shapes depending on R on_Drift and R off _Drift as shown in Fig. 2(b). In specific, case1 indicates a typical STDP curve as shown in Fig. 2(c) (orange line), while case2 represents STDP curves with a hump appearing in the middle of the curve (purple line). The hump is clearly observed in case2 and it divides the curve into two regions 'A' and 'B'. Lastly, the gray area in Fig. 2(b) is that the device condition violates the fundamental constraint, R off _Drift > R on_Drift , and thus the condition is not feasible to implement a synaptic device. It is notable that case1 produces desired STDP shapes, but only a limited range of resistance satisfies the condition. To find out the reason for the hump observed in case2, the evolution of state variable and voltage at the drift memristor is checked in Fig. 3. Due to the shorter t in the region 'A' than the region 'B', it is expected that a diffusive component undergoes negligible decaying of w Diff . in the region 'A'. Hence, the device still maintains on-state enough to make sufficient voltage drop on a drift memristor over its threshold voltage as shown in Fig. 3(a). Therefore, w Drift increases from the beginning of the second pulse. In opposition, at a longer t (region 'B'), w Diff . decays more severely before arriving at the second pulse and the synapse is not able to enable plasticity at the initial of the second pulse as shown in Fig. 3(b). Because the voltage drop at a drift memristor becomes lower than its threshold value (V thres. ). Thus, the second pulse is initially used to turn strongly on w Diff . and at some point of the second spike, the voltage drop crosses V thres. and w Drift starts to increase. It means that part of the second pulse is discarded from the plasticity perspective in the region 'B'. Thus, the region 'A' and 'B' show different responses when changing t, where the gradual STDP curve in the region 'A' and the abrupt one in the region 'B', finally causing the hump. When using low R Drift , it has a small voltage drop and thus the hump will happen at a shorter t, since the voltage at the drift memristor can easily fall below the threshold. Thus, if we decrease R Drift more, the hump point moves shorter than 10µs (minimum t of our simulation) and eventually disappears because the entire STDP curve only consists of the sole programming mode of region 'B'. This accounts for why case1 in Fig. 2(b) has no hump and the resistance conditions are bounded to very low values.
Next, it is examined how R Drift changes max_ w and max_ t, the intercepts of a STDP curve. First, Fig. 4(a) shows the results on max_ w when varying R on_Drift and R off _Drift . There are two apparent trends: 1. the larger VOLUME 10, 2022 R off _Drift is, the smaller max_ w is (red arrow), and 2. the larger R on_Drift is, the larger max_ w is (yellow arrow). The equations describing state variable dynamics ((3) in table 1) can account for these results based on the value of k Drift and flowing total current (I ). For example, Fig. 4(b) is the result of increasing R off _Drift under the condition that R on_Drift is fixed to 10K . When increasing the R off _Drift , the total current is reduced by increased R off _Drift according to (1) and (2), while the value of k Drift does not change (inset of Fig. 4(b)) since k Drift only depends on R on_Drift ((4)). Thus, the product of I and k Drift decreases as R off _Drift grows and indeed, the multiplied value well describes the change of max_ w as shown in Fig. 4(b). For the case of changing R on_Drift when R off _Drift is 500K , both I and k Drift are affected, but the direction is opposite (inset of Fig. 4(c)). Due to a more dominant change in k Drift , the product value increases with higher R on_Drift and this well matches again with the change of max_ w as shown in Fig. 4(c).
Next, the change in the maximum plasticity time max_ t, x-intercept of an STDP curve, is plotted in Fig. 5(a). It should be noted that unlike max@backslash w above, max_ t points always belongs to the region 'B' in Fig. 2(c), where the voltage drop at a drift memristor is less than its threshold at the beginning of the second pulse and it thus requires time to strongly turn on w Diff . to enhance the voltage drop. If required voltage (threshold) is not achieved until the end of the second spike, there is no synaptic plasticity at all and max_ t represents a boundary deciding the possibility of plasticity. Therefore, it can be said that max_ t is determined by voltage division between a diffusive and drift memristor at the end of the second pulse, and from this regard, our focus is placed on dynamics of a diffusive memristor ((6) and (7)) and R Drift . As the R off _Drift grows in Fig. 5(b) with R on_Drift fixed at 10K , the total current decreases very abruptly at low R off _Drift (inset of Fig. 5(b)), leading to weak dynamics of a diffusive memristor and hence low w Diff . at the end of the second pulse. This rapid change causes an insufficient voltage drop at a drift memristor despite the larger R Drift and finally max_ t decreases. However, from around 100K of R off _Drift , the current change becomes negligible and max_ t starts to increase due to the continuous growing R Drift , which results in more voltage drop at drift memristor. This is why the max_ t curve draws a U-shape when varying R off _Drift . The effect of changing R on_Drift on max_ t as shown in Fig. 5(c) can be explained with the same manner. Here, unlike the R off _Drift case, the change in total current is very gradual (inset of Fig. 5(c)) and hence max_ t simply follows the trend of R Drift , showing longer max_ t as R on_Drift grows.

C. THRESHOLD OF A DRIFT MEMRISTOR
In the previous section, the threshold of a drift device significantly affected the characteristics of STDP including the undesired hump and max_ t. Here, the map in Fig. 2(b) is redrawn with a larger threshold value as shown in Fig. 6 to confirm the impact. When increasing the threshold, the border of the case1 expands toward the case2 area by removing the hump in several conditions. This is attributed to that the higher threshold makes the synapse's program mode from the region 'A' to region 'B' in a short t just the same as the low R Drift cases and hence it moves the hump to the shorter t. Eventually, the hump disappeared within the simulation range of t and it expands region of case1. Hence, higher threshold is preferable to produce a wider range of typical STDP curves. However, if it grows too high, w will decrease at the same applied voltage and from some point, it makes the synapse not work.

D. RESISTANCE OF A DIFFUSIVE MEMRISTOR
In this section, an impact of a diffusive memristor resistance, R Diff . , on STDP characteristics is studied. The range of R Diff . used covers a wide range of possible resistance values as shown in Fig. 7(a), where the maximum and the minimum of R Diff . are set to 10M and 1K respectively given the reported practical values[47]-[52]. In the same manner to Fig. 2(b), a map showing different STDP cases is drawn again for the resistance conditions, R Diff . in Fig. 7(b). Comparing the two maps in Fig. 2(b) and 7(b), it is notable that to achieve the desired typical STDP curve without hump (case1), the resistance of R Drift (both R off _Drift and R on_Drift. ) is preferable to be close to R on_Diff . The opposite direction, where R Drift is close to R off _Diff . , leads to the appearance of case2 STDP curves due to larger voltage division to a drift memristor as analyzed in the sub-section B. On the other hand, if R off _Drift. and R off _Diff . place too close, another shape of STDP, case3, shows up as shown in Fig. 7(b). The new case3 refers to a STDP curve having infinite max_ t without touching the x-axis even if t is much longer than the time constant as shown in Fig. 7(c) (pink line). This is attributed to the very low R off _Diff . close to R off _Drift . Large voltage drop at drift memristor is sufficient to cause programming of memristor over the threshold regardless of whether the state of a diffusive memristor is turned-on or turned-off. Therefore, it is expected that plasticity may occur even when feeding a single spike to a synapse without having a pair of pre-and postspike. We tested in Fig. 7(c) and indeed, feeding a single pulse makes w in the resistance condition of case3 (blue dashed line) and the plasticity amount equal to the paired spike case (pink line) at longer t. Finally, in Fig. 7(b), case4, was also found, in which no plastic behavior, w, is detected throughout the simulated t range, When applying external bias, ion migration modifies the filament length in proportional to the flowing current and the value k as shown in dynamics (in (3)), (c) STDP curve at the resistance condition of the red circle in Fig. 7(b). w could not reach to zero. The condition is R on_Diff . = 1K , where k is a function of the on-state resistance. Therefore, with higher R off _Diff . and lower R on_Diff . making low current and small k value, it is hard to be turned on for the diffusive memristor and the case4 appears.  Next, max_ w and max_ t are investigated when varying R Diff . . In Fig. 8(a), max_ w decreases with growing R off _Diff . (red arrow), whereas it increases with larger R on_Diff . (yellow arrow). This can be accounted for by the dynamics of drift memristor (in (3)) same as in the previous section C. But, here, k Drift is a constant when changing R Diff . and thus, our focus is on total current, I , to explain the change of max_ w. In Fig. 8(b), increasing R off _Diff . reduces the flowing current and consequently, it weakens device dynamics of drift memristor and leads to the smaller max_ w. Likewise, in Fig. 8(c), with various R on_Diff . , the change of max_ w well matches with the changing of total current again. Next, in Fig. 9(a), max_ t decreases with growing R off _Diff . (red arrow) and in opposite, it increases with larger R on_Diff . (yellow arrow). Unlike the previous sub-section B, when changing R Diff . , we do not need to consider the fixed value, R Drift . Instead, dynamics of a diffusive memristor itself is suitable to understand the behavior as in Fig. 9(b) and 9(c). When increasing R off _Diff . , the total current is sharply weakened, but k Diff . is a constant (according to the (7)). Thus, the product value of the total current and k Diff . , representing dynamics of a diffusive memristor, is rapidly decreased and this causes low voltage drop on the drift memristor. Therefore, max_ t is getting shorter with increasing R off _Diff . . In the similar method, larger R on_Diff . in Fig. 9(c) makes the total current decrease as shown Fig. 9(c). However, increase of k Diff . is more abrupt with increasing R on_Diff . and as a result, the diffusive device can be on-state more easily. Therefore, max_ t is getting longer with growing R on_Diff . .

E. THE RATIO AMONG RESISTANCES
From the results of this paper, it seems obvious that voltage division is the major factor to decide STDP characteristics, and therefore, the resistance ratios between two memristors can be good dimensionless indices. For example, R on_Drift R on_Diff . , R off _Drift R on_Diff . and R off _Drift R off _Diff . −1 , are determinative expressions of the data independent of the randomly chosen values. Each ratio can be viewed as follows: first, R off _Drift R off _Diff .

−1
affects the initial switching of the diffusive memristor, second, R off _Drift R on_Diff . and R on_Drift R on_Diff . determine the voltage drop on the drift memristor at the second spike when the w = 0 and w = 1, respectively. Using the three ratios, we again mapped different STDP cases in Fig. 10. Now, all the cases are represented using the meaningful determinative resistance ratios deciding device dynamics and sequentially plasticity.

F. APPLICATION
To practically use our synapse, the multilayer network can be configured by utilizing its synapses. Here, we have chosen different voltages for pre-and post-spike since, in many papers, the various amplitudes of bias are used for Set(=potentiation in STDP) and Reset(=depression) operation respectively. Of course, the set and reset voltage are tunable parameters by changing device structures, the thickness of films, and properties of materials and thus, it could be the same voltage for both pre-and post-spike. Nevertheless, if one needs to build a multilayer network with different pre-and post-spike, it is necessary to add a LIF neuron block to the system as shown in Fig. 11(a). The neuron block consists of three sub-blocks (a conventional integration part for leaky-integration of membrane potential and spike-generation sub-blocks for both directions, backward and forward) and two switching devices (e.g. transistors). Whenever the integrated potential exceeds its threshold, the spike sub-blocks generate action potentials for the forward and backward directions independently. For proper implementation of STDP-based SNN, the architecture in Fig. 11(a) should be able to separate different operations: 1. Integration of post-neuron vs. firing of postneuron, 2. training vs. inference (Fig. 11(b)). To this end, we put the two switches, M1 and M2, in the LIF neuron block. By selectively turning-on and -off the transistors whenever the post-neuron fires, it can handle different operations in the same structure. First of all, during the integration of the LIF neuron, M1 is on-state and M2 is off-state. And then, when the integrated membrane potential exceeds its threshold, the post-neuron fires. Since the M2 switch is designed to turn on only at the post-firing, the spike from the backward spike block propagates to the synapses. This enables weight updates according to the timing difference between the pre-and post-spike as the STDP rule. On the other hand, the post-spike simultaneously makes turning-off the M1 switch. This stops the additional increase of membrane potential by terminating integration parts not only from the input side but also from the backward spikes via the M1 transistor. As a result, the LIF neuron keeps the idle (resting) state during the firing, and thus integration and firing operation works separately in the hardware. Second of all, the architecture can perform an inference mode by simply keeping the M2 transistor off regardless of the post-neuron firing, preventing any transmission of backward spikes and consequential weight updates. Hence, simple control of the M2 switch enables the separation of training and inference stages.
In short, by regulation of the M1 switch via the post-spike, the proposed STDP-based SNN architecture can separate integration and firing mode, while control of the M2 switch enables separation of training and inference operation.

IV. CONCLUSION
In this paper, we studied the characteristics of STDP curves when using a synapse with two serially connected memristors, diffusive and drift one. In the SPICE simulation, impacts of device parameters such as time constant (τ ), threshold, and resistance of on-state and off-state were systematically examined. It revealed that the synapse responses, e.g., max_ w and max_ t in STDP curves, are very sensitive to the device parameters and moreover, four different shapes of STDP curves exist depending on the parameters. One of the four cases is having the typical STDP shape, but it appears in a quite narrow parameter range. The interesting behavior was analyzed based on the device dynamics and voltage division effects between a diffusive and a drift memristor. Proper parameters should be chosen to produce desired STDP curves mimicking biological behavior when using the serially connected synaptic device. The results can provide a guideline of using the synapse, which enables faithful mimic of biological plasticity with proper STDP curves and this is thought to be important for the implementation of neuromorphic hardware.
INHO KIM received the Ph.D. degree in materials