Multilayer Crossbar Array of Amorphous Metal-Oxide Semiconductor Thin Films for Neuromorphic Systems

A multilayer crossbar array has been developed using amorphous metal-oxide semiconductor (AOS) thin films and implemented into a neuromorphic system. The multilayer structure can be realized, because the AOS thin films can be deposited by a simple sputtering method without heat treatment, which does not damage the underlying structures. First, Au thin films are deposited by vapor evaporation as electrodes, an amorphous In-Ga-Zn-O ( $\alpha$ -IGZO) thin film is deposited by a sputtering method as a conductance change layer, these processes are repeated, and a multilayer crossbar array is completed, where each of the three conductance change layers is sandwiched between the electrodes. Next, the multilayer crossbar array is implemented into a neuromorphic system with modified Hebbian learning, which enables autonomous learning without control circuitry, and an associative memory function is confirmed, which guarantees the possibility of further advanced functions. These results lead to astronomical large-scale integration (LSI) of synaptic elements in neuromorphic systems in the future.


I. INTRODUCTION
Artificial intelligences are becoming indispensable technologies as fundamental infrastructures in various societies, such as character and image recognition, information search and supply, language translation and captioning, expert system, automatic driving, autonomous brains, etc [1]- [3]. Neural networks serve as representative manners of artificial intelligences, which mimic operation principle of biological brains [4]- [8]. However, the traditional ones are redundant and intricate software to conveniently run high-spec Neumann-architecture computer hardware, which is not customized for neural networks, and the machine size is incredibly bulky and power dissipation is also unbelievably huge. Neuromorphic systems provide bioinspired systems from the device level and practical solutions that compose neural networks solely of customized hardware, whose advantages are self-organization, self-learning, parallel distributed computing, and fault tolerance, and the machine size can be compact and power consumption can be low [9]- [12]. However, the conventional ones are based on silicon device and circuit technologies, whose disadvantages are that they are digital circuits and have two-dimensional structure, which are completely dissimilar from biological brains. The digital circuit require more circuits than analog devices, and the two-dimensional structure of course has a limitation to high integration than the three-dimensional structure. Crossbar arrays of memristors are promising suggestion beyond the silicon technologies [13], [14]. On the other hand, amorphous metal-oxide semiconductor (AOS) thin films are being investigated for diverse applications [15]- [27] and proposed also for neuromorphic systems [28]- [38], whose advantages are that they have analog characteristic [39] and can have three-dimensional structure [40].
In this study, a multilayer crossbar array has been developed using AOS thin films and implemented into a neuromorphic system. The multilayer structure can be realized as three-dimensional structure, because the AOS thin films can be deposited by a simple sputtering method without pre-, in-situ-, and post-heat treatment, which does not damage the underlying structures. In this paper, first, the device structure and fabrication processes of the multilayer crossbar array will be explained. Next, the multilayer crossbar array will be implemented into a neuromorphic system with modified Hebbian learning [41], and a neuromorphic function will be confirmed. These results will lead to the feasibility of astronomical large-scale integration (LSI) of synaptic elements in neuromorphic systems in the future.

II. MULTILAYER CROSSBAR ARRAY OF AMORPHOUS METAL-OXIDE SEMICONDUCTOR THIN FILMS
The multilayer crossbar array of AOS thin films is shown in Fig. 1. The cross-sectional illustration is shown in Fig. 1(a), and the overview photograph is shown in Fig. 1(b). The device structure is extremely simple, where each of the three conductance change layers is sandwiched between the electrodes. The fabrication processes are as follows. First, a quartz glass substrate is prepared, whose thickness is 0.7 mm and size is 3 × 3 cm. Next, a Au thin film is deposited by vapor evaporation, whose thickness is 80 nm, and patterned through a metal mask, whose line and space widths are 1.2 and 1.2 mm and number of lines is 10, as the first electrodes. Sequentially, an amorphous In-Ga-Zn-O (α-IGZO) thin film is deposited by radiofrequency (RF) magnetron sputtering method with a ceramic target of In:Ga:Zn=1:1:1, sputtering gas of Ar:O 2 =5:15, deposition pressure of 2 Pa, and plasma power of 60 W, at room temperature, whose thickness is 90 nm, as the lower conductance change layer. Then, a Au thin film is again deposited through the metal mask, which is placed orthogonal to the first electrodes, as the second electrodes. Repeatedly, the α-IGZO and Au thin films are deposited two  more times, as the middle conductance change layer, third electrodes, upper conductance change layer, and fourth electrodes. Finally, a multilayer crossbar array of AOS thin films is completed, where each of the three conductance change layers is sandwiched between the electrodes, the first and third electrodes constitute the vertical electrodes of 20 lines, and the second and fourth electrodes constitute the horizontal electrodes of 20 lines, and it can be said 10 × 10 × 3 = 300 crosspoint-type devices using AOS conductance change layers are integrated in three-dimensional structure. The cross-section-polisher scanning-electron-microscope (CP-SEM) photograph is shown in Fig. 1(c). It should be noted that the first and third electrodes are staggeredly located, and the second and fourth electrodes are also the same, as shown in Fig. 1(b). In addition, they are patterned through a metal mask, and the electrode edges are gently sloping, as shown in Fig. 1(c). As consequence, there is no coverage problem. If they should be patterned by photolithography, some revision would be needed.
The resistance change over time by voltage application is shown in Fig. 2. A voltage of 3 V is applied, currents are measured, the resistance is obtained, and its ratios to the initial values are plotted. It is found that the resistance increases with time. The mechanism of the resistance increases is speculated as follows [29]. The first candidate is the decrease in oxygen vacancies due to the uptake of oxygen atoms.
The oxygen atoms in the IGZO lattice or from outside the IGZO thin film move to the oxygen vacancies and annihilate them, the oxygen vacancies that act as donors decrease, and free electrons decrease. The second candidate is the increase in trap states due to impact of the free carriers in the AOS conductance change layer. The free electrons are accelerated and collide to the IGZO lattice, trap states are generated and capture free electrons, and free electrons decrease and are simultaneously scattered by them. In any case, the resistance changes can be regarded as an analog memristive characteristic and utilized for the modified Hebbian learning [39], [41].

III. IMPLEMENTATION INTO A NEUROMORPHIC SYSTEM
The implementation into a neuromorphic system is shown in Fig. 3. The overview photograph is shown in Fig. 3(a). Neuron elements are formed externally in an FPGA board and connected to the multilayer crossbar array of AOS thin films as synapse elements, which are controlled by a personal computer.
The training phase is shown in Fig. 3(b). Here, the blue arrows indicate the directions of signals, blue bright and dark squares indicate on and off input signals, and blue bright and dark rhombuses indicate positive and negative input voltages. During the training phase, either high training voltages of ±3 V or no voltage is applied between the horizontal and vertical electrodes, so that the conductance change is induced in the conductance change layers, which is the modified Hebbian learning. Namely, the blue bright and dark rhombuses indicate +1.5 V and −1.5 V, respectively. If the voltage of either ±3 V is applied to the crosspoint-type device, the conductance change is induced, and otherwise, it is not induced. It should be noted that the training voltages themselves and their inverted voltages are applied to the neighboring electrode pairs, by which bias in the sign of the voltage can be avoided.
The inference phase is shown in Fig. 3(c). Here, the green arrows indicate the directions of signals, green bright and dark squares indicate on and off input signals, green bright and dark rhombuses indicate positive and negative input voltages, red bright and dark rhombuses indicate positive and negative output voltages, and red bright and dark squares indicate on and off output signals. During the inference phase, low input voltages of ±0.1 V are applied to only the horizontal electrodes, so that the resistance change is not induced. Namely, the green bright and dark rhombuses indicate +0.1 V and −0.1 V, respectively. After currents flow through the circuit built by the crosspoint-type devices, the voltages are determined at all nodes. Some output voltages are measured from the vertical electrodes, and they are distinguished as either on or off signals depending on whether they are positive or negative in the corresponding neighboring electrodes pairs. Namely, the red bright and dark rhombuses indicate some positive and negative voltages, respectively. If one output voltage is positive and the other neighboring voltage is negative, they are distinguished as on signals, and vice-versa, they are distinguished as off signals. Because the input voltage is low, the output voltage is also low, but it is easily possible to check only the sign. It should be noted that the output voltages are feedbacked to the input voltages until the steady state after dynamic behavior of the neuromorphic system. This is because the initial output voltage may be altered after they are feedbacked, and the final output voltage is supposed to fall into the minimum energy state of this system, that is, the trained voltage pattern during the training phase.

IV. ASSOCIATIVE MEMORY FUNCTION
The associative memory function is shown in Fig. 4. The algorithm is shown in Fig. 4(a). During the training phase, alphabet characters of "T" and "L" are learned. First, a two-dimensional pixel pattern of 3×3 pixels of "T" is transformed to a one-dimensional signal pattern of 9 components, and the signal pattern is inputted to the neuromorphic system, namely, the corresponding voltages and inverted voltages of 18 pieces are applied to the horizontal and vertical electrodes for 1 second. Next, a signal pattern of "L" is similarly inputted in sequence. During the inference phase, alphabet characters of "T" and "L" are reproduced. First, a slightly distorted pixel pattern, namely, a one-pixel flipped pattern of 3×3 pixels of "T", is transformed to a signal pattern of 9 components, the signal pattern is inputted to the neuromorphic system, namely, the corresponding voltages and inverted voltages of 18 pieces are applied to the horizontal electrodes for 1 second, some signal pattern is outputted, namely, the corresponding voltages and inverted voltages of 18 pieces are measured from the vertical electrodes, the one-dimensional signal pattern of 9 components is transformed to the two-dimensional pixel pattern of 3×3 pixels, and the outputted pixel pattern is compared with the alphabet characters of "T". Subsequently, these procedures are repeated also for different distorted pixel patterns. Next, slightly distorted signal patterns of "L" are inputted, and the outputted pixel pattern is compared with the alphabet characters of "L". Then, these procedures are reiterated many times.
The experimental results are shown in Fig. 4(b). It is confirmed that the alphabet characters of "T" and "L" are successfully learned, namely, the alphabet characters of "T" and "L" are successfully reproduced, except for one failure example, which may be due to the unwanted deviation of the analog memristive characteristic of the multilayer crossbar array of AOS thin films. In any case, it can be said that an associative memory function is confirmed. It should be noted that the associative memory function is confirmed in practical time, although the resistance change is slow, as shown in Fig. 2. This is because output voltages are determined by majority vote of the crosspoint-type device, where even the small differences in the resistance values are meaningful. Moreover, it is expedient, because multiple overrides of various trainings become possible.
The comparison of resistance change between theory and experiment is shown in Fig. 5. The voltage application combination is shown in Fig. 5(a). During the training phase, voltage is applied or not for each crosspoint-type device for each "T" and "L", and therefore there are three voltage application combinations: Combination 1, where no voltage is applied and the conductance change is not induced; Combination 2, where voltage is applied in only one case; and Combination 3, where voltage is applied in both cases. Therefore, the resistance difference between Combination 1 and Combinations 2 and 3 corresponds to the conductance change during the training phase.
The experimental results are shown in Fig. 5(b). The resistance is measured for all the crosspoint-type devices using AOS conductance change layers after the training and inference are successfully done, and its values are plotted for each voltage application combination. The resistance values for the crosspoint-type devices where the alphabet characters are successfully reproduced are plotted with the standard deviations in the figure above. It is found that the resistance value increases as the number of voltage application case increases, which is as expected. It should be noted that the alphabet characters are successfully reproduced, although the resistance change is small and the standard deviations a quite large, which is covered because the inference is done by a majority vote of many crosspoint-type devices. Moreover, the resistance values for the one failure example shown in Fig. 4(b) is also plotted in the figure below. The relationship of the resistance values between the voltage application combination is not as expected, namely, the resistance value for Combination 2 must be less than but is actually more than that for Combination 3, which clarifies that the failure example is due to the unwanted deviation of the analog memristive characteristic. Therefore, this problem will be solved by improving the characteristic uniformity of the crosspoint-type devices.
The power consumption can be considered as follows. Because the resistance value is several k as shown in Fig. 5(b), the power consumption per crosspoint-type device is several mW when a voltage of ±3 V is applied during the training phase and several µW when a voltage of ±0.1 V is applied during the inference phase, respectively. Because the size is 1.2 × 1.2 mm, the power consumption per area in the crosspoint-type device is several mW/mm 2 during the training phase and several µW/mm 2 during the inference phase, respectively. Although the size is currently large, if the crosspoint-type device is miniaturized down to, for example, 100 × 100 nm, the power consumption per crosspoint-type device is several tens pW during the training phase and several tens fW during the inference phase, respectively. Even if 100 trillion synapses, which is the same number as in a human brain, are integrated, the power consumption will be several W at most, which is less value than in a human brain.

V. CONCLUSION
A multilayer crossbar array has been developed using AOS thin films and implemented into a neuromorphic system. The multilayer structure can be realized as three-dimensional structure, because the AOS thin films can be deposited by a simple sputtering method without heat treatment, which does not damage the underlying already deposited structures. First, Au thin films were deposited by vapor evaporation as electrodes, an α-IGZO thin film was deposited by a RF magnetron sputtering method as a conductance change layer, and these processes were repeated. A multilayer crossbar array was completed, where each of the three conductance change layers was sandwiched between the electrodes, and a lot of crosspoint-type devices using AOS conductance change layers were integrated in three-dimensional structure. Next, the multilayer crossbar array was implemented into a neuromorphic system with modified Hebbian learning, which enables autonomous learning without control circuitry. Alphabet characters were learned during the training phase, and they were reproduced during the inference phase. It can be said that an associative memory function was confirmed, and the comparison between theory and experiment was also verified, which guarantees the possibility of further advanced functions. These results lead to astronomical LSI of synaptic elements in neuromorphic systems in the future.