Brain Inspired Color Feature Selection Chip

Inspired by the mechanism of visual attentional selection, a color feature selection unit consisting of photoreceptors and an attentional selection circuit (ASC) is presented. The detection and wavelength recognition of trichromatic light are accomplished by the photoreceptors with RGB filters. The ASC, fabricated in a standard CMOS technology, activates the competition among photoreceptors to select the principal color-component feature defined as the attentional focus. The number of focuses and resolution can be changed with a top-down stimulus. Electrical test results show that the ASC can select and interpret the most important color-component feature. In optical tests, the ASC chip-based unit can extract the principal color-component of single-color and dual-color light in real-time to complete color recognition. The percentage ratio between the intensity of stray light relative to the light with maximum intensity is up to 93.58%. And the maximum achievable recognition frame rate is 1 kHz, which is more than one order of magnitude higher than the frequency of the biological vision systems. Besides, the unit's outputs can be transformed into binary codes as labels for color recognition, and the color recognition process does not require the participation of ADC, CPU, and memory, consequently avoiding the von Neumann bottleneck.

of cones in the retina is significant for human beings to sense changes in their surroundings [3]. Mimicking the human eye's spectral response, the artificial image sensor system decodes visible light into three current values through photodetectors covered with wavelength selective filters. Data acquisition is completed with analog-to-digital conversion circuits, and the collected data is amplified, compared, and integrated to realize color recognition, which is completely dependent on CPU and memory [4]. These systems are based on the traditional von Neumann architecture, which consumes a lot of energy in data transmission and processing and may not be able to meet real-time recognition requirements [5]. In particular, these color recognition systems have to deal with the overloaded information at the cost of system complexity or power consumption.
In the human visual system, a tremendous amount of visual information falls on the retina at any given moment, far more than the brain's capacity to process. Surprisingly, the brain can deal successfully and meaningfully with the information in a low-power, efficient and rapid way [6]. This process owes much to the mechanism of attentional selection [7]. Visual attentional selection is the most powerful strategy used by the human visual system, which can select, modulate, and sustain focus on the most behaviorally-relevant information while discarding insignificant ones [8], [9]. Most recently, the mechanism of visual attentional selection has been introduced into visual tasks such as machine translation [10], image recognition [11], and semantic scene segmentation [12]. However, most of these applications are based on very large scale integration (VLSI) circuits, which exist bottlenecks in terms of speed and power consumption [13]. With the boom of deep learning, the artificial neural network is being used in the visual system to simulate attentional selection [14], but in practice it may lead to an inaccurate model due to insufficient data and training time [15]. Therefore, it is of great significance for promoting the development of artificial vision to design a hardware implementation of the attentional selection with simple structure, real-time recognition, low power consumption, and miniaturization.
The color recognition process of the human brain is shown in Fig. 1. The photoreceptor cells located in the retina transform visible light into nerve impulses. The neural networks improve the signal selectivity and contrast by weakening the response of nerve impulses near significant targets to select the significantly associated features, completing the color recognition [16]. Attentional selection is attributed to the regulation of neural activity in the primary visual cortex, including the first visual region (V1) and the frontal lobe eye area [17]. Inspired by the visual attentional selection, a color feature selection scheme was proposed, which is also shown in selection unit can select the most significant information from visible light, and all decisions and information processing are carried out at a low power level. Thus, the unit overcomes the limitations of the Von Neumann architecture [18]. The unit can be devided into two main functional blocks. The first block is photoreceptors, which contain photosensing devices and transimpedance amplifiers (TIAs), generating analog signals whose amplitude is proportional to the light intensity. The second block is the signal processing module, where the attentional selection circuit (ASC) emulates the visual attentional selection and shifts its attention to the most behaviorally-relevant information in real-time. The feature selection unit has the initiative and selectivity [19] like humans by adaptive adjusting the focus of attention to select the principal color-component feature, and it is a miniaturized hardware implementation of the visual attentional selection, which increases performance over existing computer vision feature extraction approaches.

II. ATTENTIONAL SELECTION AND CIRCUIT IMPLEMENTATION
In all primates, attention is a critical factor in the regulation of behavior and neurocognitive functioning [20]. The attentional selection works throughout the brain and participates in almost every stage from sensory processing to decision-making and cognition. The research showed that there are two types of attentional selection [21]. The first one driven by environmental stimuli is usually named as "bottom-up" attention. The other is the voluntary attention to external and internal stimuli controlled by the brain, called "top-down" attention. Attentional selection promotes the expression of the most important information in the human brain by improving recognition speed and contrast, enabling organisms to capture prominent targets [22]. The ASC, the hardware implementation of the attention selection mechanism, selects the most significant signal by activating the competition among inputs and actively shifts attention to the position with the salient feature.
The bottom-up attentional shift: Since the silicon-based MOS transistors operating in the subthreshold state have the work function to model biological processes [23], they can achieve a physical process similar to cell membrane current regulation of biological neurons with low power consumption [24]. Based on these fundamental properties, a three-branch attentional selection circuit for color recognition was proposed, which is shown in Fig. 2(a). The proposed circuit responds to the branch with the highest input voltage while suppressing the responses of the other branches, concentrating processing power on the most important information. As a result, its electrical behavior resembles the neurological behavior of specific neural networks in the visual cortex. Fig. 2(b) depicts the I D -V GS relationship of input transistors M B , M G and M R as excitatory neurons. When the transistors operate in the subthreshold region, the I D increase exponentially with increasing V GS (gate-source voltage), thus making the circuit extremely sensitive to the variation of input voltage. The common node voltage V C changes with the input voltage as shown in Fig. 2(c). Similar to the mechanism of inhibitory interneurons in the human brain, the transistor M inhibition , as an intermediate inhibitory neuron, mediates global inhibition through the common node. When the input voltage of a branch is stronger than that of the other two, this branch gains most of the current from M inhibition . The source voltage of the input transistor follows the change of the gate voltage, increasing the source voltage, which is V C . Since all input transistors share a common source, the responses of other branches are suppressed until they are turned off. Finally, the currents flow from the current mirror formed by M i1 and M i2 (i = 1, 2, 3) to the efferent transistors M B _OU T , M G_OU T , and M R_OU T as efferent neurons and then are converted into spike voltage signals. Based on the global inhibitory feedback mechanism [25], the proposed circuit quickly switch its attention to the branch having the strongest input and eventually tends to a stable state with only one attentional focus.
The top-down attentional distraction: To be more sensitive and accurate in perceiving changes in the external environment, the brain sometimes actively controls the visual system to distract attention toward the information outside the attentional focus. To achieve the attentional distraction, a top-down stimulus is applied to the efferent transistors.  Fig. 2(d). As V B decreases, the internal resistance, as a reward, gradually increases, and the attention of the V IN2 branch is therefore strengthened. When the reward is strong enough, the V IN2 branch transforms into an additional focus of attention. As a reward-driven attentional distraction, the reward can also adjust the degree of attentional distraction. When compared with the traditional winner-take-all circuit [27], except for the maximum signal selection, the ASC can change the number of attentional focuses and resolution by applying a top-down stimulus. When different stimuli are applied to the various branches, the circuit's attention only concentrates on the information outside the attentional focus to meet higher-level recognition requirements.
The DC simulation results of the ASC are shown in Fig. 3(a).
is far less than 2 V, the circuit only pays attention to the V IN2 branch and thus, Similarly, when V IN3 exceeds the other two values, the attention switches to its branch. As V IN3 approaches V IN2 , the ASC adjusts its attention in real-time according to the inputs. The number of attentional focuses and the resolution of the ASC versus V B are shown in Fig. 3(b). The resolution of ASC with double attentional focuses is defined as the maximum difference between the input voltages of the corresponding branches of focus. When the input difference is within the resolution, the circuit can distract its attention to the additional focus. The resolution of ASC with a single attentional focus is the minimum voltage difference that the circuit can recognize when the inhibition weight of M inhibition to each branch and the ASC's attention is constantly adjusted according to the input voltage variation. The circuit switches from the adjustment state to the steady state with only one focus once the voltage difference is outside the resolution. It can be seen that by changing V B , the voltage resolution and the number of attentional focuses can be adjusted. When V B is 0.65 V and 0.55 V, the circuit has the optimal voltage resolutions of 24 mV and 33 mV in the corresponding state, respectively.
As a whole, The ASC can simulate the dynamic process of attentional selection, and its simple structure paves the way for low power, efficient hardware designs.

A. Electrical System Measurements: the ASC Chip
The ASC was fabricated in the Taiwan Semiconductor Manufacturing Company (TSMC) 0.18 µm standard complementary metal-oxide-semiconductor (CMOS) technology, and the supply voltage is 3.3 V. To make the circuit more stable and easily accommodate to different conditions, V A can be adjusted externally. During the test process, the value of V A is about 0.5 V. The micrograph of the ASC is shown in Fig. 4(a). The ASC chip, including pads, occupies an area of 850 µm × 800 µm, whereas the core area of the ASC chip is only 0.0285 mm 2 (150 µm × 190 µm). All the pads of the chip are marked in the figure, and the ASC chip was mounted on a test printed circuit board (PCB) using wires bonding.
The test condition of the ASC is consistent with the simulation. The test results at V B of 0.65 V are shown in Fig. 3(c). The resolution with single attentional focus is about 80 mV. The test results show that the ASC can focus on the branch with the maximum input voltage in real-time. Note that there is a slight delay in the response of V OU T 3 compared to the simulation results when V IN3 slightly exceeds V IN2 , which may be caused by process variations and layout mismatches [28]. The results at V B = 0.55 V are shown in Fig. 3(d). The circuit can select double attentional focuses simultaneously when the inputs are close, and excellent agreement between experiments and simulations is achieved.

B. Optical System Measurements: The Color Feature Selection Unit
Based on the fabricated ASC chip, the proposed color feature selection unit was assembled on a specially designed PCB, as shown in Fig. 4(b). The photoreceptor is located in the blue frame, and the photo-sensing devices (photoreceptors) are in the white frame. It can be seen from in Fig. 1 that the central wavelengths of the three photoreceptors are 450 nm, 550 nm, and 650 nm, respectively. The TIA is located in the yellow frame, and its circuit diagram and frequency response is shown in Fig. 4(c). Operational amplifiers (OPA847, Texas Instruments) are used to  The upper bound on the number of attentional focuses is set by V A by controlling the current flowing through the ASC, while V B can determine whether the branch outside the attentional focus can become an additional focus by giving it enough "attention". In this paper, the ASC possesses multiple attentional focuses simultaneously. The recognition of single-color light and dual-color light can be achieved by changing the number of attentional focuses. In particular, the special attentional target can be set by adjusting V B . For example, if the red light is set as the attentional target, the unit's attention can be guided by the target and shifts to the red light while ignoring the green light with the maximum intensity. When different V B (V B1 , V B2 , V B3 ) are applied to the branches, the gain of the red branch is greater than those of the blue and green branches, thus forcing the red branch to receive extra attention. In this case, even though the green light is stronger than the red light, the chip still focuses on the red light unless the green light is strong enough to offset the gain of V B3 to the red light.
form TIAs, and the gain of the TIA is adjusted by changing the feedback resistance R F . To assure the color feature selection chip work properly and meet the input voltage requirements of ASC, whose input common mode range (ICMR) is about 3 V, the output of the TIAs ranges from 0V to 3V by varying the R F . The experimental setup to measure the optical characteristics is illustrated in Fig. 4(d). An RGB-LED is used as the experimental light source (peak wavelength 455 nm, 550 nm and 635 nm), and its color and light intensity can be adjusted by changing LED's drive voltage. For example, a dual-color light can be obtained by applying DC voltage to the two control ports of the LED, and the periodical on-off light can be realized by using a square-wave signal as a drive voltage. The optical test link is shown in Fig. 4(e). The color feature selection unit is placed on the right side of the LED, and an optimal distance between LED and photoreceptors of about 10 cm is selected. The light intensity on the surface of photoreceptors is measured by the optical power meter (PM100D, Thorlabs), and the results of the color feature selection unit are displayed on the Digital oscilloscope (DS6104, Rigol).
Steady-state optical recognition measurements: Fig. 5 is the steady-state recognition diagram of the color feature selection unit. The intensities measured on the surface of photoreceptors are shown in Fig. 6(a), and the corresponding outputs are shown in Fig. 6(b). Firstly, the RGB light is set as the strongest incident light at test point 1-3, and V B is fixed at 0.65 V. It can be seen that the unit can recognize the single-color light with maximum intensity and suppress the response of the branches that correspond to the stray light. Then, the dual-color light intensity ratio is set about 1:1 at test point 4-6 and V B is adjusted to 0.55 V. Note that the unit senses the existence of dual-color light and distracts attention to their corresponding branches in real-time.
The corresponding values of the output voltage (V OU T 1 , V OU T 2 , V OU T 3 ) can be transformed into the binary format as the color labels, and the obtained truth table is reported in Fig. 6(c). Since the main purpose of this design is to implement the selection of principal color-component features, so the special situation (R = 1, G = 1, and B = 1) is not considered. Moreover, the color feature selection unit can suppress the redundant signals and interference caused by stray light, thus reducing the negative impact on color feature selection caused by the spectral overlap of the photoreceptors [29] and the difficulties of subsequent signal processing.
To further analyze the shifting-attention of the color feature selection unit when the light intensity changes, the red light with fixed intensity and on-off green light are selected for further experiments. The blue light as the background light, whose intensity is relatively weak, would not influence the results. The light intensities and the corresponding recognition results are plotted in Fig. 6(d). By increasing the intensity of green light, the ratio of green-to-red light intensity gradually rises from 2:15 to 20:15. It can be seen that the color feature selection unit can adjust its attention to cope with the change of light intensity, and the recognition results consist with that of the ASC (Fig. 3(d)). At test point 16, the unit can recognize the percentage ratio between the intensity of stray light relative to the attentional focus is up to 93.58% (the output voltage of stray light is 10% lower than that of the strongest light), indicating the unit has excellent light intensity sensitivity and anti-interference ability. That is, the minimum recognition resolution is equivalent to 4-bit ADC. In particular, the sensitivity can be adjusted by V B to satisfy different requirements.
Color pattern recognition measurements: An experiment was carried out to demonstrate color pattern recognition in image processing. As shown in Fig. 7(a), the input map is a 12 × 12 pixel array. Each pixel represents the LED luminous color and is proportional to the light intensity. The driving voltage of the LED is configured according to the input map, and the color recognition results are recorded in the recognition map. The recognition map with single attentional focus is shown in Fig. 7(b). The unit can recognize the single-color light with maximum intensity in each pixel. The results are in good qualitative agreement with the color information of the input map, and readers will see "18" if they focus on green and red, respectively. The GB and RB recognition maps with double attentional focus are shown in Fig. 7(c) and 7(d). The recognition unit distracts the attention to the red and green light in the pixels with approximate equivalent intensity and evokes the response of both branches simultaneously. It is obvious which one has higher light intensity when compared to the recognition map of a single attentional focus. Combining the GB and RB recognition maps of the double attentional focus, the RGBY recognition map with double attentional focus is demonstrated in Fig. 7(e). The yellow pixel represents that the attention is drawn to the red and green light. The unit can recognize the input map more accurately and comprehensively, and a pattern of "is" can be obtained due to the additional focus. In practical applications, the recognition sensitivity can be adjusted to meet different functional requirements by changing V B . Unlike the traditional pattern recognition systems [30], the proposed color feature selection unit does not require relevant processing such as grayscale and retains the image's color information. More importantly, the unit can directly perform color feature selection at the data position without additional data processing and burdensome training, thereby overcoming the von Neumann bottleneck.
Dynamic optical recognition measurements: The dynamic recognition measurements are carried out to evaluate the dynamic optical characteristics and the sensitivity of the attention. The schematic diagram of dynamic recognition is shown in Fig. 8. The measurement setup is similar to that of the steady-state test. The only difference is that LED is driven by a periodic square wave voltage to generate on-off keying signal. When the driven voltage is high, the RGB light intensity ratio is 15:17:8. Conversely, the intensity ratio is 15:9:8 with low-level driven voltage. The green and red light alternately became the single-color light with maximum intensity, and the blue light as background light does not contribute to the recognition process. V B is fixed at 0.65 V during this measurement. The driving voltages and the recognition results with the green light at different frequencies are illustrated in Fig. 9(a) and Fig. 9(b). It can be observed from the results that the unit can rapidly switch its attention to the light with the maximum light intensity in any period. The maximum achievable recognition frame rate is 1 kHz. It is confirmed that the color feature selection unit has a strong anti-interference ability in capturing the dynamic change of incident light, even though the stray light has a relatively strong intensity. Furthermore, freely combined optical signals in different frequencies and light intensities (15:15:4) are fed into the recognition unit with a fixed V B of 0.55 V. Fig. 9(c) demonstrates the driving voltage and recognition results. The unit can respond to periodical on-off light and quickly shift its attention to the branch whose photoreceptor exhibits the most intensive response. And when the red and green light are simultaneously illuminated onto the photoreceptors, the unit can distract the attention to the respective branches. When the results are thoroughly examined, it is not difficult to conclude that the unit's attentional set-shifting and distraction can be implemented in real-time. The  unit can encode the color information into the peak amplitude and frequency of the output signals, enabling real-time optical communication. Moreover, the output signal period follows that of the LED driving voltage without signal cross-talk caused by the spectral overlap of photoreceptors, and the output signals are compatible with subsequent processing circuits and algorithms, thus reducing the design complexity of the whole system.
Note that, the color recognition unit has an inherent delay of 0.5 ms. It may be due to the slow carrier extraction speed of ASC, which results in the inability of the unit to complete the state transition and limits the recognition speeds. Several novel technologies have been proposed to develop the artificial neuromorphic visual system, as listed in Table I. The dynamic recognition speed is an order of magnitude faster than biological vision systems and two orders of magnitude higher than current artificial vision systems. Moreover, the unit based on standard CMOS-compatible processes has significant advantages in terms of low power consumption, anti-interference, and low hardware complexity compared with several visual systems fabricated in advanced manufacturing processes.

IV. CONCLUSION
In this paper, a color feature selection unit consisting of photoreceptors and an ASC was designed. The experiments have shown that the ASC can simulate the brain's visual attentional selection that shifts attention to the most significant color feature in real-time or pay special attention to non-significant color feature by applying a top-down stimulus. The steady-state test results show that the unit can recognize single-color and dual-color light, and the percentage ratio between the intensity of stray light relative to the strongest light is up to 93.58%. Experiments related to images processing proved that the unit can be used to recognize color patterns and satisfy different color resolution requirements by adjusting the number of attentional focuses and circuit resolution. The dynamic test results show that the unit has a high resolution to periodical on-off visible light, and the maximum achievable frame rate is 1 kHz, which is an order of magnitude faster than biological vision systems. In conclusion, the unit can filter the interference of photoreceptor's spectral overlap and overcomes the von Neumann bottleneck without memory requirements. In addition, it closes the branches outside of the attentional focus, greatly reducing the overall power consumption. Moreover, the recognition unit of RGB light presented in this paper can be easily expanded to more distinct colors and attentional focuses in accordance with the color resolution requirements, which increases performance over existing computer vision feature extraction approaches, providing new insights for the development of artificial neuromorphic visual systems to simulate the initiative, selectivity, and adaptability of biological vision systems.