A 55nm time-domain mixed-signal neuromorphic accelerator with stochastic synapses and embedded reinforcement learning for autonomous micro-robots

Even as rapid advances are being made in the areas of deep neural networks (DNNs) and convolutional neural networks (CNNs) with most hardware demonstrations geared towards inference in vision-based platforms [1-5], we recognize that true autonomy in intelligent agents will only emerge when such bio-mimetic systems can perform continuous learning through interactions with the environment. Reinforcement learning (RL) presents one such computational paradigm inspired by behaviorist psychology, where autonomous agents take actions in an environment to maximize a notion of cumulative reward. This concept is deeply rooted in the human brain where dopamine mediated neurotransmitters (in the cortex, striatum and thalamus of the brain) have been shown to encourage reward-motivated behavior in all our social interactions (Fig. 7.4.1). In this paper, we present a 690μW (V
_CC
=1.2V) neuromorphic accelerator fabricated in 55nm CMOS, which: (1) inherits unique properties of stochastic neural networks, (2) leverages recent advances in Q-learning as an implementation of RL, and (3) demonstrates energy-efficient time-domain mixed-signal (TD-MS) circuit architectures, to provide autonomy to a mobile, self-driving micro-robot at the edge of the cloud, with possible applications in disaster relief, reconnaissance and personal robotics.