A Thermodynamic Treatment of Intelligent Systems

The focus of the computing industry continues to shift towards designing and building intelligent systems that can handle and learn from large amounts of data. The availability of powerful processing hardware like GPUs and TPUs, has powered the tremendous success of many sophisticated resource intensive machine learning algorithms. However as device scaling and energy dissipation fast approach the physical limits, we have started to look away from traditional CMOS devices and the von Neumann architecture, and towards neuromorphic and other bio-inspired computing systems built using novel devices such as memristors and photonics. Biological systems like the human brain are a product of millions of years of bottom-up self- organizing evolutionary processes and often operate near the limits of energy efficiency. Improved understanding of these systems, their capabilities and the self- organization processes that produce them from a computational perspective will go a long way in helping us build better learning machines. The use of non- equilibrium thermodynamics and information theory to understand such biological processes have gained significant momentum in recent years. In this paper, based on a prediction focused definition of intelligence, results characterizing the thermodynamic constraints on finite state automata models of intelligent physical systems will be presented. As we seek to reboot computing, these constraints can prove be to extremely beneficial in guiding the design and fabrication of energy efficient intelligent systems.